Skip to content

khiops.core.dictionary

Submodule of khiops.core

Classes to manipulate Khiops Dictionary files

Note

To have a complete illustration of the access to the information of all classes in this module look at their write methods which write them in Khiops Dictionary file format (.kdic).

CLASS DESCRIPTION
DictionaryDomain

Main class containing the information of a Khiops dictionary file

Dictionary

A Khiops Dictionary

Variable

A variable of a Khiops dictionary

VariableBlock

A variable block of a Khiops dictionary

Rule

A rule of a variable or variable block in a Khiops dictionary

MetaData

A metadata container for a dictionary, a variable or variable block

FUNCTION DESCRIPTION
read_dictionary_file

Reads a Khiops dictionary file

upper_scope

Applies the upper-scope operator . to an operand

Classes

DictionaryDomain(json_data=None)

Main class containing the information of a Khiops dictionary file

A DictionaryDomain is a collection of Dictionary objects. These dictionaries usually represent either a database schema or a predictor model.

PARAMETER DESCRIPTION
json_data

Python dictionary representing the data of a Khiops Dictionary JSON file. If not specified it returns an empty instance.

Note

Prefer the read_dictionary_file function from the core API to obtain an instance of this class from a Khiops Dictionary file (kdic or kdicj).

TYPE: dict DEFAULT: None

ATTRIBUTE DESCRIPTION
tool

Name of the Khiops tool that generated the dictionary file.

TYPE: str

version

Version of the Khiops tool that generated the dictionary file.

TYPE: str

dictionaries

The domain's dictionaries.

TYPE: list of Dictionary

See class docstring

METHOD DESCRIPTION
__repr__

Returns a human readable string representation

copy

Copies this domain instance

get_dictionary

Returns the specified dictionary

add_dictionary

Adds a dictionary to this domain

remove_dictionary

Removes a dictionary from the domain

extract_data_paths

Extracts the data paths for a dictionary in a multi-table schema

get_dictionary_at_data_path

Returns the dictionary name for the specified data path

export_khiops_dictionary_file

Exports the domain in .kdic format

write

Writes the domain to a file writer in .kdic format

Methods:

__repr__()

Returns a human readable string representation

copy()

Copies this domain instance

RETURNS DESCRIPTION
DictionaryDomain

A copy of this instance.

get_dictionary(dictionary_name)

Returns the specified dictionary

PARAMETER DESCRIPTION
dictionary_name

Name of the dictionary.

TYPE: str

RETURNS DESCRIPTION
Dictionary

The specified dictionary. None is returned if the dictionary name is not found.

add_dictionary(dictionary)

Adds a dictionary to this domain

PARAMETER DESCRIPTION
dictionary

The dictionary to be added.

TYPE: Dictionary

RAISES DESCRIPTION
TypeError

If dictionary is not of type Dictionary.

remove_dictionary(dictionary_name)

Removes a dictionary from the domain

RETURNS DESCRIPTION
Dictionary

The removed dictionary.

RAISES DESCRIPTION
KeyError

If no dictionary with the specified name exists.

extract_data_paths(source_dictionary_name)

Extracts the data paths for a dictionary in a multi-table schema

See Multi-Table Learning Primer for more details about data paths.

PARAMETER DESCRIPTION
source_dictionary_name

Name of a dictionary.

TYPE: str

RETURNS DESCRIPTION
list of str

The additional data paths for the secondary tables of the specified dictionary.

get_dictionary_at_data_path(data_path)

Returns the dictionary name for the specified data path

PARAMETER DESCRIPTION
data_path

A data path for the specified table. Usually the output of extract_data_paths.

TYPE: str

RETURNS DESCRIPTION
Dictionary

The dictionary object pointed by this data path.

RAISES DESCRIPTION
ValueError

If the path is not found.

export_khiops_dictionary_file(kdic_file_path)

Exports the domain in .kdic format

PARAMETER DESCRIPTION
kdic_file_path

Path of the output dictionary file (.kdic).

TYPE: str

write(stream_or_writer)

Writes the domain to a file writer in .kdic format

PARAMETER DESCRIPTION
stream_or_writer

Output stream or writer.

TYPE: IOBase or KhiopsOutputWriter

Dictionary(json_data=None)

A Khiops Dictionary

A Khiops Dictionary is a description of a table transformation. Common uses in the Khiops framework are :

  • Describing the schema of an input table: In this case it is the identity transformation of the table(s).
  • Describing a predictor (classifier or regressor): In this case it is the transformation between the original table(s) and the prediction values or probabilities.
PARAMETER DESCRIPTION
json_data

Python dictionary representing an element of the list at the dictionaries field of a Khiops Dictionary JSON file. If not specified returns an empty instance.

TYPE: dict DEFAULT: None

ATTRIBUTE DESCRIPTION
name

Dictionary name.

TYPE: str

root

True if the dictionary is the root of an dictionary hierarchy.

TYPE: bool

key

Names of the key variables.

TYPE: list of str

variables

The dictionary variables.

TYPE: list of Variable

variable_blocks

The dictionary variable blocks.

TYPE: list of VariableBlock

label

Dictionary label.

TYPE: str

comments

List of dictionary comments.

TYPE: list of str

internal_comments

List of internal dictionary comments.

TYPE: list of str

meta_data

MetaData object of the dictionary.

TYPE: MetaData

See class docstring

METHOD DESCRIPTION
__repr__

Returns a human readable string representation

copy

Returns a copy of this instance

get_value

Returns the metadata value associated to the specified key

use_all_variables

Sets the used flag of all dictionary variables to the specified value

get_variable

Returns the specified variable

get_variable_block

Returns the specified variable block

add_variable

Adds a variable to this dictionary

add_variable_from_spec

Adds a variable to this dictionary using a complete specification

remove_variable

Removes the specified variable from this dictionary

add_variable_block

Adds a variable block to this dictionary

remove_variable_block

Removes the specified variable block from this dictionary

is_key_variable

Returns True if a variable belongs to this dictionary's key

write

Writes the dictionary to a file writer in .kdic format

Methods:

__repr__()

Returns a human readable string representation

copy()

Returns a copy of this instance

RETURNS DESCRIPTION
Dictionary

A copy of this instance.

get_value(key)

Returns the metadata value associated to the specified key

RETURNS DESCRIPTION
MetaData

Metadata value associated to the specified key. None is returned if the metadata key is not found.

use_all_variables(is_used)

Sets the used flag of all dictionary variables to the specified value

PARAMETER DESCRIPTION
is_used

Sets the used field to is_used for all the Variable objects in this dictionary.

TYPE: bool

get_variable(variable_name)

Returns the specified variable

PARAMETER DESCRIPTION
variable_name

A name of a variable.

TYPE: str

RETURNS DESCRIPTION
Variable

The specified variable. None is returned if the variable name is not found.

get_variable_block(variable_block_name)

Returns the specified variable block

PARAMETER DESCRIPTION
variable_block_name

A name of a variable block.

TYPE: str

RETURNS DESCRIPTION
VariableBlock

The specified variable block. None is returned if the variable block name is not found.

add_variable(variable)

Adds a variable to this dictionary

PARAMETER DESCRIPTION
variable

The variable to be added.

TYPE: Variable

RAISES DESCRIPTION
TypeError

If variable is not of type Variable

ValueError

If the name is empty or if there is already a variable with that name.

add_variable_from_spec(name, type, label='', used=True, object_type=None, structure_type=None, rule=None, meta_data=None)

Adds a variable to this dictionary using a complete specification

PARAMETER DESCRIPTION
name

Variable name.

TYPE: str

type

Variable type. See Variable.

TYPE: str

label

Label of the variable.

TYPE: str DEFAULT: ""

used

Usage status of the variable.

TYPE: bool DEFAULT: `True`

object_type

Object type. Ignored if variable type not in ["Entity", "Table"].

TYPE: str DEFAULT: None

structure_type

Structure type. Ignored if variable type is not "Structure".

TYPE: str DEFAULT: None

rule

String representation of a variable rule.

TYPE: str DEFAULT: None

meta_data

A Python dictionary which holds the metadata specification. The dictionary keys are str. The values can be str, bool, float or int.

TYPE: dict DEFAULT: None

RAISES DESCRIPTION
ValueError
  • If the variable name is empty or does not comply with the formatting constraints.
  • If there is already a variable with the same name.
  • If the given variable type is unknown.
  • If a native type is given 'object_type' or 'structure_type'.
  • If the 'meta_data' is not a dictionary.
remove_variable(variable_name)

Removes the specified variable from this dictionary

PARAMETER DESCRIPTION
variable_name

Name of the variable to be removed.

TYPE: str

RETURNS DESCRIPTION
Variable

The removed variable.

RAISES DESCRIPTION
KeyError

If no variable with the specified name exists.

add_variable_block(variable_block)

Adds a variable block to this dictionary

PARAMETER DESCRIPTION
variable_block

The variable block to be added.

TYPE: VariableBlock

RAISES DESCRIPTION
TypeError

If variable is not of type VariableBlock

ValueError

If the name is empty or if there is already a variable block with that name.

remove_variable_block(variable_block_name, keep_native_block_variables=True)

Removes the specified variable block from this dictionary

Note

Non-native block variables (those created from block rules) are never kept in the dictionary.

PARAMETER DESCRIPTION
variable_block_name

Name of the variable block to be removed.

TYPE: str

keep_native_block_variables

If True and the block is native then only the block structure is removed from the dictionary but the variables are kept in it; neither the variables point to the block nor the removed block points to the variables. If False the variables are removed from the dictionary; the block preserves the references to their variables.

TYPE: bool DEFAULT: `True`

RETURNS DESCRIPTION
VariableBlock

The removed variable block.

RAISES DESCRIPTION
KeyError

If no variable block with the specified name exists.

is_key_variable(variable)

Returns True if a variable belongs to this dictionary's key

PARAMETER DESCRIPTION
variable

The variable for the query.

TYPE: Variable

RETURNS DESCRIPTION
bool

True if the variable belong to the key.

write(writer)

Writes the dictionary to a file writer in .kdic format

PARAMETER DESCRIPTION
writer

Output dictionary file.

TYPE: KhiopsOutputWriter

Variable(json_data=None)

A variable of a Khiops dictionary

PARAMETER DESCRIPTION
json_data

Python dictionary representing an element of the list at the variables field of dictionaries found in a Khiops Dictionary JSON file. If not specified it returns an empty instance.

TYPE: dict DEFAULT: None

ATTRIBUTE DESCRIPTION
name

Variable name.

TYPE: str

used

True if the variable is used.

TYPE: bool

type

Variable type. It can be either native (Categorical, Numerical, Time, Date, Timestamp, TimestampTZ, Text), internal (TextList, Structure)

- See https://khiops.org/11.0.0-b.0/api-docs/kdic/text-list-rules/
- See https://khiops.org/11.0.0-b.0/api-docs/kdic/structures-introduction/

or relational (Entity - 0-1 relationship, Table - 0-n relationship)

- See https://khiops.org/11.0.0-b.0/tutorials/kdic_multi_table/

TYPE: str

object_type

Type complement for the Table and Entity types.

TYPE: str

structure_type

Type complement for the Structure type. Set to "" for other types.

TYPE: str

rule

Derivation rule or external table reference. Set to "" if there is no rule associated to this variable. Examples:

- standard rule: "Sum(Var1, Var2)"
- reference rule: "[TableName]"

TYPE: str

variable_block

Block to which the variable belongs. Not set if the variable does not belong to a block.

TYPE: VariableBlock

label

Variable label.

TYPE: str

comments

List of variable comments.

TYPE: list of str

meta_data

Variable metadata.

TYPE: MetaData

Examples:

See the following function of the samples.py documentation script: - samples.create_dictionary_domain()

See class docstring

METHOD DESCRIPTION
__repr__

Returns a human-readable string representation

copy

Copies this variable instance

get_value

Returns the metadata value associated to the specified key

is_native

Returns True if the variable comes directly from a data column

is_relational

Returns True if the variable is of relational type

is_reference_rule

Returns True if the special reference rule is used

full_type

Returns the variable's full type

write

Writes the domain to a file writer in .kdic format

Methods:

__repr__()

Returns a human-readable string representation

copy()

Copies this variable instance

RETURNS DESCRIPTION
Variable

A copy of this instance.

get_value(key)

Returns the metadata value associated to the specified key

RETURNS DESCRIPTION
MetaData

Metadata value associated to the specified key. None is returned if the metadata key is not found.

is_native()

Returns True if the variable comes directly from a data column

Variables are not native if they come from a derivation rule, an external entity, a sub-table or structures.

RETURNS DESCRIPTION
bool

True if a variables comes directly from a data column.

is_relational()

Returns True if the variable is of relational type

Relational variables reference other tables or external entities.

RETURNS DESCRIPTION
bool

True if the variable is of relational type.

is_reference_rule()

Returns True if the special reference rule is used

The reference rule is used to make reference to an external entity.

RETURNS DESCRIPTION
bool

True if the special reference rule is used.

full_type()

Returns the variable's full type

RETURNS DESCRIPTION
str

The full type is the variable type plus its complement if the type is not basic.

write(writer)

Writes the domain to a file writer in .kdic format

PARAMETER DESCRIPTION
writer

Output writer.

TYPE: KhiopsOutputWriter

VariableBlock(json_data=None)

A variable block of a Khiops dictionary

PARAMETER DESCRIPTION
json_data

Python dictionary representing an element of the list at the variables field of a dictionary object in a Khiops Dictionary JSON file. The element must have a blockName field. If not specified it returns an empty instance.

TYPE: dict DEFAULT: None

ATTRIBUTE DESCRIPTION
name

Block name.

TYPE: str

rule

Block derivation rule.

variables

List of the Variable objects of the block.

label

Block label.

TYPE: str

comments

List of block comments.

TYPE: list of str

internal_comments

List of internal block comments.

TYPE: list of str

meta_data

Metadata object of the block.

See class docstring

METHOD DESCRIPTION
__repr__

Returns a human readable string representation

add_variable

Adds a variable to this block

remove_variable

Removes a variable from this block

get_value

Returns the metadata value associated to the specified key

write

Writes the variable block to a file writer in .kdic format

Methods:

__repr__()

Returns a human readable string representation

add_variable(variable)

Adds a variable to this block

PARAMETER DESCRIPTION
variable

The variable to be added.

TYPE: Variable

RAISES DESCRIPTION
TypeError

If the variable is not of type Variable.

remove_variable(variable)

Removes a variable from this block

PARAMETER DESCRIPTION
variable

The variable to be removed.

TYPE: Variable

RAISES DESCRIPTION
TypeError

If the variable is not of type Variable.

get_value(key)

Returns the metadata value associated to the specified key

RETURNS DESCRIPTION
MetaData

Metadata value associated to the specified key. None is returned if the metadata key is not found.

write(writer)

Writes the variable block to a file writer in .kdic format

PARAMETER DESCRIPTION
writer

Output writer.

TYPE: KhiopsOutputWriter

Rule(*name_and_operands, verbatim=None, is_reference=False)

A rule of a variable or variable block in a Khiops dictionary

This object is a convenience feature which eases rule creation and serialization, especially in complex cases (rule operands which are variables or rules themselves, sometimes upper-scoped). A Rule instance must be converted to str before setting it in a Variable or VariableBlock instance.

Rule instances can be created either from full operand specifications, or from verbatim rules. The latter is useful when the rule is retrieved from an existing variable or variable block and is used as an operand in another rule.

PARAMETER DESCRIPTION
name_and_operands

Each tuple member can have one of the following types:

- str
- bytes
- int
- float
- [Variable][]
- [Rule][]
- upper-scoped [Variable][]
- upper-scoped [Rule][]

The first element of the name_and_operands tuple is the name of the rule and must be str or bytes and non-empty for a standard rule, i.e. if is_reference is not set.

TYPE: tuple DEFAULT: ()

verbatim

Verbatim representation of an entire rule. If set, then names_and_operands must be empty.

TYPE: str or bytes DEFAULT: None

is_reference

If set to True, then the rule is serialized as a reference rule: Rule(Operand1, Operand2, ...) is serialized as [Operand1, Operand2, ...].

TYPE: bool DEFAULT: `False`

ATTRIBUTE DESCRIPTION
name

Name of the rule. It is None for reference rules.

TYPE: str or bytes or `None`

operands

Each operand has one of the following types:

- str
- bytes
- int
- float
- [Variable][]
- [Rule][]
- upper-scoped [Variable][]
- upper-scoped [Rule][]

TYPE: tuple of operands

is_reference

The reference status of the rule.

Note

This attribute cannot be changed on a Rule instance.

TYPE: bool

Examples:

  • basic rule, with variables as operands:

    • verbatim:

      Product(PetalLength, PetalWidth)
      
    • object construction:

      petal_length_var = kh.Variable()
      petal_length_var.name = "PetalLength"
      petal_length_var.type = "Numerical"
      petal_width_var = kh.Variable()
      petal_width_var.name = "PetalWidth"
      petal_width_var.type = "Numerical"
      rule = kh.Rule("Product", petal_length_var, petal_width_var)
      
  • multi-table rule:

    • verbatim:

      TableCount(
          TableSelection(
              Vehicles,
              EQ(PassengerNumber, 1)
          )
      )
      
    • object construction:

      vehicles_var = accidents_dictionary.get_variable("Vehicles")
      passenger_number_var = vehicles_dictionary.get_variable(
          "PassengerNumber"
      )
      rule = kh.Rule(
          "TableCount",
          kh.Rule(
              "TableSelection",
              vehicles_var,
              kh.Rule("EQ", passenger_number_var, 1)
          )
      )
      
  • multi-table rule with upper-scoped operands (advanced usage):

    • verbatim:

      TableSelection(
          Vehicles,
          EQ(
              PassengerNumber,
              .TableMax(Vehicles, PassengerNumber)
          )
      )
      
    • object construction:

      vehicles_var = accidents_dictionary.get_variable("Vehicles")
      passenger_number_var = vehicles_dictionary.get_variable(
          "PassengerNumber"
      )
      rule = kh.Rule(
          "TableSelection",
          vehicles_var,
          kh.Rule(
              "EQ",
              passenger_number_var,
              kh.upper_scope(
                  kh.Rule(
                      "TableMax",
                      vehicle_var,
                      passenger_number_var
                  )
              )
          )
      )
      

See class docstring

METHOD DESCRIPTION
copy

Copies this rule instance

write

Writes the rule to a file writer in the .kdic format

Methods:

copy()

Copies this rule instance

RETURNS DESCRIPTION
Rule

A copy of this instance.

write(writer)

Writes the rule to a file writer in the .kdic format

This method ensures proper Rule serialization, automatically handling:

- back-quote recoding in variable names
- double-quote recoding in categorical constants
- missing data (`inf`, `-inf`, `NaN`) serialization as `#Missing`
- upper-scope operator serialization as `.`
PARAMETER DESCRIPTION
writer

Output writer.

Note

self.name is not included in the serialization of reference rules.

TYPE: [KhiopsOutputWriter][]

MetaData(json_data=None)

A metadata container for a dictionary, a variable or variable block

The metadata for both dictionaries and variables is a list of key-value pairs. The values can be set either to a string, to a number, or to the boolean value True. The latter represents flag metadata: they are either present (True) or absent.

PARAMETER DESCRIPTION
json_data

Python dictionary representing the object at a metaData field of a dictionary domain, dictionary or variable in a Khiops Dictionary JSON file. If None it returns an empty instance.

TYPE: dict DEFAULT: None

ATTRIBUTE DESCRIPTION
keys

The metadata keys.

TYPE: list of str

values

Metadata values for each key in keys (synchronized lists). They can be either str, int or float.

TYPE: list

See class docstring

METHOD DESCRIPTION
copy

Copies this metadata instance

is_empty

Returns True if the meta-data is empty

get_value

Returns the value at the specified key

add_value

Adds a value at the specified key

remove_key

Removes the value at the specified key

write

Writes the metadata to a file writer in .kdic format

Methods:

copy()

Copies this metadata instance

RETURNS DESCRIPTION
MetaData

A copy of this instance.

is_empty()

Returns True if the meta-data is empty

RETURNS DESCRIPTION
bool

Returns True if the meta-data is empty

get_value(key)

Returns the value at the specified key

RETURNS DESCRIPTION
(int, str or float)

The value at the specified key. None is returned if the key is not found.

RAISES DESCRIPTION
TypeError

If key is not str.

add_value(key, value)

Adds a value at the specified key

PARAMETER DESCRIPTION
key

Key to be added. A valid key is a sequence of non-accented alphanumeric characters which starts with a non-numeric character.

TYPE: str

value

Value to be added.

TYPE: (bool, int, float or str)

RAISES DESCRIPTION
TypeError
  • If the key is not a valid string
  • If the value is not a valid string or if is not bool, int, float.
ValueError

If the key is already stored.

remove_key(key)

Removes the value at the specified key

PARAMETER DESCRIPTION
key

The key to be removed.

TYPE: str

RETURNS DESCRIPTION
(bool, int, float, str)

The value associated to the key removed.

RAISES DESCRIPTION
TypeError

If the key is not str.

KeyError

If the key is not contained in this metadata.

write(writer)

Writes the metadata to a file writer in .kdic format

PARAMETER DESCRIPTION
writer

Output writer.

TYPE: [KhiopsOutputWriter][]

Functions:

read_dictionary_file(dictionary_file_path)

Reads a Khiops dictionary file

PARAMETER DESCRIPTION
dictionary_file_path

Path of the file to be imported. The file can be either Khiops Dictionary (extension kdic) or Khiops JSON Dictionary (extension .json or .kdicj).

TYPE: str

RETURNS DESCRIPTION
DictionaryDomain

An dictionary domain representing the information in the dictionary file.

RAISES DESCRIPTION
ValueError

When the file has an extension other than .kdic, .kdicj or .json.

Examples:

See the following functions of the samples.py documentation script: - samples.export_dictionary_files() - samples.train_predictor_with_cross_validation() - samples.multiple_train_predictor() - samples.deploy_model_expert()

upper_scope(operand)

Applies the upper-scope operator . to an operand

PARAMETER DESCRIPTION
operand

Operand that is upper-scoped.

TYPE: Variable, Rule, upper-scoped Variable or upper-scoped Rule

RAISES DESCRIPTION
TypeError

If the type of operand is not Variable, Rule, upper-scoped Variable or upper-scoped Rule.

RETURNS DESCRIPTION
upper-scoped operand

The upper-scoped operand, as if the upper-scope operator . were applied to an operand in a rule in the .kdic dictionary language.