Variables in correlators

If tracking values in event fields, active lists, or dictionaries is not enough to cover some specific security scenarios, you can use global and local variables. You can use them to take various actions on the values received by the correlators by implementing complex logic for threat detection. Variables can be declared in the correlator (global variables) or in the correlation rule (local variables) by assigning a function to them, then querying them from correlation rules as if they were ordinary event fields and receiving the triggered function result in response.

Usage scope of variables:

• When searching for identical or unique field values in correlation rules.
• In the correlation rule selectors, in the filters of the conditions under which the correlation rule must be triggered.
• When enriching correlation events. Select Event as the source type.
• When populating active lists with values.

Variables can be queried the same way as event fields by preceding their names with the $ character.

Local variables in identical and unique fields

You can use local variables in the Identical fields and Unique fields sections of 'standard' type correlation rules. To use a local variable, its name must be preceded with the "$" character.

For an example of using local variables in the Identical fields and Unique fields sections, refer to the rule provided with KUMA: R403_Access to malicious resources from a host with disabled protection or an out-of-date anti-virus database.

Local variables in selector

To use a local variable in a selector:

1. Add a local variable to the rule.
2. In the Correlation rules window, go to the General tab and add the created local variable to the Identical fields section. Prefix the local variable name with a "$" character.
3. In Correlation rules window, go to the Selectors tab, select an existing filter or create a new filter and click Add condition.
4. Select the event field as the operand.
5. Select the local variable as the event field value and prefix the variable name with a "$" character.
6. Specify the remaining filter settings.
7. Click Save.

For an example of using local variables, refer to the rule provided with KUMA: R403_Access to malicious resources from a host with disabled protection or an out-of-date anti-virus database.

Local Variables in event enrichment

You can use 'standard' and 'simple' correlation rules to enrich events with local variables.

Enrichment with text and numbers

You can enrich events with text (strings). To do so, you can use functions that modify strings: to_lower, to_upper, str_join, append, prepend, substring, tr, replace, str_join.

You can enrich events with numbers. To do so, you can use the following functions: addition ("+"), subtraction ("-"), multiplication ("*"), division ("/"), round, ceil, floor, abs, pow.

You can also use regular expressions to manage data in local variables.

Using regular expressions in correlation rules is computationally intensive compared to other operations. Therefore, when designing correlation rules, we recommend limiting the use of regular expressions to the necessary minimum and using other available operations.

Timestamp enrichment

You can enrich events with timestamps (date and time). To do so, you can use functions that let you get or modify timestamps: now, extract_from_timestamp, parse_timestamp, format_timestamp, truncate_timestamp, time_diff.

Operations with active lists and tables

You can enrich events with local variables and data from active lists and tables.

To enrich events with data from an active list, use the active_list, active_list_dyn functions.

To enrich events with data from a table, use the table_dict, dict functions.

You can create conditional statements by using the 'conditional' function in local variables. In this way, the variable can return one of the values depending on what data was received for processing.

Enriching events with a local variable

To use a local variable to enrich events:

1. Add a local variable to the rule.
2. In the Correlation rules window, go to the General tab and add the created local variable to the Identical fields section. Prefix the local variable name with a "$" character.
3. In the Correlation rules window, go to the Actions tab, and under Enrichment, in the Source kind drop-down list, select Event.
4. From the Target field drop-down list, select the KUMA event field to which you want to pass the value of the local variable.
5. From the Source field drop-down list, select a local variable. Prefix the local variable name with a "$" character.
6. Specify the remaining rule settings.
7. Click Save.

Local variables in active list enrichment

You can use local variables to enrich active lists.

To enrich the active list with a local variable:

1. Add a local variable to the rule.
2. In the Correlation rules window, go to the General tab and add the created local variable to the Identical fields section. Prefix the local variable name with a "$" character.
3. In the Correlation rules window, go to the Actions tab and under Active lists update, add the local variable to the Key fields field. Prefix the local variable name with a "$" character.
4. Under Mapping, specify the correspondence between the event fields and the active list fields.
5. Click the Save button.

Properties of variables

Local and global variables

The properties of global variables differ from the properties of local variables.

Global variables:

• Global variables are declared at the correlator level and are applied only within the scope of this correlator.
• The global variables of the correlator can be queried from all correlation rules that are specified in it.
• In standard correlation rules, the same global variable can take different values in each selector.
• It is not possible to transfer global variables between different correlators.

Local variables:

• Local variables are declared at the correlation rule level and are applied only within the limits of this rule.
• In standard correlation rules, the scope of a local variable consists of only the selector in which the variable was declared.
• Local variables can be declared in any type of correlation rule.
• Local variables cannot be transferred between rules or selectors.
• A local variable cannot be used as a global variable.

Variables used in various types of correlation rules

• In operational correlation rules, on the Actions tab, you can specify all variables available or declared in this rule.
• In standard correlation rules, on the Actions tab, you can provide only those variables specified in these rules on the General tab, in the Identical fields field.
• In simple correlation rules, on the Actions tab, you can provide only those variables specified in these rules on the General tab, in the Inherited Fields field.

Requirements for variables

When adding a variable function, you must first specify the name of the function, and then list its parameters in parentheses. Basic mathematical operations (addition, subtraction, multiplication, division) are an exception to this requirement. When these operations are used, parentheses are used to designate the severity of the operations.

Requirements for function names:

• Must be unique within the correlator.
• Must contain 1 to 128 Unicode characters.
• Must not begin with the character $.
• Must be written in camelCase or CamelCase.

Special considerations when specifying functions of variables:

• The sequence of parameters is important.
• Parameters are separated by a comma: ,.
• String parameters are passed in single quotes: '.
• Event field names and variables are specified without quotation marks.
• When querying a variable as a parameter, add the $ character before its name.
• You do not need to add a space between parameters.
• In all functions in which a variable can be used as parameters, nested functions can be created.

Functions of variables

Operations with active lists and dictionaries

"active_list" and "active_list_dyn" functions

These functions allow you to receive information from an active list and dynamically generate a field name for an active list and key.

You must specify the parameters in the following sequence:

1. Name of the active list.
2. Expression that returns the field name of the active list.
3. One or more expressions whose results are used to generate the key.

   Usage example	Result
   active_list('Test', to_lower('DeviceHostName'), to_lower(DeviceCustomString2), to_lower(DeviceCustomString1))	Gets the field value of the active list.

Use these functions to query the active list of the shared tenant from a variable. To do so, add the @Shared suffix after the name of the active list (case sensitive). For example, active_list('exampleActiveList@Shared', 'score', SourceAddress, SourceUserName).

"table_dict" function

Gets information about the value in the specified column of a dictionary of the table type.

You must specify the parameters in the following sequence:

1. Dictionary name.
2. Dictionary column name.
3. One or more expressions whose results are used to generate the dictionary row key.

   Usage example	Result
   table_dict('exampleTableDict', 'office', SourceUserName)	Gets data from the exampleTableDict dictionary from the row with the SourceUserName key in the office column.
   table_dict('exampleTableDict', 'office', SourceAddress, to_lower(SourceUserName))	Gets data from the exampleTableDict dictionary from a composite key string from the SourceAddress field value and the lowercase value of the SourceUserName field from the office column.

Use this function to access the dictionary of the shared tenant from a variable. To do so, add the @Shared suffix after the name of the active list (case sensitive). For example, table_dict('exampleTableDict@Shared', 'office', SourceUserName).

"dict" function

Gets information about the value in the specified column of a dictionary of the dictionary type.

You must specify the parameters in the following sequence:

1. Dictionary name.
2. One or more expressions whose results are used to generate the dictionary row key.

   Usage example	Result
   dict('exampleDictionary', SourceAddress)	Gets data from exampleDictionary from the row with the SourceAddress key.
   dict('exampleDictionary', SourceAddress, to_lower(SourceUserName))	Gets data from the exampleDictionary from a composite key string from the SourceAddress field value and the lowercase value of the SourceUserName field.

Use this function to access the dictionary of the shared tenant from a variable. To do so, add the @Shared suffix after the name of the active list (case sensitive). For example, dict('exampleDictionary@Shared', SourceAddress).

Operations with context tables

"context_table" function

Returns the value of the specified field in the base type (for example, integer, array of integers).

You must specify the parameters in the following sequence:

1. Name of the context table. The name must be specified.
2. Expression that returns the field name of context table.
3. Expression that returns the name of key field 1 of the context table.
4. Expression that returns the value of key field 1 of the context table.

The function must contain at least 4 parameters.

"len" function

Returns the length of a string or array.

The function returns the length of the array if the passed array is of one of the following types:

• array of integers
• array of floats
• array of strings
• array of booleans

If an array of a different type is passed, the data of the array is cast to the string type, and the function returns the length of the resulting string.

"distinct_items" function

Returns a list of unique elements in an array.

The function returns the list of unique elements of the array if the passed array is of one of the following types:

• array of integers
• array of floats
• array of strings
• array of booleans

If an array of a different type is passed, the data of the array is cast to the string type, and the function returns a string consisting of the unique characters from the original string.

"sort_items" function

Returns a sorted list of array elements.

You must specify the parameters in the following sequence:

1. Expression that returns the object of the sorting.
2. Sorting order possible values: asc, desc. If the parameter is not specified, the default value is asc.

The function returns the list of sorted elements of the array if the passed array is of one of the following types:

• array of integers
• array of floats
• array of strings

For a boolean array, the function returns the list of array elements in the original order.

If an array of a different type is passed, the data of the array is cast to the string type, and the function returns a string of sorted characters.

"item" function

Returns the array element with the specified index or the character of a string with the specified index if an array of integers, floats, strings, or boolean values is passed.

You must specify the parameters in the following sequence:

1. Expression that returns the object of the indexing.
2. Expression that returns the index of the element or character.

The function must contain at least 2 parameters.

The function returns the array element with the specified index or the string character with the specified index if the index falls within the range of the array and the passed array is of one of the following types:

• array of integers
• array of floats
• array of strings
• array of booleans

If an array of a different type is passed and the index falls within the range of the array, the data is cast to the string type, and the function returns the string character with the specified index. If an array of a different type is passed and the index is outside the range of the array, the function returns an empty string.

Operation with rows

"len" function

Returns the number of characters in a string. Supported for standard fields and extended event schema fields of the "string" type.

A string can be passed as a string, field name or variable.

"to_lower" function

Converts characters in a string to lowercase. Supported for standard fields and extended event schema fields of the "string" type.

A string can be passed as a string, field name or variable.

"to_upper" function

Converts characters in a string to uppercase. Supported for standard fields and extended event schema fields of the "string" type. A string can be passed as a string, field name or variable.

"append" function

Adds characters to the end of a string. Supported for standard fields and extended event schema fields of the "string" type.

You must specify the parameters in the following sequence:

1. Original string.
2. Added string.

Strings can be passed as a string, field name or variable.

"prepend" function

Adds characters to the beginning of a string. Supported for standard fields and extended event schema fields of the "string" type.

You must specify the parameters in the following sequence:

1. Original string.
2. Added string.

Strings can be passed as a string, field name or variable.

"substring" function

Returns a substring from a string. Supported for standard fields and extended event schema fields of the "string" type.

You must specify the parameters in the following sequence:

1. Original string.
2. Substring start position (natural number or 0).
3. (Optional) substring end position.

Strings can be passed as a string, field name or variable. If the position number is greater than the original data string length, an empty string is returned.

"tr" function

Deletes the specified characters from the beginning and end of a string. Supported for standard fields and extended event schema fields of the "string" type.

You must specify the parameters in the following sequence:

1. Original string.
2. (Optional) string that should be removed from the beginning and end of the original string.

Strings can be passed as a string, field name or variable. If you do not specify a string to be deleted, spaces will be removed from the beginning and end of the original string.

"replace" function

Replaces all occurrences of character sequence A in a string with character sequence B. Supported for standard fields and extended event schema fields of the "string" type.

You must specify the parameters in the following sequence:

1. Original string.
2. Search string: sequence of characters to be replaced.
3. Replacement string: sequence of characters to replace the search string.

Strings can be passed as an expression.

"regexp_replace" function

Replaces a sequence of characters that match a regular expression with a sequence of characters and regular expression capturing groups. Supported for standard fields and extended event schema fields of the "string" type.

You must specify the parameters in the following sequence:

1. Original string.
2. Search string: regular expression.
3. Replacement string: sequence of characters to replace the search string, and IDs of the regular expression capturing groups. A string can be passed as an expression.

Strings can be passed as a string, field name or variable. Unnamed capturing groups can be used.

In regular expressions used in variable functions, each backslash character must be additionally escaped. For example, ^example\\\\ must be used instead of the regular expression ^example\\.

"regexp_capture" function

Gets the result matching the regular expression condition from the original string. Supported for standard fields and extended event schema fields of the "string" type.

You must specify the parameters in the following sequence:

1. Original string.
2. Search string: regular expression.

Strings can be passed as a string, field name or variable. Unnamed capturing groups can be used.

In regular expressions used in variable functions, each backslash character must be additionally escaped. For example, ^example\\\\ must be used instead of the regular expression ^example\\.

Operations with timestamps

now function

Gets a timestamp in epoch format. Runs with no arguments.

"extract_from_timestamp" function

Gets atomic time representations (year, month, day, hour, minute, second, day of the week) from fields and variables with time in the epoch format.

The parameters must be specified in the following sequence:

1. Event field of the timestamp type, or variable.
2. Notation of the atomic time representation. This parameter is case sensitive.

   Possible variants of atomic time notation:

   • y refers to the year in number format.
   • M refers to the month in number notation.
   • d refers to the number of the month.
   • wd refers to the day of the week: Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday.
   • h refers to the hour in 24-hour format.
   • m refers to the minutes.
   • s refers to the seconds.
3. (optional) Time zone notation. If this parameter is not specified, the time is calculated in UTC format.

   Usage examples

   extract_from_timestamp(Timestamp, 'wd')

   extract_from_timestamp(Timestamp, 'h')

   extract_from_timestamp($otherVariable, 'h')

   extract_from_timestamp(Timestamp, 'h', 'Europe/Moscow')

"parse_timestamp" function

Converts the time from RFC3339 format (for example, "2022-05-24 00:00:00", "2022-05-24 00:00:00+0300) to epoch format.

"format_timestamp" function

Converts the time from epoch format to RFC3339 format.

The parameters must be specified in the following sequence:

1. Event field of the timestamp type, or variable.
2. Time format notation: RFC3339.
3. (optional) Time zone notation. If this parameter is not specified, the time is calculated in UTC format.

   Usage examples

   format_timestamp(Timestamp, 'RFC3339')

   format_timestamp($otherVariable, 'RFC3339')

   format_timestamp(Timestamp, 'RFC3339', 'Europe/Moscow')

"truncate_timestamp" function

Rounds the time in epoch format. After rounding, the time is returned in epoch format. Time is rounded down.

The parameters must be specified in the following sequence:

1. Event field of the timestamp type, or variable.
2. Rounding parameter:
   • 1s rounds to the nearest second.
   • 1m rounds to the nearest minute.
   • 1h rounds to the nearest hour.
   • 24h rounds to the nearest day.
3. (optional) Time zone notation. If this parameter is not specified, the time is calculated in UTC format.

   Usage examples	Examples of rounded values	Usage result
   truncate_timestamp(Timestamp, '1m')	1654631774175 (7 June 2022, 19:56:14.175)	1654631760000 (7 June 2022, 19:56:00)
   truncate_timestamp($otherVariable, '1h')	1654631774175 (7 June 2022, 19:56:14.175)	1654628400000 (7 June 2022, 19:00:00)
   truncate_timestamp(Timestamp, '24h', 'Europe/Moscow')	1654631774175 (7 June 2022, 19:56:14.175)	1654560000000 (7 June 2022, 0:00:00)

"time_diff" function

Gets the time interval between two timestamps in epoch format.

The parameters must be specified in the following sequence:

1. Interval end time. Event field of the timestamp type, or variable.
2. Interval start time. Event field of the timestamp type, or variable.
3. Time interval notation:
   • ms refers to milliseconds.
   • s refers to seconds.
   • m refers to minutes.
   • h refers to hours.
   • d refers to days.

   Usage examples

   time_diff(EndTime, StartTime, 's')

   time_diff($otherVariable, Timestamp, 'h')

   time_diff(Timestamp, DeviceReceiptTime, 'd')

Mathematical operations

These are comprised of basic mathematical operations and functions.

Basic mathematical operations

Supported for integer and float fields of the extended event schema.

Operations:

• Addition
• Subtraction
• Multiplication
• Division
• Modulo division

Parentheses determine the sequence of actions

Available arguments:

• Numeric event fields
• Numeric variables
• Real numbers

  When modulo dividing, only natural numbers can be used as arguments.

Usage constraints:

• Division by zero returns zero.
• Mathematical operations between numbers and strings return zero.
• Integers resulting from operations are returned without a dot.

  Usage examples (Type=3; otherVariable=2; Message=text)	Usage result
  Type + 1	4
  $otherVariable - Type	-1
  2 * 2.5	5
  2 / 0	0
  Type * Message	0
  (Type + 2) * 2	10
  Type % $otherVariable	1

"round" function

Rounds numbers. Supported for integer and float fields of the extended event schema.

Available arguments:

• Numeric event fields
• Numeric variables
• Numeric constants

  Usage examples (DeviceCustomFloatingPoint1=7.75; DeviceCustomFloatingPoint2=7.5 otherVariable=7.2)	Usage result
  round(DeviceCustomFloatingPoint1)	8
  round(DeviceCustomFloatingPoint2)	8
  round($otherVariable)	7

"ceil" function

Rounds up numbers. Supported for integer and float fields of the extended event schema.

Available arguments:

• Numeric event fields
• Numeric variables
• Numeric constants

  Usage examples (DeviceCustomFloatingPoint1=7.15; otherVariable=8.2)	Usage result
  ceil(DeviceCustomFloatingPoint1)	8
  ceil($otherVariable)	9

"floor" function

Rounds down numbers. Supported for integer and float fields of the extended event schema.

Available arguments:

• Numeric event fields
• Numeric variables
• Numeric constants

  Usage examples (DeviceCustomFloatingPoint1=7.15; otherVariable=8.2)	Usage result
  floor(DeviceCustomFloatingPoint1)	7
  floor($otherVariable)	8

"abs" function

Gets the modulus of a number. Supported for integer and float fields of the extended event schema.

Available arguments:

• Numeric event fields
• Numeric variables
• Numeric constants

  Usage examples (DeviceCustomNumber1=-7; otherVariable=-2)	Usage result
  abs(DeviceCustomFloatingPoint1)	7
  abs($otherVariable)	2

"pow" function

Exponentiates a number. Supported for integer and float fields of the extended event schema.

The parameters must be specified in the following sequence:

1. Base — real numbers.
2. Power — natural numbers.

Available arguments:

• Numeric event fields
• Numeric variables
• Numeric constants

  Usage examples

  pow(DeviceCustomNumber1, DeviceCustomNumber2)

  pow($otherVariable, DeviceCustomNumber1)

"str_join" function

Join multiple strings into one using a separator. Supported for integer and float fields of the extended event schema.

The parameters must be specified in the following sequence:

1. Separator. String.
2. String1, string2, stringN. At least 2 expressions.

   Usage examples	Usage result
   str_join('|', to_lower(Name), to_upper(Name), Name)	String.

"conditional" function

Get one value if a condition is met and another value if the condition is not met. Supported for integer and float fields of the extended event schema.

The parameters must be specified in the following sequence:

1. Condition. String. The syntax is similar to the conditions of the Where statement in SQL. You can use the functions of the KUMA variables and references to other variables in a condition.
2. The value if the condition is met. Expression.
3. The value if the condition is not met. Expression.

Supported operators:

• AND
• OR
• NOT
• =
• !=
• <
• <=
• >
• >=
• LIKE (RE2 regular expression is used, rather than an SQL expression)
• ILIKE (RE2 regular expression is used, rather than an SQL expression)
• BETWEEN
• IN
• IS NULL (check for an empty value, such as 0 or an empty string)

  Usage examples (the value depends on arguments 2 and 3)

  conditional('SourceUserName = \\'root\\' AND DestinationUserName = SourceUserName', 'match', 'no match')

  conditional(`DestinationUserName ILIKE 'svc_.*'`, 'match', 'no match')

  conditional(`DestinationUserName NOT LIKE 'svc_.*'`, 'match', 'no match')

Operations for extended event schema fields

For extended event schema fields of the "string" type, the following kinds of operations are supported:

• "len" function
• "to_lower" function
• "to_upper" function
• "append" function
• "prepend" function
• "substring" function
• "tr" function
• "replace" function
• "regexp_replace" function
• "regexp_capture" function

For extended event schema fields of the integer or float type, the following kinds of mathematical operations are supported:

• Basic mathematical operations:
• "round" function
• "ceil" function
• "floor" function
• "abs" function
• "pow" function
• "str_join" function
• "conditional" function

For extended event schema fields of the "array of numbers", "array of floats", and "array of strings" types, the following kinds of mathematical operations are supported:

• item(SA.someStringArray, i) — gets the i-th element of the someStringArray[i] field.
• SA.someStringArray , returns ["string1", "string2", "string1"] — gets the array of values from the someStringArray field.
• len(SA.someStringArray) — gets the number of elements in the someStringArray array.
• distinct_items(SA.someStringArray), returns ["string1", "string2"] — gets unique elements from the someStringArray array.
• to_string(SA.someStringArray) — generates a TSV string from the array.
• sort_items(<type>.someStringArray); instead of <type>, you must specify the array type: 'sa' for an array of strings, 'fa' for an array of floats, 'na' for an array of integers. Example: sort_items (SA.StringArray, DESC).

For fields of the "array of integers" and "array of floats" types, the following functions are supported:

• math_min — returns the minimum element of an array. Example: math_min(NA.NumberArray), math_min(FA.FloatArray).
• math_max — returns the maximum element of an array. Example: math_max(NA.NumberArray), math_max(FA.FloatArray).
• math_avg — returns the average value of an array. Example: math_avg(NA.NumberArray), math_avg(FA.FloatArray).

Declaring variables

To declare variables, they must be added to a correlator or correlation rule.

To add a global variable to an existing correlator:

1. In the KUMA Console, under Resources → Correlators, select the resource set of the relevant correlator.

   The Correlator Installation Wizard opens.

2. Select the Global variables step of the Installation Wizard.
3. Click the Add variable button and specify the following parameters:
   • In the Variable window, enter the name of the variable.

     Variable naming requirements

     • Must be unique within the correlator.
     • Must contain 1 to 128 Unicode characters.
     • Must not begin with the character $.
     • Must be written in camelCase or CamelCase.
   • In the Value window, enter the variable function.

     Description of variable functions.

   Multiple variables can be added. Added variables can be edited or deleted by using the icon.

4. Select the Setup validation step of the Installation Wizard and click Save.

A global variable is added to the correlator. It can be queried like an event field by inserting the $ character in front of the variable name. The variable will be used for correlation after restarting the correlator service.

To add a local variable to an existing correlation rule:

1. In the KUMA Console, under Resources → Correlation rules, select the relevant correlation rule.

   The correlation rule settings window opens. The parameters of a correlation rule can also be opened from the correlator to which it was added by proceeding to the Correlation step of the Installation Wizard.

2. Open the Selectors tab.
3. In the selector, open the Local variables tab, click the Add variable button and specify the following parameters:
   • In the Variable window, enter the name of the variable.

     Variable naming requirements

     • Must be unique within the correlator.
     • Must contain 1 to 128 Unicode characters.
     • Must not begin with the character $.
     • Must be written in camelCase or CamelCase.
   • In the Value window, enter the variable function.

     Description of variable functions.

   Multiple variables can be added. Added variables can be edited or deleted by using the icon.

   For standard correlation rules, repeat this step for each selector in which you want to declare variables.

4. Click Save.

The local variable is added to the correlation rule. It can be queried like an event field by inserting the $ character in front of the variable name. The variable will be used for correlation after restarting the correlator service.

Added variables can be edited or deleted. If the correlation rule queries an undeclared variable (for example, if its name has been changed), an empty string is returned.

If you change the name of a variable, you will need to manually change the name of this variable in all correlation rules where you have used it.