KUMA resources

Resources are KUMA components that contain parameters for implementing various functions: for example, establishing a connection with a given web address or converting data according to certain rules. Like parts of an erector set, these components are assembled into resource sets for services that are then used as the basis for creating KUMA services.

Resources are contained in the Resources section, Resources block of KUMA Console. The following resource types are available:

• Correlation rules—resources of this type contain rules for identifying event patterns that indicate threats. If the conditions specified in these resources are met, a correlation event is generated.
• Normalizers—resources of this type contain rules for converting incoming events into the format used by KUMA. After processing in the normalizer, the "raw" event becomes normalized and can be processed by other KUMA resources and services.
• Connectors—resources of this type contain settings for establishing network connections.
• Aggregation rules—resources of this type contain rules for combining several basic events of the same type into one aggregation event.
• Enrichment rules—resources of this type contain rules for supplementing events with information from third-party sources.
• Destinations—resources of this type contain settings for forwarding events to a destination for further processing or storage.
• Filters—resources of this type contain conditions for rejecting or selecting individual events from the stream of events.
• Response rules—resources of this type are used in correlators to, for example, execute scripts or launch Kaspersky Security Center tasks when certain conditions are met.
• Notification templates—resources of this type are used when sending notifications about new alerts.
• Active lists—resources of this type are used by correlators for dynamic data processing when analyzing events according to correlation rules.
• Dictionaries—resources of this type are used to store keys and their values, which may be required by other KUMA resources and services.
• Proxies—resources of this type contain settings for using proxy servers.
• Secrets—resources of this type are used to securely store confidential information (such as credentials) that KUMA needs to interact with external services.
• Context tables—resources of this type are used by KUMA correlators for analyzing events in accordance with correlation rules.

When you click on a resource type, a window opens displaying a table with the available resources of this type. The resource table contains the following columns:

• Name—the name of a resource. Can be used to search for resources and sort them.
• Updated—the date and time of the last update of a resource. Can be used to sort resources.
• Created by—the name of the user who created a resource.
• Description—the description of a resource.

The maximum table size is not limited. If you want to select all resources, scroll to the end of the table and select the Select all check box, which selects all available resources in the table.

Resources can be organized into folders. The folder structure is displayed in the left part of the window: root folders correspond to tenants and contain a list of all resources of the tenant. All other folders nested within the root folder display the resources of an individual folder. When a folder is selected, the resources it contains are displayed as a table in the right pane of the window.

Resources can be created, edited, copied, moved from one folder to another, and deleted. Resources can also be exported and imported.

KUMA comes with a set of predefined resources, which can be identified by the "[OOTB]<resource_name>" name. OOTB resources are protected from editing.

If you want to adapt a predefined OOTB resource to your organization's infrastructure:

1. In the Resources-<resource type> section, select the OOTB resource that you want to edit.
2. In the upper part of the KUMA Console, click Duplicate, then click Save.
3. A new resource named "[OOTB]<resource_name> - copy" is displayed in the web interface.
4. Edit the copy of the predefined resource as necessary and save your changes.

The adapted resource is available for use.

Operations with resources

To manage KUMA resources, you can create, move, copy, edit, delete, import, and export them. These operations are available for all resources, regardless of the resource type.

KUMA resources reside in folders. You can add, rename, move, or delete resource folders.

Creating, renaming, moving, and deleting resource folders

Resources can be organized into folders. The folder structure is displayed in the left part of the window: root folders correspond to tenants and contain a list of all resources of the tenant. All other folders nested within the root folder display the resources of an individual folder. When a folder is selected, the resources it contains are displayed as a table in the right pane of the window.

You can create, rename, move and delete folders.

To create a folder:

1. Select the folder in the tree where the new folder is required.
2. Click the Add folder button.

The folder will be created.

To rename a folder:

1. Locate required folder in the folder structure.
2. Hover over the name of the folder.

   The icon will appear near the name of the folder.

3. Open the drop-down list and select Rename.

   The folder name will become active for editing.

4. Enter the new folder name and press ENTER.

   The folder name cannot be empty.

The folder will be renamed.

To move a folder,

Drag and drop the folder to a required place in folder structure by clicking its name.

Folders cannot be dragged from one tenant to another.

To delete a folder:

1. Locate required folder in the folder structure.
2. Hover over the name of the folder.

   The icon will appear near the name of the folder.

3. Open the drop-down list and select Delete.

   The conformation window appears.

4. Click OK.

The folder will be deleted.

The program does not delete folders that contain files or subfolders.

Creating, duplicating, moving, editing, and deleting resources

You can create, move, copy, edit, and delete resources.

To create the resource:

1. In the Resources → <resource type> section, select or create a folder where you want to add the new resource.

   Root folders correspond to tenants. For a resource to be available to a specific tenant, it must be created in the folder of that tenant.

2. Click the Add <resource type> button.

   The window for configuring the selected resource type opens. The available configuration parameters depend on the resource type.

3. Enter a unique resource name in the Name field.
4. Specify the required parameters (marked with a red asterisk).
5. If necessary, specify the optional parameters (not required).
6. Click Save.

The resource will be created and available for use in services and other resources.

To move the resource to a new folder:

1. In the Resources → <resource type> section, find the required resource in the folder structure.
2. Select the check box near the resource you want to move. You can select multiple resources.

   The icon appears near the selected resources.

3. Use the icon to drag and drop resources to the required folder.

The resources will be moved to the new folders.

You can only move resources to folders of the tenant in which the resources were created. Resources cannot be moved to another tenant's folders.

To copy the resource:

1. In the Resources → <resource type> section, find the required resource in the folder structure.
2. Select the check box next to the resource that you want to copy and click Duplicate.

   A window opens with the settings of the resource that you have selected for copying. The available configuration parameters depend on the resource type.

   The <selected resource name> - copy value is displayed in the Name field.

3. Make the necessary changes to the parameters.
4. Enter a unique name in the Name field.
5. Click Save.

The copy of the resource will be created.

To edit the resource:

1. In the Resources → <resource type> section, find the required resource in the folder structure.
2. Select the resource.

   A window with the settings of the selected resource opens. The available configuration parameters depend on the resource type.

3. Make the necessary changes to the parameters.
4. Click Save.

The resource will be updated. If this resource is used in a service, restart the service to apply the new settings.

To delete the resource:

1. In the Resources → <resource type> section, find the required resource in the folder structure.
2. Select the check box next to the resource that you want to delete and click Delete.

   A confirmation window opens.

3. Click OK.

The resource will be deleted.

Link correlators to a correlation rule

The Link correlators option is available for the created correlation rules.

To link correlators:

1. In the KUMA Console → Resources → Correlation rules section, select the created correlation rule and click Link correlators.
2. This opens the Correlators window; in that window, select one or more correlators by selecting the check box next to them.
3. Click OK.

Correlators are linked to a correlation rule.

The rule is added to the end of the execution queue in each selected correlator. If you want to move the rule up in the execution queue, go to Resources → Correlators → <selected correlator> → Edit correlator → Correlation, select the check box next to the relevant rule and click the Move up or Move down buttons to reorder the rules as necessary.

Updating resources

Kaspersky regularly releases packages with resources that can be imported from the repository. You can specify an email address in the settings of the Repository update task. After the first execution of the task, KUMA starts sending notifications about the packages available for update to the specified address. You can update the repository, analyze the contents of each update, and decide if to import and deploy the new resources in the operating infrastructure. KUMA supports updates from Kaspersky servers and from custom sources, including offline update using the update mirror mechanism. If you have other Kaspersky applications in the infrastructure, you can connect KUMA to existing update mirrors. The update subsystem expands KUMA capabilities to respond to the changes in the threat landscape and the infrastructure. The possibility of using it without direct Internet access helps ensure the privacy of the data processed by the system.

To update resources, perform the following steps:

1. Update the repository to deliver the resource packages to the repository. The repository update is available in two modes:
   • Automatic update
   • Manual update
2. Import the resource packages from the updated repository into the tenant.

For the service to start using the resources, make sure that the updated resources are mapped after performing the import. If necessary, link the resources to collectors, correlators, or agents, and update the settings.

To enable automatic update:

1. In the Settings → Repository update section, configure the Data refresh interval in hours. The default value is 24 hours.
2. Specify the Update source. The following options are available:
   • Kaspersky update servers

     

     The servers are located in many countries around the world, which ensures the high reliability of the update. If the update cannot be performed from one server, KUMA switches to another server.

     .

     You can view the list of servers in the Knowledge Base, article 15998.

   • Custom source:
     • The URL to the shared folder on the HTTP server.
     • The full path to the local folder on the host where the KUMA Core is installed.

       If a local folder is used, the kuma system user must have read access to this folder and its contents.

3. Specify the Emails for notification by clicking the Add button. The notifications that new packages or new versions of the packages imported into the tenant are available in the repository are sent to the specified email addresses.

   If you specify the email address of a KUMA user, the Receive email notifications check box must be selected in the user profile. For emails that do not belong to any KUMA user, the messages are received without additional settings. The settings for connecting to the SMTP server must be specified in all cases.

4. Click Save. The update task starts shortly. Then the task restarts according to the schedule.

To manually start the repository update:

1. To disable automatic updates, in the Settings → Repository update section, select the Disable automatic update check box. This check box is cleared by default. You can also start a manual repository update without disabling automatic update. Starting an update manually does not affect the automatic update schedule.
2. Specify the Update source. The following options are available:
   • Kaspersky update servers.
   • Custom source:
     • The URL to the shared folder on the HTTP server.
     • The full path to the local folder on the host where the KUMA Core is installed.

       If a local folder is used, the kuma user must have access to this folder and its contents.

3. Specify the Emails for notification by clicking the Add button. The notifications that new packages or new versions of the packages imported into the tenant are available in the repository are sent to the specified email addresses.

   If you specify the email address of a KUMA user, the Receive email notifications check box must be selected in the user profile. For emails that do not belong to any KUMA user, the messages are received without additional settings. The settings for connecting to the SMTP server must be specified in all cases.

4. Click Run update. Thus, you simultaneously save the settings and manually start the Repository update task.

Configuring a custom source using Kaspersky Update Utility

You can update resources without Internet access by using a custom update source via the Kaspersky Update Utility.

Configuration consists of the following steps:

1. Configuring a custom source using Kaspersky Update Utility:
   1. Installing and configuring Kaspersky Update Utility on one of the computers in the corporate LAN.
   2. Configuring copying of updates to a shared folder in Kaspersky Update Utility settings.
2. Configuring update of the KUMA repository from a custom source.

Configuring a custom source using Kaspersky Update Utility:

You can download the Kaspersky Update Utility distribution kit from the Kaspersky Technical Support website.

1. In Kaspersky Update Utility, enable the download of updates for KUMA:
   • Under Applications – Perimeter control, select the check box next to KUMA to enable the update capability.
   • If you work with Kaspersky Update Utility using the command line, add the following line to the [ComponentSettings] section of the updater.ini configuration file or specify the true value for an existing line:

     KasperskyUnifiedMonitoringAndAnalysisPlatform_XDR_1_1=true

2. In the Downloads section, specify the update source. By default, Kaspersky update servers are used as the update source.
3. In the Downloads section, in the Update folders group of settings, specify the shared folder for Kaspersky Update Utility to download updates to. The following options are available:
   • Specify the local folder on the host where Kaspersky Update Utility is installed. Deploy the HTTP server for distributing updates and publish the local folder on it. In KUMA, in the Settings → Repository update → Custom source section, specify the URL of the local folder published on the HTTP server.
   • Specify the local folder on the host where Kaspersky Update Utility is installed. Make this local folder available over the network. Mount the network-accessible local folder on the host where KUMA is installed. In KUMA, in the Settings → Repository update → Custom source section, specify the full path to the local folder.

For detailed information about working with Kaspersky Update Utility, refer to the Kaspersky Knowledge Base.

Exporting resources

If shared resources are hidden for a user, the user cannot export shared resources or resources that use shared resources.

To export resources:

1. In the Resources section, click Export resources.

   The Export resources window opens with the tree of all available resources.

2. In the Password field enter the password that must be used to protect exported data.
3. In the Tenant drop-down list, select the tenant whose resources you want to export.
4. Check boxes near the resources you want to export.

   If selected resources are linked to other resources, linked resources will be exported, too.

5. Click the Export button.

The resources in a password-protected file are saved on your computer using your browser settings. The Secret resources are exported blank.

Importing resources

To import resources:

1. In the Resources section, click Import resources.

   The Resource import window opens.

2. In the Tenant drop-down list, select the tenant to assign the imported resources to.
3. In the Import source drop-down list, select one of the following options:
   • File

     If you select this option, enter the password and click the Import button.

   • Repository

     If you select this option, a list of packages available for import is displayed. We recommend you to ensure that the repository update date is relatively recent and configure automatic updates if necessary.

     You can select one or more packages to import and click the Import button. The dependent resources of the Shared tenant are imported into the Shared tenant, the rest of the resources are imported into the selected tenant. You do not need special rights for the Shared tenant; you must only have the right to import in the selected tenant.

     The imported resources can only be deleted. To rename, edit or move an imported resource, make a copy of the resource by clicking the Duplicate button and perform the desired actions with the resource copy. When importing future versions of the package, the duplicate is not updated because it is a separate object.

4. Resolve the conflicts between the resources imported from the file and the existing resources if they occur. Read more about resource conflicts below.
   1. If the name, type, and guid of an imported resource fully match the name, type, and guid of an existing resource, the Conflicts window opens with the table displaying the type and the name of the conflicting resources. Resolve displayed conflicts:
      • To replace the existing resource with a new one, click Replace.

        To replace all conflicting resources, click Replace all.

      • To leave the existing resource, click Skip.

        For dependent resources, that is, resources that are associated with other resources, the Skip option is not available; you can only Replace dependent resources.

        To keep all existing resources, click Skip all.

   2. Click the Resolve button.

   The resources are imported to KUMA. The Secret resources are imported blank.

Importing resources that use the extended event schema

If you import a normalizer that uses one or more fields of the extended event schema, KUMA automatically creates an extended schema field that is used in the normalizer.

If you import other types of resources that use fields of the extended event schema in their logic, the resources are imported successfully. To ensure the functioning of imported resources, you must create the corresponding fields of the extended event schema in a resource of the "normalizer" type.

If a normalizer that uses an extended event schema field is imported into KUMA and the same field already exists in KUMA, the previously created field is used.

About conflict resolving

When resources are imported into KUMA from a file, they are compared with existing resources; the following parameters are compared:

• Name and kind. If an imported resource's name and kind parameters match those of the existing one, the imported resource's name is automatically changed.
• ID. If identifiers of two resources match, a conflict appears that must be resolved by the user. This could happen when you import resources to the same KUMA server from which they were exported.

When resolving a conflict you can choose either to replace existing resource with the imported one or to keep exiting resource, skipping the imported one.

Some resources are linked: for example, in some types of connectors, the connector secret must be specified. The secrets are also imported if they are linked to a connector. Such linked resources are exported and imported together.

Special considerations of import:

1. Resources are imported to the selected tenant.
2. If a linked resource was in the Shared tenant, it ends up in the Shared tenant when imported.
3. In the Conflicts window, the Parent column always displays the top-most parent resource among those that were selected during import.
4. If a conflict occurs during import and you choose to replace existing resource with a new one, it would mean that all the other resources linked to the one being replaced are automatically replaced with the imported resources.

Known errors:

1. The linked resource ends up in the tenant specified during the import, and not in the Shared tenant, as indicated in the Conflicts window, under the following conditions:
   1. The associated resource is initially in the Shared tenant.
   2. In the Conflicts window, you select Skip for all parent objects of the linked resource from the Shared tenant.
   3. You leave the linked resource from the Shared tenant for replacement.
2. After importing, the categories do not have a tenant specified in the filter under the following conditions:
   1. The filter contains linked asset categories from different tenants.
   2. Asset category names are the same.
   3. You are importing this filter with linked asset categories to a new server.
3. In Tenant 1, the name of the asset category is duplicated under the following conditions:
   1. in Tenant 1, you have a filter with linked asset categories from Tenant 1 and the Shared tenant.
   2. The names of the linked asset categories are the same.
   3. You are importing such a filter from Tenant 1 to the Shared tenant.
4. You cannot import conflicting resources into the same tenant.

   The error "Unable to import conflicting resources into the same tenant" means that the imported package contains conflicting resources from different tenants and cannot be imported into the Shared tenant.

   Solution: Select a tenant other than Shared to import the package. In this case, during the import, resources originally located in the Shared tenant are imported into the Shared tenant, and resources from the other tenant are imported into the tenant selected during import.

5. Only the Main Administrator can import categories into the Shared tenant.

   The error "Only the Main administrator can import categories into the Shared tenant" means that the imported package contains resources with linked shared asset categories. You can see the categories or resources with linked shared asset categories in the KUMA Core log. Path to the Core log:

   /opt/kaspersky/kuma/core/log/core

   Solution. Choose one of the following options:

   • Do not import resources to which shared categories are linked: clear the check boxes next to the relevant resources.
   • Perform the import under a Main administrator account.
6. Only the Main administrator can import resources into the Shared tenant.

   The error "Only the Main administrator can import resources into the Shared tenant" means that the imported package contains resources with linked shared resources. You can see the resources with linked shared resources in the KUMA Core log. Path to the Core log:

   /opt/kaspersky/kuma/core/log/core

   Solution. Choose one of the following options:

   • Do not import resources that have linked resources from the Shared tenant, and the shared resources themselves: clear the check boxes next to the relevant resources.
   • Perform the import under a Main administrator account.

Destinations

Destinations define network settings for sending normalized events. Collectors and correlators use destinations to describe where to send processed events. Typically, the destination points are the correlator and storage.

The settings of destinations are configured on two tabs: Basic settings and Advanced settings. The available settings depend on the selected type of destination:

• nats-jetstream—used for NATS communications.
• tcp—used for communications over TCP.
• http—used for HTTP communications.
• diode—used to transmit events using a data diode.
• kafka—used for Kafka communications.
• file—used for writing to a file.
• storage—used to transmit data to the storage.
• correlator—used to transmit data to the correlator.

nats-jetstream type

The nats-jetstream type is used for NATS communications.

Basic settings tab

Advanced settings tab

Tcp type

The tcp type is used for TCP communications.

Basic settings tab

Advanced settings tab

Http type

The http type is used for HTTP communications.

Basic settings tab

Advanced settings tab

Diode type

The diode type is used to transmit events using a data diode.

Basic settings tab

Advanced settings tab

Kafka type

The kafka type is used for Kafka communications.

Basic settings tab

Advanced settings tab

File type

The file type is used for writing data to a file.

If you delete a destination of the 'file' type used in a service, that service must be restarted.

Basic settings tab

Advanced settings tab

Storage type

The storage type is used to transmit data to the storage.

Basic settings tab

Advanced settings tab

Correlator type

The correlator type is used to transmit data to the correlator.

Basic settings tab

Advanced settings tab

Predefined destinations

Destinations listed in the table below are included in the OSMP distribution kit.

Predefined destinations

Normalizers

Normalizers are used for converting raw events that come from various sources in different formats to the KUMA event data model. Normalized events become available for processing by other KUMA resources and services.

A normalizer consists of the main event parsing rule and optional additional event parsing rules. By creating a main parsing rule and a set of additional parsing rules, you can implement complex event processing logic. Data is passed along the tree of parsing rules depending on the conditions specified in the Extra normalization conditions setting. The sequence in which parsing rules are created is significant: the event is processed sequentially and the processing sequence is indicated by arrows.

The following event normalization options are now available:

• 1 collector — 1 normalizer

  We recommend using this method if you have many events of the same type or many IP addresses from which events of the same type may originate. You can configure one collector with only one normalizer, which is optimal in terms of performance.

• 1 collector — multiple normalizers linked to IP

  This method is available for collectors with a connector of UDP, TCP, or HTTP type. If a UDP, TCP, or HTTP connector is specified in the collector at the 'Transport' step, then at the 'Event parsing' step, you can specify multiple IP addresses on the 'Parsing settings' tab and choose the normalizer that you want to use for events coming from the specified addresses. The following types of normalizers are available: json, cef, regexp, syslog, csv, kv, xml. For normalizers of the syslog and regexp types, you can specify extra normalization conditions depending on the value of the DeviceProcessName field.

A normalizer is created in several steps:

1. Preparing to create a normalizer

   A normalizer can be created in the KUMA Console:

   • In the Resources → Normalizers section.
   • When creating a collector, at the Event parsing step.

   Then parsing rules must be created in the normalizer.

2. Creating the main parsing rule for an event

   The main parsing rule is created by clicking the Add event parsing button. This opens the Event parsing window, where you can specify the settings of the main parsing rule:

   • Specify event parsing settings.
   • Specify event enrichment settings.

   The main parsing rule for an event is displayed in the normalizer as a dark circle. You can view or modify the settings of the main parsing rule by clicking this circle. When you hover the mouse over the circle, a plus sign is displayed. Click it to add the parsing rules.

   The name of the main parsing rule is used in KUMA as the normalizer name.

3. Creating additional event parsing rules

   Clicking the plus icon that is displayed when you hover the mouse over the circle or the block corresponding to the normalizer opens the Additional event parsing window where you can specify the settings of the additional parsing rule:

   • Specify the conditions for sending data to the new normalizer.
   • Specify event parsing settings.
   • Specify event enrichment settings.

   The additional event parsing rule is displayed in the normalizer as a dark block. The block displays the triggering conditions for the additional parsing rule, the name of the additional parsing rule, and the event field. When this event field is available, the data is passed to the normalizer. Click the block of the additional parsing rule to view or modify its settings.

   If you hover the mouse over the additional normalizer, a plus button appears. You can click this button to create a new additional event parsing rule. To delete a normalizer, click the button with the trash icon.

4. Completing the creation of the normalizer

   To finish the creation of the normalizer, click Save.

In the upper right corner, in the search field, you can search for additional parsing rules by name.

For normalizer resources, you can enable the display of control characters in all input fields except the Description field.

If, when changing the settings of a collector resource set, you change or delete conversions in a normalizer connected to it, the edits will not be saved, and the normalizer itself may be corrupted. If you need to modify conversions in a normalizer that is already part of a service, the changes must be made directly to the normalizer under Resources → Normalizers in the web interface.

Event parsing settings

Expand all | Collapse all

You can configure the rules for converting incoming events to the KUMA format when creating event parsing rules in the normalizer settings window, on the Normalization scheme tab.

To define the event parsing settings:

1. In the Name field (required), enter the unique name of the parsing rule. Must contain 1 to 128 Unicode characters. The name of the main parsing rule is used as the name of the normalizer.
2. In the Tenant field (required), enter the name of the tenant that owns the resource.

   This setting is not available for extra parsing rules.

3. In the Parsing method drop-down list, select the type of events to receive. Depending on your choice, you can use the preconfigured rules for matching event fields or set your own rules. When you select some of the parsing methods, additional settings fields may need to be filled.

   Available parsing methods:

   • json

     This parsing method is used to process JSON data where each object, including its nested objects, occupies a single line in a file.

     When processing files with hierarchically arranged data, you can access the fields of nested objects by specifying the names of the parameters dividing them by a period. For example, the username parameter from the string "user": {"username": "system: node: example-01"} can be accessed by using the user.username query.

     Files are processed line by line. Multi-line objects with nested structures may be normalized incorrectly.

     In complex normalization schemes where additional normalizers are used, all nested objects are processed at the first normalization level, except for cases when the extra normalization conditions are not specified and, therefore, the event being processed is passed to the additional normalizer in its entirety.

     Newline characters can be \n and \r\n. Strings must be UTF-8 encoded.

     If you want to send the raw event for advanced normalization, at each nesting level in the Advanced event parsing window, select Yes in the Keep raw event drop-down list.

   • cef

     This parsing method is used to process CEF data.

     When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button.

   • regexp

     This parsing method is used to create custom rules for processing data in a format using regular expressions.

     In the Normalization parameter block field, add a regular expression (RE2 syntax) with named capture groups. The name of a group and its value will be interpreted as the field and the value of the raw event, which can be converted into an event field in KUMA format.

     To add event handling rules:

     1. Copy an example of the data you want to process to the Event examples field. This is an optional but recommended step.
     2. In the Normalization parameter block field add a regular expression with named capture groups in RE2 syntax, for example "(?P<name>regexp)". The regular expression added to the Normalization parameter must exactly match the event. Also, when developing the regular expression, it is recommended to use special characters that match the starting and ending positions of the text: ^, $.

        You can add multiple regular expressions by clicking the Add regular expression button. If you need to remove the regular expression, click the button.

     3. Click the Copy field names to the mapping table button.

        Capture group names are displayed in the KUMA field column of the Mapping table. Now you can select the corresponding KUMA field in the column next to each capture group. Otherwise, if you named the capture groups in accordance with the CEF format, you can use the automatic CEF mapping by selecting the Use CEF syntax for normalization check box.

     Event handling rules were added.

   • syslog

     This parsing method is used to process data in syslog format.

     When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button.

   • csv

     This parsing method is used to create custom rules for processing CSV data.

     When choosing this method, you must specify the separator of values in the string in the Delimiter field. Any single-byte ASCII character can be used as a delimiter.

   • kv

     This parsing method is used to process data in key-value pair format.

     If you select this method, you must provide values in the following required fields:

     • Pair delimiter—specify a character that will serve as a delimiter for key-value pairs. You can specify any one-character (1 byte) value, provided that the character does not match the value delimiter.
     • Value delimiter—specify a character that will serve as a delimiter between the key and the value. You can specify any one-character (1 byte) value, provided that the character does not match the delimiter of key-value pairs.
   • xml

     This parsing method is used to process XML data in which each object, including its nested objects, occupies a single line in a file. Files are processed line by line.

     If you want to send the raw event for advanced normalization, at each nesting level in the Advanced event parsing window, select Yes in the Keep raw event drop-down list.

     When this method is selected in the parameter block XML attributes you can specify the key attributes to be extracted from tags. If an XML structure has several attributes with different values in the same tag, you can indicate the necessary value by specifying its key in the Source column of the Mapping table.

     To add key XML attributes,

     Click the Add field button, and in the window that appears, specify the path to the required attribute.

     You can add more than one attribute. Attributes can be removed one at a time using the cross icon or all at once by clicking the Reset button.

     If XML key attributes are not specified, then in the course of field mapping the unique path to the XML value will be represented by a sequence of tags.

     Tag numbering

     Tag numbering is available as of KUMA 2.1.3. This functionality allows automatically numbering tags in XML events, which lets you parse an event with identical tags or unnamed tags, such as <Data>.

     As an example, we will use the Tag numbering functionality to number the tags of the EventData attribute of Microsoft Windows PowerShell event ID 800.

     

     To parse such events, you must:

     • Configure tag numbering.
     • Configure data mapping for numbered tags with KUMA event fields.

     KUMA 3.0.x supports using XML attributes and Tag numbering functionality at the same time in the same extra normalizer. If an attribute contains unnamed tags or identical tags, we recommend using the Tag numbering functionality. If the attribute contains only named tags, use XML attributes. To use this functionality in extra normalizers, you must sequentially enable the "Keep raw event" setting in each extra normalizer along the path that the event follows to the target extra normalizer, and in the target extra normalizer itself.

     For an example of this functionality in action, you can refer to the MicrosoftProducts normalizer — the "Keep raw event" setting is enabled sequentially in the "AD FS" and "424" extra normalizers.

     To configure parsing of events with identically named or unnamed tags:

     1. Create a new normalizer or open an existing normalizer for editing.
     2. In the Basic event parsing window of the normalizer, in the Parsing method drop-down list, select 'xml' and in the Tag numbering field, click Add field.

        In the displayed field, enter the full path to the tag to whose elements you want to assign a number. For example, Event.EventData.Data. The first number to be assigned to a tag is 0. If the tag is empty, for example, <Data />, it is also assigned a number.

     3. To configure data mapping, under Mapping, click Add row and do the following:
        1. In the new row, in the Source field, enter the full path to the tag and its index. For the Microsoft Windows event from the example above, the full path with indices look like this:
           • Event.EventData.Data.0
           • Event.EventData.Data.1
           • Event.EventData.Data.2 and so on
        2. In the KUMA field drop-down list, select the field in the KUMA event that will receive the value from the numbered tag after parsing.
     4. To save changes:
        • If you created a new normalizer, click Save.
        • If you edited an existing normalizer, click Update configuration in the collector to which the normalizer is linked.

     Parsing is configured.

   • netflow5

     This parsing method is used to process data in the NetFlow v5 format.

     When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button. If the netflow5 type is selected for the main parsing, extra normalization is not available.

     In mapping rules, the protocol type for netflow5 is not indicated in the fields of KUMA events by default. When parsing data in NetFlow format, on the Enrichment normalizer tab, you must create a constant data enrichment rule that adds the netflow value to the DeviceProduct target field.

   • netflow9

     This parsing method is used to process data in the NetFlow v9 format.

     When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button. If the netflow9 type is selected for the main parsing, extra normalization is not available.

     In mapping rules, the protocol type for netflow9 is not indicated in the fields of KUMA events by default. When parsing data in NetFlow format, on the Enrichment normalizer tab, you must create a constant data enrichment rule that adds the netflow value to the DeviceProduct target field.

   • sflow5

     This parsing method is used to process data in sflow5 format.

     When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button. If the sflow5 type is selected for the main parsing, extra normalization is not available.

   • ipfix

     This parsing method is used to process IPFIX data.

     When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button. If the ipfix type is selected for the main parsing, extra normalization is not available.

     In mapping rules, the protocol type for ipfix is not indicated in the fields of KUMA events by default. When parsing data in NetFlow format, on the Enrichment normalizer tab, you must create a constant data enrichment rule that adds the netflow value to the DeviceProduct target field.

   • sql—this method becomes available only when using a sql type connector.

     The normalizer uses this method to process data obtained by making a selection from the database.

4. In the Keep raw event drop-down list, specify whether to store the original raw event in the newly created normalized event. Available values:
   • Don't save—do not save the raw event. This is the default setting.
   • Only errors—save the raw event in the Raw field of the normalized event if errors occurred when parsing it. This value is convenient to use when debugging a service. In this case, every time an event has a non-empty Raw field, you know there was a problem.

     If fields containing the names *Address or *Date* do not comply with normalization rules, these fields are ignored. No normalization error occurs in this case, and the values of the fields are not displayed in the Raw field of the normalized event even if the Keep raw event → Only errors option was selected.

   • Always—always save the raw event in the Raw field of the normalized event.

   This setting is not available for extra parsing rules.

5. In the Keep extra fields drop-down list, choose whether you want to store the raw event fields in the normalized event if no mapping rules have been configured for them (see below). The data is stored in the Extra event field. Normalized events can be searched and filtered based on the data stored in the Extra field.

   Filtering based on data from the Extra event field

   Conditions for filters based on data from the Extra event field:

   • Condition—If.
   • Left operand—event field.
   • In this event field, you can specify one of the following values:
     • Extra field.
     • Value from the Extra field in the following format:

       Extra.<field name>

       For example, Extra.app.

       A value of this type is specified manually.

     • Value from the array written to the Extra field in the following format:

       Extra.<field name>.<array element>

       For example, Extra.array.0.

       The values in the array are numbered starting from 0.

       A value of this type is specified manually.

       To work with a value from the Extra field at depth 3 and below, use backquotes ``. For example, `Extra.lev1.lev2.lev3`.

   • Operator – =.
   • Right operand—constant.
   • Value—the value by which you need to filter events.

   By default, fields are not saved.

6. In the Description field, specify the resource description: up to 4,000 Unicode characters.

   This setting is not available for extra parsing rules.

7. In the Event examples field, you can provide an example of data that you want to process.

   This setting is not available for the following parsing methods: netflow5, netflow9, sflow5, ipfix, sql.

   The Event examples field is populated with data obtained from the raw event if the event was successfully parsed and the type of data obtained from the raw event matches the type of the KUMA field.

   For example, the value "192.168.0.1" enclosed in quotation marks is not displayed in the SourceAddress field, in this case the value 192.168.0.1 is displayed in the Event examples field.

8. In the Mapping table, configure the mapping of raw event fields to fields of the event in KUMA format:
   1. In the Source column, provide the name of the raw event field that you want to convert into the KUMA event field.

      For details about the field format, refer to the Normalized event data model article. For a description of the mapping, refer to the Mapping fields of predefined normalizers article.

      Clicking the button next to the field names in the Source column opens the Conversion window, in which you can click the Add conversion button to create rules for modifying the original data before they are written to the KUMA event fields.

      Available conversions

      Conversions are changes that can be applied to a value before it gets written to the event field. The conversion type is selected from a drop-down list.

      Available conversions:

      • lower—is used to make all characters of the value lowercase
      • upper—is used to make all characters of the value uppercase
      • regexp – used to convert a value using the regular expression RE2. When this conversion type is selected, the field appears where regular expression should be added.
      • substring—is used to extract characters in the position range specified in the Start and End fields. These fields appear when this conversion type is selected.
      • replace—is used to replace specified character sequence with the other character sequence. When this type of conversion is selected, new fields appear:
        • Replace chars—in this field you can specify the character sequence that should be replaced.
        • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
      • trim—used to simultaneously remove the characters specified in the Chars field from the leading and end positions of the value. The field appears when this type of conversion is selected. For example, a trim conversion with the Micromon value applied to Microsoft-Windows-Sysmon results in soft-Windows-Sys.
      • append is used to add the characters specified in the Constant field to the end of the event field value. The field appears when this type of conversion is selected.
      • prepend—used to prepend the characters specified in the Constant field to the start of the event field value. The field appears when this type of conversion is selected.
      • replace with regexp—is used to replace RE2 regular expression results with the character sequence.
        • Expression—in this field you can specify the regular expression which results that should be replaced.
        • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
      • Converting encoded strings to text:
        • decodeHexString—used to convert a HEX string to text.
        • decodeBase64String—used to convert a Base64 string to text.
        • decodeBase64URLString—used to convert a Base64url string to text.

        When converting a corrupted string or if conversion error occur, corrupted data may be written to the event field.

        During event enrichment, if the length of the encoded string exceeds the size of the field of the normalized event, the string is truncated and is not decoded.

        If the length of the decoded string exceeds the size of the event field into which the decoded value is to be written, such a string is truncated to fit the size of the event field.

      Conversions when using the extended event schema

      Whether or not a conversion can be used depends on the type of extended event schema field being used:

      • For an additional field of the "String" type, all types of conversions are available.
      • For fields of the "Number" and "Float" types, the following types of conversions are available: regexp, substring, replace, trim, append, prepend, replaceWithRegexp, decodeHexString, decodeBase64String, decodeBase64URLString.
      • For fields of "Array of strings", "Array of numbers", and "Array of floats" types, the following types of conversions are available: append, prepend.

      In the Conversion window, you can swap the added rules by dragging them by the icon; you can also delete them using the icon.

   2. In the KUMA field column, select the required KUMA event field from the drop-down list. You can search for fields by entering their names in the field.

      Recommendations concerning the KUMA field column

      We recommend that you configure the mapping for the following KUMA fields. Otherwise, you will not be able to view observables in alert details and incident details.

      The recommended KUMA fields depend on the observable types:

      • For observables of the MD5 hash and SHA256 types:
        • FileHash
      • For observables of the URL type:
        • RequestUrl
      • For observables of the IP address type:
        • DeviceCustomIPv6Address1
        • DeviceCustomIPv6Address2
        • DeviceCustomIPv6Address3
        • DeviceCustomIPv6Address4
        • DestinationTranslatedAddress
        • DeviceTranslatedAddress
        • DestinationAddress
        • DeviceAddress
        • SourceTranslatedAddress
        • SourceAddress
      • For observables of the Domain name type:
        • DestinationDnsDomain
        • DeviceDnsDomain
        • DeviceNtDomain
        • DestinationNtDomain
        • SourceDnsDomain
        • SourceNtDomain
      • For observables of the UserName type:
        • DestinationUserName
        • SourceUserName
      • For observables of the HostName type:
        • DestinationHostName
        • DeviceHostName
        • SourceHostName
   3. If the name of the KUMA event field selected at the previous step begins with DeviceCustom* or Flex*, you can add a unique custom label in the Label field.

   New table rows can be added by clicking the Add row button. Rows can be deleted individually by clicking the button or all at once by clicking the Clear all button.

   If you want KUMA to enrich events with asset information, and the asset information to be available in the alert card when a correlation rule is triggered, in the Mapping table, configure a mapping of host address and host name fields depending on the purpose of the asset. For example, the mapping can apply to SourceAddress and SourceHostName, or DestinationAddress and DestinationHostName fields. As a result of enrichment, the event card includes a SourceAssetID or DestinationAssetID field, and a link to the asset card. Also, as a result of enrichment, asset information is available in the alert card.

   If you have loaded data into the Event examples field, the table will have an Examples column containing examples of values carried over from the raw event field to the KUMA event field.

   If the size of the KUMA event field is less than the length of the value placed in it, the value is truncated to the size of the event field.

Extended event schema

When normalizing events, extended event schema fields can be used in addition to standard KUMA event schema fields. Information about the types of extended event schema fields is shown in the table below.

Using many unique fields of the extended event schema can reduce the performance of the system, increase the amount of disk space required for storing events, and make the information difficult to understand.

We recommend consciously choosing a minimal set of additional fields of the extended event schema that you want to use in normalizers and correlation.

To use extended event schema fields:

• Open an existing event normalizer or create a new event normalizer.
• Specify the basic settings of the normalizer.
• Click "Add row".
• For the "Source" setting, enter the name of the source field in the raw event.
• For the "KUMA field" setting, enter the name of the extended event schema field that you are creating (see the table below). You can also use an existing field of the extended event schema.

  Fields of the extended data model of normalized events:

  Field name Specified in the KUMA field setting	Data type	Availability in the normalizer	Description
  S.<field name>	String	All types	Field of the "String" type
  N.<field name>	Number	All types	Field of the "Number" type
  F.<field name>	Float	All types	Field of the "Float" type
  SA.<field name>	Array of strings	KV, JSON	Field of the "Array of strings" type. The order of the array elements is the same as the order of the elements of the raw event.
  NA.<field name>	Array of integers	KV, JSON	A field of the "Array of integers" type. The order of the array elements is the same as the order of the elements of the raw event.
  FA.<field name>	Array of floats	KV, JSON	Field of the "Array of floats" type. The order of the array elements is the same as the order of the elements of the raw event.

The prefixes "S.", "N.", "F.", "SA.", "NA.", "FA." are required when creating fields of the extended event schema; the prefixes must be strictly uppercase.

Replace <field name> with the field name. You may use letters of the English alphabet and numerals in the field name. The space character is not allowed.

• Click OK.
• Click Save to finish editing the event normalizer.

The normalizer is saved, and the additional field is created. After saving the normalizer, the additional field can be used in other normalizers.

Note: If the data in the fields of the raw event does not match the type of the KUMA field, the value is not saved during the normalization of events. For example, the string "test" cannot be written to the DeviceCustomNumber1 KUMA field of the Number type.

If you want to minimize the load on the storage server when searching events, preparing reports, and performing other operations on events in storage, use KUMA event schema fields as your first preference, extended event schema fields as your second preference, and the Extra fields as your last resort.

Enrichment in the normalizer

When creating event parsing rules in the normalizer settings window, on the Enrichment tab, you can configure the rules for adding extra data to the fields of the normalized event using enrichment rules. These enrichment rules are stored in the settings of the normalizer where they were created.

Enrichments are created by clicking the Add enrichment button. There can be more than one enrichment rule. You can delete enrichment rules by clicking the button.

Settings available in the enrichment rule settings block:

• Source kind (required)—drop-down list for selecting the type of enrichment. Depending on the selected type, you may see advanced settings that will also need to be completed.

  Available Enrichment rule source types:

  • constant

    This type of enrichment is used when a constant needs to be added to an event field. Settings of this type of enrichment:

    • In the Constant field, specify the value that should be added to the event field. The value may not be longer than 255 Unicode characters. If you leave this field blank, the existing event field value will be cleared.
    • In the Target field drop-down list, select the KUMA event field to which you want to write the data.

    If you are using the event enrichment functions for extended schema fields of "String", "Number", or "Float" type with a constant, the constant is added to the field.

    If you are using the event enrichment functions for extended schema fields of "Array of strings", "Array of numbers", or "Array of floats" type with a constant, the constant is added to the elements of the array.

  • dictionary

    This type of enrichment is used if you need to add a value from the dictionary of the Dictionary type.

    When this type is selected in the Dictionary name drop-down list, you must select the dictionary that will provide the values. In the Key fields settings block, you have to click the Add field button and select the event fields whose values will be used for dictionary entry selection.

    If you are using event enrichment with the "Dictionary" type selected as the "Source kind" setting, and an array field is specified in the "Key enrichment fields" setting, when an array is passed as the dictionary key, the array is serialized into a string in accordance with the rules of serializing a single value in the TSV format.

    Example: The "Key enrichment fields" setting uses the SA.StringArrayOne extended schema field. The SA.StringArrayOne extended schema field contains 3 elements: "a", "b" and "c". The following value is passed to the dictionary as the key: ['a','b','c'].

    If the "Key enrichment fields" setting uses an extended schema array field and a regular event schema field, the field values are separated by the "|" character when the dictionary is queried.

    Example: The "Key enrichment fields" setting uses two fields: the SA.StringArrayOne extended schema field and the Code field. The SA.StringArrayOne extended schema field contains 3 elements: "a", "b", and "c"; the Code string field contains the character sequence "myCode". The following value is passed to the dictionary as the key: ['a','b','c']|myCode.

  • table

    This type of enrichment is used if you need to add a value from the dictionary of the Table type.

    When this enrichment type is selected in the Dictionary name drop-down list, select the dictionary for providing the values. In the Key fields group of settings, click the Add field button to select the event fields whose values are used for dictionary entry selection.

    In the Mapping table, configure the dictionary fields to provide data and the event fields to receive data:

    • In the Dictionary field column, select the dictionary field. The available fields depend on the selected dictionary resource.
    • In the KUMA field column, select the event field to which the value is written. For some of the selected fields (*custom* and *flex*), in the Label column, you can specify a name for the data written to them.

    New table rows can be added by clicking the Add new element button. Columns can be deleted by clicking the button.

  • event

    This type of enrichment is used when you need to write a value from another event field to the current event field. Settings of this type of enrichment:

    • In the Target field drop-down list, select the KUMA event field to which you want to write the data.
    • In the Source field drop-down list, select the event field whose value will be written to the target field.
    • Clicking the button opens the Conversion window in which you can, by clicking the Add conversion button, create rules for modifying the original data before writing them to the KUMA event fields.

      Available conversions

      Conversions are changes that can be applied to a value before it gets written to the event field. The conversion type is selected from a drop-down list.

      Available conversions:

      • lower—is used to make all characters of the value lowercase
      • upper—is used to make all characters of the value uppercase
      • regexp – used to convert a value using the regular expression RE2. When this conversion type is selected, the field appears where regular expression should be added.
      • substring—is used to extract characters in the position range specified in the Start and End fields. These fields appear when this conversion type is selected.
      • replace—is used to replace specified character sequence with the other character sequence. When this type of conversion is selected, new fields appear:
        • Replace chars—in this field you can specify the character sequence that should be replaced.
        • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
      • trim—used to simultaneously remove the characters specified in the Chars field from the leading and end positions of the value. The field appears when this type of conversion is selected. For example, a trim conversion with the Micromon value applied to Microsoft-Windows-Sysmon results in soft-Windows-Sys.
      • append is used to add the characters specified in the Constant field to the end of the event field value. The field appears when this type of conversion is selected.
      • prepend—used to prepend the characters specified in the Constant field to the start of the event field value. The field appears when this type of conversion is selected.
      • replace with regexp—is used to replace RE2 regular expression results with the character sequence.
        • Expression—in this field you can specify the regular expression which results that should be replaced.
        • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
      • Converting encoded strings to text:
        • decodeHexString—used to convert a HEX string to text.
        • decodeBase64String—used to convert a Base64 string to text.
        • decodeBase64URLString—used to convert a Base64url string to text.

        When converting a corrupted string or if conversion error occur, corrupted data may be written to the event field.

        During event enrichment, if the length of the encoded string exceeds the size of the field of the normalized event, the string is truncated and is not decoded.

        If the length of the decoded string exceeds the size of the event field into which the decoded value is to be written, such a string is truncated to fit the size of the event field.

      Conversions when using the extended event schema

      Whether or not a conversion can be used depends on the type of extended event schema field being used:

      • For an additional field of the "String" type, all types of conversions are available.
      • For fields of the "Number" and "Float" types, the following types of conversions are available: regexp, substring, replace, trim, append, prepend, replaceWithRegexp, decodeHexString, decodeBase64String, decodeBase64URLString.
      • For fields of "Array of strings", "Array of numbers", and "Array of floats" types, the following types of conversions are available: append, prepend.

    When using enrichment of events that have the "Event" selected as the "Source kind" setting and the fields of the extended event schema are used as arguments, the following special considerations apply:

    • If the source field is an "Array of strings" field and the target field is a "String" field, the values are written to the target field in the TSV format.

      Example: The SA.StringArray extended event schema field contains values: "string1", "string2", "string3". An event enrichment operation is performed. The result of the operation is written to the DeviceCustomString1 event schema field. As a result of the operation, the DeviceCustomString1 field contains ["string1", "string2", "string3"].

    • If the source field is an "Array of strings" field and the target field is an "Array of strings" field, the values of the source field are appended to the values of the target field and are placed in the target field, with commas (",") used as the separator character.

      Example: The SA.StringArrayOne extended event schema field contains values: "string1", "string2", "string3". An event enrichment operation is performed. The result of the operation is written to the SA.StringArrayTwo event schema field. As a result of the operation, the SA.StringArrayTwo field contains "string1", "string2", "string3".

  • template

    This type of enrichment is used when you need to write a value obtained by processing Go templates into the event field. Settings of this type of enrichment:

    • Put the Go template into the Template field.

      Event field names are passed in the {{.EventField}} format, where EventField is the name of the event field from which the value must be passed to the script.

      Example: Attack on {{.DestinationAddress}} from {{.SourceAddress}}.

    • In the Target field drop-down list, select the KUMA event field to which you want to write the data.

    To convert the data in an array field in a template into the TSV format, you must use the toString function.

    If you are using enrichment of events that have the "Template" type selected as the "Source kind" setting, in which the target field has the "String" type, and the source field is an extended event schema field containing an array of strings, you can use one of the following examples for the template.

    Example:

    {{.SA.StringArrayOne}}

    Example:

    {{- range $index, $element := . SA.StringArrayOne -}}

    {{- if $index}}, {{end}}"{{$element}}"{{- end -}}

• Target field (required)—drop-down list for selecting the KUMA event field that should receive the data.

  This setting is not available for the enrichment source of the Table type.

Conditions for forwarding data to an extra normalizer

When creating additional event parsing rules, you can specify the conditions. When these conditions are met, the events are sent to the created parsing rule for processing. Conditions can be specified in the Additional event parsing window, on the Extra normalization conditions tab. This tab is not available for the basic parsing rules.

Available settings:

• Use raw event — If you want to send a raw event for extra normalization, select Yes in the Keep raw event drop-down list. The default value is No. We recommend passing a raw event to normalizers of json and xml types. If you want to send a raw event for extra normalization to the second, third, etc nesting levels, at each nesting level, select Yes in the Keep raw event drop-down list.
• Field to pass into normalizer—indicates the event field if you want only events with fields configured in normalizer settings to be sent for additional parsing.

  If this field is blank, the full event is sent to the extra normalizer for processing.

• Set of filters—used to define complex conditions that must be met by the events received by the normalizer.

  You can click the Add condition button to add a string containing fields for identifying the condition (see below).

  You can click the Add group button to add a group of filters. Group operators can be switched between AND, OR, and NOT. You can add other condition groups and individual conditions to filter groups.

  You can swap conditions and condition groups by dragging them by the icon; you can also delete them using the icon.

Filter condition settings:

• Left operand and Right operand—used to specify the values to be processed by the operator.

  In the left operand, you must specify the source field of events coming into the normalizer. For example, if the eventType - DeviceEventClass mapping is configured in the Basic event parsing window, then in the Additional event parsing window on the Extra normalization conditions tab, you must specify eventType in the left operand field of the filter. Data is processed only as text strings.

• Operators:
  • = – full match of the left and right operands.
  • startsWith – the left operand starts with the characters specified in the right operand.
  • endsWith – the left operand ends with the characters specified in the right operand.
  • match – the left operand matches the regular expression (RE2) specified in the right operand.
  • in – the left operand matches one of the values specified in the right operand.

The incoming data can be converted by clicking the button. The Conversion window opens, where you can click the Add conversion button to create the rules for converting the source data before any actions are performed on them. In the Conversion window, you can swap the added rules by dragging them by the icon; you can also delete them using the icon.

Available conversions

Supported event sources

KUMA supports the normalization of events coming from systems listed in the "Supported event sources" table. Normalizers for these systems are included in the distribution kit.

Supported event sources

Aggregation rules

Aggregation rules let you combine repetitive events of the same type and replace them with one common event. Aggregation rules support fields of the standard KUMA event schema as well as fields of the extended event schema. In this way, you can reduce the number of similar events sent to the storage and/or correlator, reduce the workload on services, conserve data storage space and licensing quota (EPS). An aggregation event is created when a time or number of events threshold is reached, whichever occurs first.

For aggregation rules, you can configure a filter and apply it only to events that match the specified conditions.

You can configure aggregation rules under Resources - Aggregation rules, and then select the created aggregation rule from the drop-down list in the collector settings. You can also configure aggregation rules directly in the collector settings.

Available aggregation rule settings

The OSMP distribution kit includes aggregation rules listed in the table below.

Predefined aggregation rules

Enrichment rules

Event enrichment involves adding information to events that can be used to identify and investigate an incident.

Enrichment rules let you add supplementary information to event fields by transforming data that is already present in the fields, or by querying data from external systems. For example, suppose that a user name is recorded in the event. You can use an enrichment rule to add information about the department, position, and manager of this user to the event fields.

Enrichment rules can be used in the following KUMA services and features:

• Collector.
• Correlator.
• Normalizer.

Available enrichment rule settings are listed in the table below.

Basic settings tab

Predefined enrichment rules

The OSMP distribution kit includes enrichment rules listed in the table below.

Predefined enrichment rules

Correlation rules

Correlation rules are used to recognize specific sequences of processed events and to take certain actions after recognition, such as creating correlation events/alerts or interacting with an active list.

Correlation rules can be used in the following KUMA services and features:

• Correlator.
• Notification rule.
• Links of segmentation rules.
• Retroscan.

The available correlation rule settings depend on the selected type. Types of correlation rules:

• standard—used to find correlations between several events. Resources of this kind can create correlation events.

  This rule kind is used to determine complex correlation patterns. For simpler patterns you should use other correlation rule kinds that require less resources to operate.

• simple—used to create correlation events if a certain event is found.
• operational—used for operations with Active lists and context tables. This rule kind cannot create correlation events.

For these resources, you can enable the display of control characters in all input fields except the Description field.

If a correlation rule is used in the correlator and an alert was created based on it, any change to the correlation rule will not result in a change to the existing alert even if the correlator service is restarted. For example, if the name of a correlation rule is changed, the name of the alert will remain the same. If you close the existing alert, a new alert will be created and it will take into account the changes made to the correlation rule.

Standard correlation rules

Standard correlation rules are used to identify complex patterns in processed events.

The search for patterns is conducted by using buckets

The correlation rule window contains the following tabs:

• General—used to specify the main settings of the correlation rule. On this tab, you can select the type of correlation rule.
• Selectors—used to define the conditions that the processed events must fulfill to trigger the correlation rule. Available settings vary based on the selected rule type.
• Actions—used to set the triggers that will activate when the conditions configured in the Selectors settings block are fulfilled. The Correlation rule resource must have at least one trigger. Available settings vary based on the selected rule type.
• Correlators—used for linking correlators. Available only for created correlation rules that are open for editing.

General tab

• Name (required)—a unique name for this type of resource. Must contain 1 to 128 Unicode characters.
• Tenant (required)—the tenant that owns the correlation rule.
• Type (required)—a drop-down list for selecting the type of correlation rule. Select standard if you want to create a standard correlation rule.
• Identical fields (required)—the event fields that should be grouped in a Bucket. The hash of the values of the selected fields is used as the Bucket key. If the selector (see below) triggers, the selected fields will be copied to the correlation event.

  If different selectors of the correlation rule use fields that have different values ​​in events, do not specify these fields in the Identical fields section.

• Unique fields—event fields that should be sent to the Bucket. If this parameter is set, the Bucket will receive only unique events. The hash of the selected fields' values is used as the Bucket key.

  You can use local variables in the Identical fields and Unique fields sections. To access a variable, its name must be preceded with the "$" character.
  For an example of using local variables in these sections, refer to the rule provided with KUMA: R403_Access to malicious resources from a host with disabled protection or an out-of-date anti-virus database.

• Rate limit—maximum number of times a correlation rule can be triggered per second. The default value is 100.

  If correlation rules employing complex logic for pattern detection are not triggered, this may be due to the specific method used to count rule triggers in KUMA. In this case, try to increase the value of Rate limit to 1000000, for example.

• Window, sec (required)—bucket lifetime, in seconds. Default value: 86,400 seconds (24 hours). This timer starts when the Bucket is created (when it receives the first event). The lifetime is not updated, and when it runs out, the On timeout trigger from the Actions group of settings is activated and the bucket is deleted. The On every threshold and On subsequent thresholds triggers can be activated more than once during the lifetime of the bucket.
• Base events keep policy—this drop-down list is used to specify which base events must be stored in the correlation event:
  • first (default value)—this option is used to store the first base event of the event collection that triggered creation of the correlation event.
  • last—this option is used to store the last base event of the event collection that triggered creation of the correlation event.
  • all—this option is used to store all base events of the event collection that triggered creation of the correlation event.
• Priority—base coefficient used to determine the importance of a correlation rule. The default value is Low.
• Order by—in this drop-down list, you can select the event field that will be used by the correlation rule selectors to track situational changes. This could be useful if you want to configure a correlation rule to be triggered when several types of events occur sequentially, for example.
• Description—the description of a resource. Up to 4,000 Unicode characters.

Selectors tab

A rule of the standard kind can have multiple selectors. You can add selectors by clicking the Add selector button and can remove them by clicking the Delete selector button. Selectors can be moved by clicking the button.

For each selector, the following two tabs are available: Settings and Local variables.

The Settings tab contains the following settings:

• Alias (required)—unique name of the event group that meets the conditions of the selector. Must contain 1 to 128 Unicode characters.
• Selector threshold (event count) (required)—the number of events that must be received by the selector to trigger. The default value is 1.
• Filter (required)—used to set the criteria for determining events that should trigger the selector. You can select an existing filter from the drop-down list or create a new filter.

  Creating a filter in resources

  1. In the Filter drop-down list, select Create new.
  2. If you want to keep the filter as a separate resource, select the Save filter check box.

     In this case, you will be able to use the created filter in various services.

     This check box is cleared by default.

  3. If you selected the Save filter check box, enter a name for the created filter resource in the Name field. The name must contain 1 to 128 Unicode characters.
  4. In the Conditions settings block, specify the conditions that the events must meet:
     1. Click the Add condition button.
     2. In the Left operand and Right operand drop-down lists, specify the search parameters.

        Depending on the data source selected in the Right operand field, you may see fields of additional parameters that you need to use to define the value that will be passed to the filter. For example, when choosing active list you will need to specify the name of the active list, the entry key, and the entry key field.

     3. In the operator drop-down list, select the relevant operator.

        Filter operators

        • =—the left operand equals the right operand.
        • <—the left operand is less than the right operand.
        • <=—the left operand is less than or equal to the right operand.
        • >—the left operand is greater than the right operand.
        • >=—the left operand is greater than or equal to the right operand.
        • inSubnet—the left operand (IP address) is in the subnet of the right operand (subnet).
        • contains—the left operand contains values of the right operand.
        • startsWith—the left operand starts with one of the values of the right operand.
        • endsWith—the left operand ends with one of the values of the right operand.
        • match—the left operand matches the regular expression of the right operand. The RE2 regular expressions are used.
        • hasBit—checks whether the left operand (string or number) contains bits whose positions are listed in the right operand (in a constant or in a list).

          The value to be checked is converted to binary and processed right to left. Chars are checked whose index is specified as a constant or a list.

          If the value being checked is a string, then an attempt is made to convert it to integer and process it in the way described above. If the string cannot be converted to a number, the filter returns False.

        • hasVulnerability—checks whether the left operand contains an asset with the vulnerability and vulnerability severity specified in the right operand.

          If you do not specify the ID and severity of the vulnerability, the filter is triggered if the asset in the event being checked has any vulnerability.

        • inActiveList—this operator has only one operand. Its values are selected in the Key fields field and are compared with the entries in the active list selected from the Active List drop-down list.
        • inDictionary—checks whether the specified dictionary contains an entry defined by the key composed with the concatenated values of the selected event fields.
        • inCategory—the asset in the left operand is assigned at least one of the asset categories of the right operand.
        • inActiveDirectoryGroup—the Active Directory account in the left operand belongs to one of the Active Directory groups in the right operand.
        • TIDetect—this operator is used to find events using CyberTrace Threat Intelligence (TI) data. This operator can be used only on events that have completed enrichment with data from CyberTrace Threat Intelligence. In other words, it can only be used in collectors at the destination selection stage and in correlators.
        • inContextTable—presence of the entry in the specified context table.
        • intersect—presence in the left operand of the list items specified in the right operand.
     4. If necessary, select the do not match case check box. When this check box is selected, the operator ignores the case of the values.

        The selection of this check box does not apply to the InSubnet, InActiveList, InCategory or InActiveDirectoryGroup operators.

        This check box is cleared by default.

     5. If you want to add a negative condition, select If not from the If drop-down list.
     6. You can add multiple conditions or a group of conditions.
  5. If you have added multiple conditions or groups of conditions, choose a search condition (and, or, not) by clicking the AND button.
  6. If you want to add existing filters that are selected from the Select filter drop-down list, click the Add filter button.

     You can view the nested filter settings by clicking the button.

  Filtering based on data from the Extra event field

  Conditions for filters based on data from the Extra event field:

  • Condition—If.
  • Left operand—event field.
  • In this event field, you can specify one of the following values:
    • Extra field.
    • Value from the Extra field in the following format:

      Extra.<field name>

      For example, Extra.app.

      A value of this type is specified manually.

    • Value from the array written to the Extra field in the following format:

      Extra.<field name>.<array element>

      For example, Extra.array.0.

      The values in the array are numbered starting from 0.

      A value of this type is specified manually.

      To work with a value from the Extra field at depth 3 and below, use backquotes ``. For example, `Extra.lev1.lev2.lev3`.

  • Operator – =.
  • Right operand—constant.
  • Value—the value by which you need to filter events.

  The order of conditions specified in the selector filter of the correlation rule is significant and affects system performance. We recommend putting the most unique condition in the first place in the selector filter.

  Consider two examples of selector filters that select successful authentication events in Microsoft Windows.

  Selector filter 1:

  Condition 1. DeviceProduct = Microsoft Windows

  Condition 2. DeviceEventClassID = 4624

  Selector filter 2:

  Condition 1. DeviceEventClassID = 4624

  Condition 2. DeviceProduct = Microsoft Windows

  The order of conditions in Selector filter 2 is preferable because it causes less load on the system.

• Recovery—this check box must be selected when the Correlation rule must NOT trigger if a certain number of events are received from the selector. By default, this check box is cleared.

Select the Local variables tab and click Add variable to declare variables that you want to use within the limits of this correlation rule.

In the selector of the correlation rule, you can use regular expressions conforming to the RE2 standard.

Using regular expressions in correlation rules is computationally intensive compared to other operations. Therefore, when designing correlation rules, we recommend limiting the use of regular expressions to the necessary minimum and using other available operations.

To use a regular expression, you must use the match comparison operator. The regular expression must be placed in a constant. The use of capture groups in regular expressions is optional. For the correlation rule to trigger, the field text matched against the regexp must exactly match the regular expression.

For a primer on syntax and examples of correlation rules that use regular expressions in their selectors, see the following rules that are provided with KUMA:

• R105_04_Suspicious PowerShell commands. Suspected obfuscation.
• R333_Suspicious creation of files in the autorun folder.

Actions tab

A rule of the standard kind can have multiple triggers.

• On first threshold—this trigger activates when the Bucket registers the first triggering of the selector during the lifetime of the Bucket.
• On subsequent thresholds—this trigger activates when the Bucket registers the second and all subsequent triggering of the selector during the lifetime of the Bucket.
• On every threshold—this trigger activates every time the Bucket registers the triggering of the selector.
• On timeout—this trigger activates when the lifetime of the Bucket ends, and is linked to the selector with the Recovery check box selected. In other words, this trigger activates if the situation detected by the correlation rule is not resolved within the defined amount of time.

Every trigger is represented as a group of settings with the following parameters available:

• Output—if this check box is selected, the correlation event is sent for post-processing: for external enrichment outside the correlation rule, for response, and to destinations.
• Loop to correlator—if this check box is selected, the created correlation event is processed by the rule chain of the current correlator. This allows hierarchical correlation.

  If the Output and Loop to correlator check boxes are set, the correlation rule is sent to post-processing first, and then to the selectors of the current correlation rule.

• Do not create alert—if this check box is selected, an alert is not created when this correlation rule is triggered. If you do not want to create an alert when a correlation rule is triggered, but you still want to send a correlation event to the storage, select the Output and No alert check boxes. If you select only the No alert check box, a correlation event is not saved in the storage.
• Under Enrichment, you can modify the fields of correlation events by using enrichment rules. These enrichment rules are stored in the correlation rule where they were created. You can create multiple enrichment rules. Enrichment rules can be added or deleted by clicking the Add enrichment or Remove enrichment buttons, respectively.
  • Source kind—you can select the type of enrichment in this drop-down list. Depending on the selected type, you may see advanced settings that will also need to be completed.

    Available types of enrichment:

    • constant

      This type of enrichment is used when a constant needs to be added to an event field. Settings of this type of enrichment:

      • In the Constant field, specify the value that should be added to the event field. The value may not be longer than 255 Unicode characters. If you leave this field blank, the existing event field value will be cleared.
      • In the Target field drop-down list, select the KUMA event field to which you want to write the data.

      If you are using the event enrichment functions for extended schema fields of "String", "Number", or "Float" type with a constant, the constant is added to the field.

      If you are using the event enrichment functions for extended schema fields of "Array of strings", "Array of numbers", or "Array of floats" type with a constant, the constant is added to the elements of the array.

    • dictionary

      This type of enrichment is used if you need to add a value from the dictionary of the Dictionary type.

      When this type is selected in the Dictionary name drop-down list, you must select the dictionary that will provide the values. In the Key fields settings block, you have to click the Add field button and select the event fields whose values will be used for dictionary entry selection.

      If you are using event enrichment with the "Dictionary" type selected as the "Source kind" setting, and an array field is specified in the "Key enrichment fields" setting, when an array is passed as the dictionary key, the array is serialized into a string in accordance with the rules of serializing a single value in the TSV format.

      Example: The "Key enrichment fields" setting uses the SA.StringArrayOne extended schema field. The SA.StringArrayOne extended schema field contains 3 elements: "a", "b" and "c". The following value is passed to the dictionary as the key: ['a','b','c'].

      If the "Key enrichment fields" setting uses an extended schema array field and a regular event schema field, the field values are separated by the "|" character when the dictionary is queried.

      Example: The "Key enrichment fields" setting uses two fields: the SA.StringArrayOne extended schema field and the Code field. The SA.StringArrayOne extended schema field contains 3 elements: "a", "b", and "c"; the Code string field contains the character sequence "myCode". The following value is passed to the dictionary as the key: ['a','b','c']|myCode.

    • table

      This type of enrichment is used if you need to add a value from the dictionary of the Table type.

      When this enrichment type is selected in the Dictionary name drop-down list, select the dictionary for providing the values. In the Key fields group of settings, click the Add field button to select the event fields whose values are used for dictionary entry selection.

      In the Mapping table, configure the dictionary fields to provide data and the event fields to receive data:

      • In the Dictionary field column, select the dictionary field. The available fields depend on the selected dictionary resource.
      • In the KUMA field column, select the event field to which the value is written. For some of the selected fields (*custom* and *flex*), in the Label column, you can specify a name for the data written to them.

      New table rows can be added by clicking the Add new element button. Columns can be deleted by clicking the button.

    • event

      This type of enrichment is used when you need to write a value from another event field to the current event field. Settings of this type of enrichment:

      • In the Target field drop-down list, select the KUMA event field to which you want to write the data.
      • In the Source field drop-down list, select the event field whose value will be written to the target field.
      • Clicking the button opens the Conversion window in which you can, by clicking the Add conversion button, create rules for modifying the original data before writing them to the KUMA event fields.

        Available conversions

        Conversions are changes that can be applied to a value before it gets written to the event field. The conversion type is selected from a drop-down list.

        Available conversions:

        • lower—is used to make all characters of the value lowercase
        • upper—is used to make all characters of the value uppercase
        • regexp – used to convert a value using the regular expression RE2. When this conversion type is selected, the field appears where regular expression should be added.
        • substring—is used to extract characters in the position range specified in the Start and End fields. These fields appear when this conversion type is selected.
        • replace—is used to replace specified character sequence with the other character sequence. When this type of conversion is selected, new fields appear:
          • Replace chars—in this field you can specify the character sequence that should be replaced.
          • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
        • trim—used to simultaneously remove the characters specified in the Chars field from the leading and end positions of the value. The field appears when this type of conversion is selected. For example, a trim conversion with the Micromon value applied to Microsoft-Windows-Sysmon results in soft-Windows-Sys.
        • append is used to add the characters specified in the Constant field to the end of the event field value. The field appears when this type of conversion is selected.
        • prepend—used to prepend the characters specified in the Constant field to the start of the event field value. The field appears when this type of conversion is selected.
        • replace with regexp—is used to replace RE2 regular expression results with the character sequence.
          • Expression—in this field you can specify the regular expression which results that should be replaced.
          • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
        • Converting encoded strings to text:
          • decodeHexString—used to convert a HEX string to text.
          • decodeBase64String—used to convert a Base64 string to text.
          • decodeBase64URLString—used to convert a Base64url string to text.

          When converting a corrupted string or if conversion error occur, corrupted data may be written to the event field.

          During event enrichment, if the length of the encoded string exceeds the size of the field of the normalized event, the string is truncated and is not decoded.

          If the length of the decoded string exceeds the size of the event field into which the decoded value is to be written, such a string is truncated to fit the size of the event field.

        Conversions when using the extended event schema

        Whether or not a conversion can be used depends on the type of extended event schema field being used:

        • For an additional field of the "String" type, all types of conversions are available.
        • For fields of the "Number" and "Float" types, the following types of conversions are available: regexp, substring, replace, trim, append, prepend, replaceWithRegexp, decodeHexString, decodeBase64String, decodeBase64URLString.
        • For fields of "Array of strings", "Array of numbers", and "Array of floats" types, the following types of conversions are available: append, prepend.

      When using enrichment of events that have the "Event" selected as the "Source kind" setting and the fields of the extended event schema are used as arguments, the following special considerations apply:

      • If the source field is an "Array of strings" field and the target field is a "String" field, the values are written to the target field in the TSV format.

        Example: The SA.StringArray extended event schema field contains values: "string1", "string2", "string3". An event enrichment operation is performed. The result of the operation is written to the DeviceCustomString1 event schema field. As a result of the operation, the DeviceCustomString1 field contains ["string1", "string2", "string3"].

      • If the source field is an "Array of strings" field and the target field is an "Array of strings" field, the values of the source field are appended to the values of the target field and are placed in the target field, with commas (",") used as the separator character.

        Example: The SA.StringArrayOne extended event schema field contains values: "string1", "string2", "string3". An event enrichment operation is performed. The result of the operation is written to the SA.StringArrayTwo event schema field. As a result of the operation, the SA.StringArrayTwo field contains "string1", "string2", "string3".

    • template

      This type of enrichment is used when you need to write a value obtained by processing Go templates into the event field. Settings of this type of enrichment:

      • Put the Go template into the Template field.

        Event field names are passed in the {{.EventField}} format, where EventField is the name of the event field from which the value must be passed to the script.

        Example: Attack on {{.DestinationAddress}} from {{.SourceAddress}}.

      • In the Target field drop-down list, select the KUMA event field to which you want to write the data.

      To convert the data in an array field in a template into the TSV format, you must use the toString function.

      If you are using enrichment of events that have the "Template" type selected as the "Source kind" setting, in which the target field has the "String" type, and the source field is an extended event schema field containing an array of strings, you can use one of the following examples for the template.

      Example:

      {{.SA.StringArrayOne}}

      Example:

      {{- range $index, $element := . SA.StringArrayOne -}}

      {{- if $index}}, {{end}}"{{$element}}"{{- end -}}

  • Debug—you can use this toggle switch to enable logging of service operations.
  • Description—the description of a resource. Up to 4,000 Unicode characters.
• Categorization settings group—used to change the categories of assets indicated in events. There can be several categorization rules. You can add or delete them by clicking the Add categorization or Remove categorization buttons. Only reactive categories can be added to assets or removed from assets.
  • Operation—this drop-down list is used to select the operation to perform on the category:
    • Add—assign the category to the asset.
    • Delete—unbind the asset from the category.
  • Event field—event field that indicates the asset requiring the operation.
  • Category ID—the drop-down list displays a tree of categories, in which you can select a category to perform the operation on. Clicking the row expands the list.
• Active lists update group of settings—used to assign the trigger for one or more operations with active lists. You can click the Add active list action and Delete active list action buttons to add or delete operations with active lists, respectively.

  Available settings:

  • Name (required)—this drop-down list is used to select the Active list resources.
  • Operation (required)—this drop-down list is used to select the operation that must be performed:
    • Sum—add a constant, the value of a correlation event field, or the value of a local variable to the value of the active list.
    • Get—get the Active list entry and write the values of the selected fields into the correlation event.
    • Set—write the values of the selected fields of the correlation event into the Active list by creating a new or updating an existing Active list entry. When the Active list entry is updated, the data is merged and only the specified fields are overwritten.
    • Delete—delete the Active list entry.
  • Key fields (required)—this is the list of event fields used to create the Active list entry. It is also used as the Active list entry key.

    The active list entry key depends on the available fields and does not depend on the order in which they are displayed in the KUMA Console.

  • Mapping (required for Get and Set operations)—used to map Active list fields with events fields. More than one mapping rule can be set.
    • The left field is used to specify the Active list field.

      The field must not contain special characters or numbers only.

    • The middle drop-down list is used to select event fields.
    • The right field can be used to assign a constant to the Active list field is the Set operation was selected.
    • Under Context table update, you can assign the trigger for one or more operations with context tables. You can click "Add context table action" or "Delete context table action" to add or delete operations with context tables.
    • Available settings:
    • Name (required)—this drop-down list is used to select context table resources.
    • Operation (required)—this drop-down list is used to select the operation that must be performed.
    • Sum—add a constant, the value of a correlation event field, or the value of a local variable to the value of the context table. This operation is used only for fields of number and float types.
    • Set—write the values of the selected fields of the correlation event into the context table by creating a new or updating an existing context table entry. When the context table entry is updated, the data is merged and only the specified fields are overwritten.
    • Get—get the fields of the context table and write the values of the specified fields into the correlation event. Table fields of the boolean type and lists of boolean values are excluded from mapping because the event does not contain boolean fields.
    • Merge—append the value of a correlation event field, local variable, or constant to the current value of a field of the context table.
    • Delete—delete the context table entry.
    • Key fields (required)—this is the list of event fields used to create the context table entry. It is also used as the key of the context table entry. As a key field, you can specify an event field or a local variable declared on the "Selectors" tab.
    • The composite key of the context table entry depends only on the values of fields and does not depend on the order in which they are displayed in the KUMA Console.
    • Mapping (required for all operations except "Delete")—used to map context table fields to event fields or variables. More than one mapping rule can be set. You can specify the same context table field multiple times.
    • The left field is used to specify the context table field.
    • The field must not contain a field name that is already used in the mapping, tab characters, special characters, or only numerals. The maximum number of characters is 128. The name cannot begin with an underscore.
    • The middle drop-down list is used to select event fields or a local variable.
    • The right field can be used to assign a constant to the context table field is the "Set" operation was selected. "Merge" or "Sum". The maximum number of characters is 1,024.

Correlators tab

• Add—Used when editing the created correlation rule. You can click Add to open the Correlators window and select a correlator from the list. After you click OK, the rule is linked to the selected correlator. You can select multiple correlators at the same time. The rule is added to the end of the execution queue. If you want to move the rule up in the execution queue, go to Resources - Correlator - <selected correlator> - Edit correlator - Correlation, select the check box next to the relevant rule and click the Move up or Move down buttons to reorder the rules as necessary.
• Delete—Used to unlink the correlation rule from the correlator.

Simple correlation rules

Simple correlation rules are used to define simple sequences of events.

The correlation rule window contains the following settings tabs:

• General—used to specify the main settings of the correlation rule. On this tab, you can select the type of correlation rule.
• Selectors—used to define the conditions that the processed events must fulfill to trigger the correlation rule. Available settings vary based on the selected rule kind.
• Actions—used to set the triggers that will activate when the conditions configured in the Selectors settings block are fulfilled. A correlation rule must have at least one trigger. Available settings vary based on the selected rule type.
• Correlators—used for linking correlators. Available only for created correlation rules that are open for editing.

General tab

• Name (required)—a unique name for this type of resource. Must contain 1 to 128 Unicode characters.
• Tenant (required)—the tenant that owns the correlation rule.
• Type (required)—a drop-down list for selecting the type of correlation rule. Select simple if you want to create a simple correlation rule.
• Propagated fields (required)—event fields used for event selection. If the selector (see below) is triggered, these fields will be written to the correlation event.
• Rate limit—maximum number of times a correlation rule can be triggered per second. The default value is 100.

  If correlation rules employing complex logic for pattern detection are not triggered, this may be due to the specific method used to count rule triggers in KUMA. In this case, try to increase the value of Rate limit to 1000000, for example.

• Priority—base coefficient used to determine the importance of a correlation rule. The default value is Low.
• Description—the description of a resource. Up to 4,000 Unicode characters.

Selectors tab

A rule of the simple kind can have only one selector for which the Settings and Local variables tabs are available.

The Settings tab contains settings with the Filter group of settings:

• Filter (required)—used to set the criteria for determining events that should trigger the selector. You can select an existing filter from the drop-down list or create a new filter.

  Creating a filter in resources

  1. In the Filter drop-down list, select Create new.
  2. If you want to keep the filter as a separate resource, select the Save filter check box.

     In this case, you will be able to use the created filter in various services.

     This check box is cleared by default.

  3. If you selected the Save filter check box, enter a name for the created filter resource in the Name field. The name must contain 1 to 128 Unicode characters.
  4. In the Conditions settings block, specify the conditions that the events must meet:
     1. Click the Add condition button.
     2. In the Left operand and Right operand drop-down lists, specify the search parameters.

        Depending on the data source selected in the Right operand field, you may see fields of additional parameters that you need to use to define the value that will be passed to the filter. For example, when choosing active list you will need to specify the name of the active list, the entry key, and the entry key field.

     3. In the operator drop-down list, select the relevant operator.

        Filter operators

        • =—the left operand equals the right operand.
        • <—the left operand is less than the right operand.
        • <=—the left operand is less than or equal to the right operand.
        • >—the left operand is greater than the right operand.
        • >=—the left operand is greater than or equal to the right operand.
        • inSubnet—the left operand (IP address) is in the subnet of the right operand (subnet).
        • contains—the left operand contains values of the right operand.
        • startsWith—the left operand starts with one of the values of the right operand.
        • endsWith—the left operand ends with one of the values of the right operand.
        • match—the left operand matches the regular expression of the right operand. The RE2 regular expressions are used.
        • hasBit—checks whether the left operand (string or number) contains bits whose positions are listed in the right operand (in a constant or in a list).

          The value to be checked is converted to binary and processed right to left. Chars are checked whose index is specified as a constant or a list.

          If the value being checked is a string, then an attempt is made to convert it to integer and process it in the way described above. If the string cannot be converted to a number, the filter returns False.

        • hasVulnerability—checks whether the left operand contains an asset with the vulnerability and vulnerability severity specified in the right operand.

          If you do not specify the ID and severity of the vulnerability, the filter is triggered if the asset in the event being checked has any vulnerability.

        • inActiveList—this operator has only one operand. Its values are selected in the Key fields field and are compared with the entries in the active list selected from the Active List drop-down list.
        • inDictionary—checks whether the specified dictionary contains an entry defined by the key composed with the concatenated values of the selected event fields.
        • inCategory—the asset in the left operand is assigned at least one of the asset categories of the right operand.
        • inActiveDirectoryGroup—the Active Directory account in the left operand belongs to one of the Active Directory groups in the right operand.
        • TIDetect—this operator is used to find events using CyberTrace Threat Intelligence (TI) data. This operator can be used only on events that have completed enrichment with data from CyberTrace Threat Intelligence. In other words, it can only be used in collectors at the destination selection stage and in correlators.
        • inContextTable—presence of the entry in the specified context table.
        • intersect—presence in the left operand of the list items specified in the right operand.
     4. If necessary, select the do not match case check box. When this check box is selected, the operator ignores the case of the values.

        The selection of this check box does not apply to the InSubnet, InActiveList, InCategory or InActiveDirectoryGroup operators.

        This check box is cleared by default.

     5. If you want to add a negative condition, select If not from the If drop-down list.
     6. You can add multiple conditions or a group of conditions.
  5. If you have added multiple conditions or groups of conditions, choose a search condition (and, or, not) by clicking the AND button.
  6. If you want to add existing filters that are selected from the Select filter drop-down list, click the Add filter button.

     You can view the nested filter settings by clicking the button.

  Filtering based on data from the Extra event field

  Conditions for filters based on data from the Extra event field:

  • Condition—If.
  • Left operand—event field.
  • In this event field, you can specify one of the following values:
    • Extra field.
    • Value from the Extra field in the following format:

      Extra.<field name>

      For example, Extra.app.

      A value of this type is specified manually.

    • Value from the array written to the Extra field in the following format:

      Extra.<field name>.<array element>

      For example, Extra.array.0.

      The values in the array are numbered starting from 0.

      A value of this type is specified manually.

      To work with a value from the Extra field at depth 3 and below, use backquotes ``. For example, `Extra.lev1.lev2.lev3`.

  • Operator – =.
  • Right operand—constant.
  • Value—the value by which you need to filter events.

  The order of conditions specified in the selector filter of the correlation rule is significant and affects system performance. We recommend putting the most unique condition in the first place in the selector filter.

  Consider two examples of selector filters that select successful authentication events in Microsoft Windows.

  Selector filter 1:

  Condition 1. DeviceProduct = Microsoft Windows

  Condition 2. DeviceEventClassID = 4624

  Selector filter 2:

  Condition 1. DeviceEventClassID = 4624

  Condition 2. DeviceProduct = Microsoft Windows

  The order of conditions in Selector filter 2 is preferable because it causes less load on the system.

Select the Local variables tab and click Add variable to declare variables that you want to use within the limits of this correlation rule.

Actions tab

A rule of the simple kind can have only one trigger: On every event. It is activated every time the selector triggers.

Available parameters of the trigger:

• Output—if this check box is selected, the correlation event is sent for post-processing: for enrichment, for a response, and to destinations.
• Loop to correlator—if this check box is selected, the created correlation event is processed by the rule chain of the current correlator. This allows hierarchical correlation.

  If the Output and Loop to correlator check boxes are set, the correlation rule is sent to post-processing first, and then to the selectors of the current correlation rule.

• Do not create alert—if this check box is selected, an alert is not created when this correlation rule is triggered. If you do not want to create an alert when a correlation rule is triggered, but you still want to send a correlation event to the storage, select the Output and No alert check boxes. If you select only the No alert check box, a correlation event is not saved in the storage.
• Enrichment settings group—you can modify the fields of correlation events by using enrichment rules. These enrichment rules are stored in the correlation rule where they were created. You can create multiple enrichment rules. Enrichment rules can be added or deleted by clicking the Add enrichment or Remove enrichment buttons, respectively.
  • Source kind—you can select the type of enrichment in this drop-down list. Depending on the selected type, you may see advanced settings that will also need to be completed.

    Available types of enrichment:

    • constant

      This type of enrichment is used when a constant needs to be added to an event field. Settings of this type of enrichment:

      • In the Constant field, specify the value that should be added to the event field. The value may not be longer than 255 Unicode characters. If you leave this field blank, the existing event field value will be cleared.
      • In the Target field drop-down list, select the KUMA event field to which you want to write the data.

      If you are using the event enrichment functions for extended schema fields of "String", "Number", or "Float" type with a constant, the constant is added to the field.

      If you are using the event enrichment functions for extended schema fields of "Array of strings", "Array of numbers", or "Array of floats" type with a constant, the constant is added to the elements of the array.

    • dictionary

      This type of enrichment is used if you need to add a value from the dictionary of the Dictionary type.

      When this type is selected in the Dictionary name drop-down list, you must select the dictionary that will provide the values. In the Key fields settings block, you have to click the Add field button and select the event fields whose values will be used for dictionary entry selection.

      If you are using event enrichment with the "Dictionary" type selected as the "Source kind" setting, and an array field is specified in the "Key enrichment fields" setting, when an array is passed as the dictionary key, the array is serialized into a string in accordance with the rules of serializing a single value in the TSV format.

      Example: The "Key enrichment fields" setting uses the SA.StringArrayOne extended schema field. The SA.StringArrayOne extended schema field contains 3 elements: "a", "b" and "c". The following value is passed to the dictionary as the key: ['a','b','c'].

      If the "Key enrichment fields" setting uses an extended schema array field and a regular event schema field, the field values are separated by the "|" character when the dictionary is queried.

      Example: The "Key enrichment fields" setting uses two fields: the SA.StringArrayOne extended schema field and the Code field. The SA.StringArrayOne extended schema field contains 3 elements: "a", "b", and "c"; the Code string field contains the character sequence "myCode". The following value is passed to the dictionary as the key: ['a','b','c']|myCode.

    • table

      This type of enrichment is used if you need to add a value from the dictionary of the Table type.

      When this enrichment type is selected in the Dictionary name drop-down list, select the dictionary for providing the values. In the Key fields group of settings, click the Add field button to select the event fields whose values are used for dictionary entry selection.

      In the Mapping table, configure the dictionary fields to provide data and the event fields to receive data:

      • In the Dictionary field column, select the dictionary field. The available fields depend on the selected dictionary resource.
      • In the KUMA field column, select the event field to which the value is written. For some of the selected fields (*custom* and *flex*), in the Label column, you can specify a name for the data written to them.

      New table rows can be added by clicking the Add new element button. Columns can be deleted by clicking the button.

    • event

      This type of enrichment is used when you need to write a value from another event field to the current event field. Settings of this type of enrichment:

      • In the Target field drop-down list, select the KUMA event field to which you want to write the data.
      • In the Source field drop-down list, select the event field whose value will be written to the target field.
      • Clicking the button opens the Conversion window in which you can, by clicking the Add conversion button, create rules for modifying the original data before writing them to the KUMA event fields.

        Available conversions

        Conversions are changes that can be applied to a value before it gets written to the event field. The conversion type is selected from a drop-down list.

        Available conversions:

        • lower—is used to make all characters of the value lowercase
        • upper—is used to make all characters of the value uppercase
        • regexp – used to convert a value using the regular expression RE2. When this conversion type is selected, the field appears where regular expression should be added.
        • substring—is used to extract characters in the position range specified in the Start and End fields. These fields appear when this conversion type is selected.
        • replace—is used to replace specified character sequence with the other character sequence. When this type of conversion is selected, new fields appear:
          • Replace chars—in this field you can specify the character sequence that should be replaced.
          • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
        • trim—used to simultaneously remove the characters specified in the Chars field from the leading and end positions of the value. The field appears when this type of conversion is selected. For example, a trim conversion with the Micromon value applied to Microsoft-Windows-Sysmon results in soft-Windows-Sys.
        • append is used to add the characters specified in the Constant field to the end of the event field value. The field appears when this type of conversion is selected.
        • prepend—used to prepend the characters specified in the Constant field to the start of the event field value. The field appears when this type of conversion is selected.
        • replace with regexp—is used to replace RE2 regular expression results with the character sequence.
          • Expression—in this field you can specify the regular expression which results that should be replaced.
          • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
        • Converting encoded strings to text:
          • decodeHexString—used to convert a HEX string to text.
          • decodeBase64String—used to convert a Base64 string to text.
          • decodeBase64URLString—used to convert a Base64url string to text.

          When converting a corrupted string or if conversion error occur, corrupted data may be written to the event field.

          During event enrichment, if the length of the encoded string exceeds the size of the field of the normalized event, the string is truncated and is not decoded.

          If the length of the decoded string exceeds the size of the event field into which the decoded value is to be written, such a string is truncated to fit the size of the event field.

        Conversions when using the extended event schema

        Whether or not a conversion can be used depends on the type of extended event schema field being used:

        • For an additional field of the "String" type, all types of conversions are available.
        • For fields of the "Number" and "Float" types, the following types of conversions are available: regexp, substring, replace, trim, append, prepend, replaceWithRegexp, decodeHexString, decodeBase64String, decodeBase64URLString.
        • For fields of "Array of strings", "Array of numbers", and "Array of floats" types, the following types of conversions are available: append, prepend.

      When using enrichment of events that have the "Event" selected as the "Source kind" setting and the fields of the extended event schema are used as arguments, the following special considerations apply:

      • If the source field is an "Array of strings" field and the target field is a "String" field, the values are written to the target field in the TSV format.

        Example: The SA.StringArray extended event schema field contains values: "string1", "string2", "string3". An event enrichment operation is performed. The result of the operation is written to the DeviceCustomString1 event schema field. As a result of the operation, the DeviceCustomString1 field contains ["string1", "string2", "string3"].

      • If the source field is an "Array of strings" field and the target field is an "Array of strings" field, the values of the source field are appended to the values of the target field and are placed in the target field, with commas (",") used as the separator character.

        Example: The SA.StringArrayOne extended event schema field contains values: "string1", "string2", "string3". An event enrichment operation is performed. The result of the operation is written to the SA.StringArrayTwo event schema field. As a result of the operation, the SA.StringArrayTwo field contains "string1", "string2", "string3".

    • template

      This type of enrichment is used when you need to write a value obtained by processing Go templates into the event field. Settings of this type of enrichment:

      • Put the Go template into the Template field.

        Event field names are passed in the {{.EventField}} format, where EventField is the name of the event field from which the value must be passed to the script.

        Example: Attack on {{.DestinationAddress}} from {{.SourceAddress}}.

      • In the Target field drop-down list, select the KUMA event field to which you want to write the data.

      To convert the data in an array field in a template into the TSV format, you must use the toString function.

      If you are using enrichment of events that have the "Template" type selected as the "Source kind" setting, in which the target field has the "String" type, and the source field is an extended event schema field containing an array of strings, you can use one of the following examples for the template.

      Example:

      {{.SA.StringArrayOne}}

      Example:

      {{- range $index, $element := . SA.StringArrayOne -}}

      {{- if $index}}, {{end}}"{{$element}}"{{- end -}}

  • Debug—you can use this toggle switch to enable logging of service operations.
  • Description—the description of a resource. Up to 4,000 Unicode characters.
  • Filter settings block—lets you select which events will be forwarded for enrichment. Configuration is performed as described above.
• Categorization settings group—used to change the categories of assets indicated in events. There can be several categorization rules. You can add or delete them by clicking the Add categorization or Remove categorization buttons. Only reactive categories can be added to assets or removed from assets.
  • Operation—this drop-down list is used to select the operation to perform on the category:
    • Add—assign the category to the asset.
    • Delete—unbind the asset from the category.
  • Event field—event field that indicates the asset requiring the operation.
  • Category ID—the drop-down list displays a tree of categories, in which you can select a category to perform the operation on. Clicking the row expands the list.
• Active lists update group of settings—used to assign the trigger for one or more operations with active lists. You can click the Add active list action and Delete active list action buttons to add or delete operations with active lists, respectively.

  Available settings:

  • Name (required)—this drop-down list is used to select the active list.
  • Operation (required)—this drop-down list is used to select the operation that must be performed:
    • Sum—add a constant, the value of a correlation event field, or the value of a local variable to the value of the active list.
    • Get—get the Active list entry and write the values of the selected fields into the correlation event.
    • Set—write the values of the selected fields of the correlation event into the Active list by creating a new or updating an existing Active list entry. When the Active list entry is updated, the data is merged and only the specified fields are overwritten.
    • Get—get the Active list entry and write the values of the selected fields into the correlation event.
    • Delete—delete the Active list entry.
  • Key fields (required)—this is the list of event fields used to create the Active list entry. It is also used as the Active list entry key.

    The active list entry key depends on the available fields and does not depend on the order in which they are displayed in the KUMA Console.

  • Mapping (required for Get and Set operations)—used to map Active list fields with events fields. More than one mapping rule can be set.
    • The left field is used to specify the Active list field.

      The field must not contain special characters or numbers only.

    • The middle drop-down list is used to select event fields.
    • The right field can be used to assign a constant to the Active list field is the Set operation was selected.
• Under Context table update, you can assign the trigger for one or more operations with context tables. You can click "Add context table action" or "Delete context table action" to add or delete operations with context tables.
• Available settings:
• Name (required)—this drop-down list is used to select context table resources.
• Operation (required)—this drop-down list is used to select the operation that must be performed.
• Sum—add a constant, the value of a correlation event field, or the value of a local variable to the value of the context table. This operation is used only for fields of number and float types.
• Set—write the values of the selected fields of the correlation event into the context table by creating a new or updating an existing context table entry. When the context table entry is updated, the data is merged and only the specified fields are overwritten.
• Get—get the fields of the context table and write the values of the specified fields into the correlation event. Table fields of the boolean type and lists of boolean values are excluded from mapping because the event does not contain boolean fields.
• Merge—append the value of a correlation event field, local variable, or constant to the current value of a field of the context table.
• Delete—delete the context table entry.
• Key fields (required)—this is the list of event fields used to create the context table entry. It is also used as the key of the context table entry. As a key field, you can specify an event field or a local variable declared on the "Selectors" tab.
• The composite key of the context table entry depends only on the values of fields and does not depend on the order in which they are displayed in the KUMA Console.
• Mapping (required for all operations except "Delete")—used to map context table fields to event fields or variables. More than one mapping rule can be set. You can specify the same context table field multiple times.
• The left field is used to specify the context table field.
• The field must not contain a field name that is already used in the mapping, tab characters, special characters, or only numerals. The maximum number of characters is 128. The name cannot begin with an underscore.
• The middle drop-down list is used to select event fields or a local variable.
• The right field can be used to assign a constant to the context table field is the "Set" operation was selected. "Merge" or "Sum". The maximum number of characters is 1,024.