User guide

This chapter provides information about managing the KUMA SIEM system.

KUMA resources

Resources are KUMA components that contain parameters for implementing various functions: for example, establishing a connection with a given web address or converting data according to certain rules. Like parts of an erector set, these components are assembled into resource sets for services that are then used as the basis for creating KUMA services.

Resources are contained in the Resources section, Resources block of KUMA web interface. The following resource types are available:

• Correlation rules—resources of this type contain rules for identifying event patterns that indicate threats. If the conditions specified in these resources are met, a correlation event is generated.
• Normalizers—resources of this type contain rules for converting incoming events into the format used by KUMA. After processing in the normalizer, the "raw" event becomes normalized and can be processed by other KUMA resources and services.
• Connectors—resources of this type contain settings for establishing network connections.
• Aggregation rules—resources of this type contain rules for combining several basic events of the same type into one aggregation event.
• Enrichment rules—resources of this type contain rules for supplementing events with information from third-party sources.
• Destinations—resources of this type contain settings for forwarding events to a destination for further processing or storage.
• Filters—resources of this type contain conditions for rejecting or selecting individual events from the stream of events.
• Response rules—resources of this type are used in correlators to, for example, execute scripts or launch Kaspersky Security Center tasks when certain conditions are met.
• Notification templates—resources of this type are used when sending notifications about new alerts.
• Active lists—resources of this type are used by correlators for dynamic data processing when analyzing events according to correlation rules.
• Dictionaries—resources of this type are used to store keys and their values, which may be required by other KUMA resources and services.
• Proxies—resources of this type contain settings for using proxy servers.
• Secrets—resources of this type are used to securely store confidential information (such as credentials) that KUMA needs to interact with external services.

When you click on a resource type, a window opens displaying a table with the available resources of this type. The resource table contains the following columns:

• Name—the name of a resource. Can be used to search for resources and sort them.
• Updated—the date and time of the last update of a resource. Can be used to sort resources.
• Created by—the name of the user who created a resource.
• Description—the description of a resource.

The maximum table size is not limited. If you want to select all resources, scroll to the end of the table and select the Select all check box, which selects all available resources in the table.

Resources can be organized into folders. The folder structure is displayed in the left part of the window: root folders correspond to tenants and contain a list of all resources of the tenant. All other folders nested within the root folder display the resources of an individual folder. When a folder is selected, the resources it contains are displayed as a table in the right pane of the window.

Resources can be created, edited, copied, moved from one folder to another, and deleted. Resources can also be exported and imported.

KUMA comes with a set of predefined resources, which can be identified by the "[OOTB]<resource_name>" name. OOTB resources are protected from editing.

If you want to adapt a predefined OOTB resource to your organization's infrastructure:

1. In the Resources-<resource type> section, select the OOTB resource that you want to edit.
2. In the upper part of the KUMA web interface, click Duplicate, then click Save.
3. A new resource named "[OOTB]<resource_name> - copy" is displayed in the web interface.
4. Edit the copy of the predefined resource as necessary and save your changes.

The adapted resource is available for use.

Operations with resources

To manage KUMA resources, you can create, move, copy, edit, delete, import, and export them. These operations are available for all resources, regardless of the resource type.

KUMA resources reside in folders. You can add, rename, move, or delete resource folders.

Creating, renaming, moving, and deleting resource folders

Resources can be organized into folders. The folder structure is displayed in the left part of the window: root folders correspond to tenants and contain a list of all resources of the tenant. All other folders nested within the root folder display the resources of an individual folder. When a folder is selected, the resources it contains are displayed as a table in the right pane of the window.

You can create, rename, move and delete folders.

To create a folder:

1. Select the folder in the tree where the new folder is required.
2. Click the Add folder button.

The folder will be created.

To rename a folder:

1. Locate required folder in the folder structure.
2. Hover over the name of the folder.

   The icon will appear near the name of the folder.

3. Open the drop-down list and select Rename.

   The folder name will become active for editing.

4. Enter the new folder name and press ENTER.

   The folder name cannot be empty.

The folder will be renamed.

To move a folder,

Drag and drop the folder to a required place in folder structure by clicking its name.

Folders cannot be dragged from one tenant to another.

To delete a folder:

1. Locate required folder in the folder structure.
2. Hover over the name of the folder.

   The icon will appear near the name of the folder.

3. Open the drop-down list and select Delete.

   The conformation window appears.

4. Click OK.

The folder will be deleted.

The program does not delete folders that contain files or subfolders.

Creating, duplicating, moving, editing, and deleting resources

You can create, move, copy, edit, and delete resources.

To create the resource:

1. In the Resources → <resource type> section, select or create a folder where you want to add the new resource.

   Root folders correspond to tenants. For a resource to be available to a specific tenant, it must be created in the folder of that tenant.

2. Click the Add <resource type> button.

   The window for configuring the selected resource type opens. The available configuration parameters depend on the resource type.

3. Enter a unique resource name in the Name field.
4. Specify the required parameters (marked with a red asterisk).
5. If necessary, specify the optional parameters (not required).
6. Click Save.

The resource will be created and available for use in services and other resources.

To move the resource to a new folder:

1. In the Resources → <resource type> section, find the required resource in the folder structure.
2. Select the check box near the resource you want to move. You can select multiple resources.

   The icon appears near the selected resources.

3. Use the icon to drag and drop resources to the required folder.

The resources will be moved to the new folders.

You can only move resources to folders of the tenant in which the resources were created. Resources cannot be moved to another tenant's folders.

To copy the resource:

1. In the Resources → <resource type> section, find the required resource in the folder structure.
2. Select the check box next to the resource that you want to copy and click Duplicate.

   A window opens with the settings of the resource that you have selected for copying. The available configuration parameters depend on the resource type.

   The <selected resource name> - copy value is displayed in the Name field.

3. Make the necessary changes to the parameters.
4. Enter a unique name in the Name field.
5. Click Save.

The copy of the resource will be created.

To edit the resource:

1. In the Resources → <resource type> section, find the required resource in the folder structure.
2. Select the resource.

   A window with the settings of the selected resource opens. The available configuration parameters depend on the resource type.

3. Make the necessary changes to the parameters.
4. Click Save.

The resource will be updated. If this resource is used in a service, restart the service to apply the new settings.

To delete the resource:

1. In the Resources → <resource type> section, find the required resource in the folder structure.
2. Select the check box next to the resource that you want to delete and click Delete.

   A confirmation window opens.

3. Click OK.

The resource will be deleted.

Updating resources

Kaspersky regularly releases packages with resources that can be imported from the repository. You can specify an email address in the settings of the Repository update task. After the first execution of the task, KUMA starts sending notifications about the packages available for update to the specified address. You can update the repository, analyze the contents of each update, and decide if to import and deploy the new resources in the operating infrastructure. KUMA supports updates from Kaspersky servers and from custom sources, including offline update using the update mirror mechanism. If you have other Kaspersky products in the infrastructure, you can connect KUMA to existing update mirrors. The update subsystem expands KUMA capabilities to respond to the changes in the threat landscape and the infrastructure. The capability to use it without direct Internet access ensures the privacy of the data processed by the system.

To update resources, perform the following steps:

1. Update the repository to deliver the resource packages to the repository. The repository update is available in two modes:
   • Automatic update
   • Manual update
2. Import the resource packages from the updated repository into the tenant.

For the service to start using the resources, make sure that the updated resources are mapped after performing the import. If necessary, link the resources to collectors, correlators, or agents, and update the settings.

To enable automatic update:

1. In the Settings → Repository update section, configure the Data refresh interval in hours. The default value is 24 hours.
2. Specify the Update source. The following options are available:
   • Kaspersky update servers

     

     The servers are located in many countries around the world, which ensures the high reliability of the update. If the update cannot be performed from one server, KUMA switches to another server.

     .

     You can view the list of servers in the Knowledge Base, article 15998.

   • Custom source:
     • The URL to the shared folder on the HTTP server.
     • The full path to the local folder on the host where the KUMA Core is installed.

       If a local folder is used, the kuma system user must have read access to this folder and its contents.

3. Specify the Emails for notification by clicking the Add button. The notifications that new packages or new versions of the packages imported into the tenant are available in the repository are sent to the specified email addresses.

   If you specify the email address of a KUMA user, the Receive email notifications check box must be selected in the user profile. For emails that do not belong to any KUMA user, the messages are received without additional settings. The settings for connecting to the SMTP server must be specified in all cases.

4. Click Save. The update task starts shortly. Then the task restarts according to the schedule.

To manually start the repository update:

1. To disable automatic updates, in the Settings → Repository update section, select the Disable automatic update check box. This check box is cleared by default. You can also start a manual repository update without disabling automatic update. Starting an update manually does not affect the automatic update schedule.
2. Specify the Update source. The following options are available:
   • Kaspersky update servers.
   • Custom source:
     • The URL to the shared folder on the HTTP server.
     • The full path to the local folder on the host with the KUMA Core

       If a local folder is used, the kuma user must have access to this folder and its contents.

3. Specify the Emails for notification by clicking the Add button. The notifications that new packages or new versions of the packages imported into the tenant are available in the repository are sent to the specified email addresses.

   If you specify the email address of a KUMA user, the Receive email notifications check box must be selected in the user profile. For emails that do not belong to any KUMA user, the messages are received without additional settings. The settings for connecting to the SMTP server must be specified in all cases.

4. Click Run update. Thus, you simultaneously save the settings and manually start the Repository update task.

Configuring a custom source using Kaspersky Update Utility

You can update resources without Internet access by using a custom update source via the Kaspersky Update Utility.

Configuration consists of the following steps:

1. Configuring a custom source using Kaspersky Update Utility:
   1. Installing and configuring Kaspersky Update Utility on one of the computers in the corporate LAN.
   2. Configuring copying of updates to a shared folder in Kaspersky Update Utility settings.
2. Configuring update of the KUMA repository from a custom source.

Configuring a custom source using Kaspersky Update Utility:

You can download the Kaspersky Update Utility distribution kit from the Kaspersky Technical Support website.

1. In Kaspersky Update Utility, enable the download of updates for KUMA 2.1:
   • Under Applications – Perimeter control, select the check box next to KUMA 2.1 to enable the update capability.
   • If you work with Kaspersky Update Utility using the command line, add the following line to the [ComponentSettings] section of the updater.ini configuration file or specify the true value for an existing line:

     KasperskyUnifiedMonitoringAndAnalysisPlatform_2_1=true

2. In the Downloads section, specify the update source. By default, Kaspersky update servers are used as the update source.
3. In the Downloads section, in the Update folders group of settings, specify the shared folder for Kaspersky Update Utility to download updates to. The following options are available:
   • Specify the local folder on the host where Kaspersky Update Utility is installed. Deploy the HTTP server for distributing updates and publish the local folder on it. In KUMA, in the Settings → Repository update → Custom source section, specify the URL of the local folder published on the HTTP server.
   • Specify the local folder on the host where Kaspersky Update Utility is installed. Make this local folder available over the network. Mount the network-accessible local folder on the host where KUMA is installed. In KUMA, in the Settings → Repository update → Custom source section, specify the full path to the local folder.

For detailed information about working with Kaspersky Update Utility, refer to the Kaspersky Knowledge Base.

Exporting resources

If shared resources are hidden for a user, the user cannot export shared resources or resources that use shared resources.

To export resources:

1. In the Resources section, click Export resources.

   The Export resources window opens with the tree of all available resources.

2. In the Password field enter the password that must be used to protect exported data.
3. In the Tenant drop-down list, select the tenant whose resources you want to export.
4. Check boxes near the resources you want to export.

   If selected resources are linked to other resources, linked resources will be exported, too.

5. Click the Export button.

The resources in a password-protected file are saved on your computer using your browser settings. The Secret resources are exported blank.

Importing resources

To import resources:

1. In the Resources section, click Import resources.

   The Resource import window opens.

2. In the Tenant drop-down list, select the tenant to assign the imported resources to.
3. In the Import source drop-down list, select one of the following options:
   • File

     If you select this option, enter the password and click the Import button.

   • Repository

     If you select this option, a list of packages available for import is displayed. We recommend starting the import with the "OOTB resources for KUMA 2.1" package and then importing the packages one by one. If you receive a "Database error" error when importing packages, try importing the package mentioned in the error message again, selecting only the specified package for import. You can also configure automatic updates.

     You can select one or more packages to import and click the Import button.

     The imported resources can only be deleted. To rename, edit or move an imported resource, make a copy of the resource using the Duplicate button and perform the desired actions with the resource copy. When importing future versions of the package, the duplicate is not updated because it is a separate object.

4. Resolve the conflicts between the resources imported from the file and the existing resources if they occur. Read more about resource conflicts below.
   1. If the name, type, and guid of an imported resource fully match the name, type, and guid of an existing resource, the Conflicts window opens with the table displaying the type and the name of the conflicting resources. Resolve displayed conflicts:
      • To replace the existing resource with a new one, click Replace.

        To replace all conflicting resources, click Replace all.

      • To leave the existing resource, click Skip.

        To keep all existing resources, click Skip all.

   2. Click the Resolve button.

   The resources are imported to KUMA. The Secret resources are imported blank.

About conflict resolving

When resources are imported into KUMA from a file, they are compared with existing resources; the following parameters are compared:

• Name and kind. If an imported resource's name and kind parameters match those of the existing one, the imported resource's name is automatically changed.
• ID. If identifiers of two resources match, a conflict appears that must be resolved by the user. This could happen when you import resources to the same KUMA server from which they were exported.

When resolving a conflict you can choose either to replace existing resource with the imported one or to keep exiting resource, skipping the imported one.

Some resources are linked: for example, in some types of connectors, the connector secret must be specified. The secrets are also imported if they are linked to a connector. Such linked resources are exported and imported together.

Special considerations of import:

1. Resources are imported to the selected tenant.
2. Starting with version 2.1.3, if a linked resource is in the Shared tenant, it ends up in the Shared tenant when imported.
3. In version 2.1.3 and later, in the Conflicts window, the Parent column always displays the top-most parent resource among those that were selected during import.
4. In version 2.1.3 and later, if a conflict occurs during import and you choose to replace existing resource with a new one, it would mean that all the other resources linked to the one being replaced are automatically replaced with the imported resources.

Known bugs in version 2.1.3:

1. The linked resource ends up in the tenant specified during the import, and not in the Shared tenant, as indicated in the Conflicts window, under the following conditions:
   1. The associated resource is initially in the Shared tenant.
   2. In the Conflicts window, you select Skip for all parent objects of the linked resource from the Shared tenant.
   3. You leave the linked resource from the Shared tenant for replacement.
2. After importing, the categories do not have a tenant specified in the filter under the following conditions:
   1. The filter contains linked asset categories from different tenants.
   2. Asset category names are the same.
   3. You are importing this filter with linked asset categories to a new server.
3. In Tenant 1, the name of the asset category is duplicated under the following conditions:
   1. in Tenant 1, you have a filter with linked asset categories from Tenant 1 and the Shared tenant.
   2. The names of the linked asset categories are the same.
   3. You are importing such a filter from Tenant 1 to the Shared tenant.
4. You cannot import conflicting resources into the same tenant.

   The error "Unable to import conflicting resources into the same tenant" means that the imported package contains conflicting resources from different tenants and cannot be imported into the Shared tenant.

   Solution: Select a tenant other than Shared to import the package. In this case, during the import, resources originally located in the Shared tenant are imported into the Shared tenant, and resources from the other tenant are imported into the tenant selected during import.

5. Only the general administrator can import categories into the Shared tenant.

   The error "Only the general administrator can import categories into the Shared tenant" means that the imported package contains resources with linked shared asset categories. You can see the categories or resources with linked shared asset categories in the KUMA Core log. Path to the Core log:

   /opt/kaspersky/kuma/core/log/core

   Solution. Choose one of the following options:

   • Do not import resources to which shared categories are linked: clear the check boxes next to the relevant resources.
   • Perform the import under a General administrator account.
6. Only the general administrator can import resources into the Shared tenant.

   The error "Only the general administrator can import resources into the Shared tenant" means that the imported package contains resources with linked shared resources. You can see the resources with linked shared resources in the KUMA Core log. Path to the Core log:

   /opt/kaspersky/kuma/core/log/core

   Solution. Choose one of the following options:

   • Do not import resources that have linked resources from the Shared tenant, and the shared resources themselves: clear the check boxes next to the relevant resources.
   • Perform the import under a General administrator account.

Destinations

Destinations define network settings for sending normalized events. Collectors and correlators use destinations to describe where to send processed events. Typically, the destination points are the correlator and storage.

The settings of destinations are configured on two tabs: Basic settings and Advanced settings. The available settings depend on the selected type of destination:

• nats-jetstream—used for NATS communications.
• tcp—used for communications over TCP.
• http—used for HTTP communications.
• diode—used to transmit events using a data diode.
• kafka—used for Kafka communications.
• file—used for writing to a file.
• storage—used to transmit data to the storage.
• correlator—used to transmit data to the correlator.

Nats type

The nats-jetstream type is used for NATS communications.

Basic settings tab

Advanced settings tab

Tcp type

The tcp type is used for TCP communications.

Basic settings tab

Advanced settings tab

Http type

The http type is used for HTTP communications.

Basic settings tab

Advanced settings tab

Diode type

The diode type is used to transmit events using a data diode.

Basic settings tab

Advanced settings tab

Kafka type

The kafka type is used for Kafka communications.

Basic settings tab

Advanced settings tab

File type

The file type is used for writing data to a file.

If you delete a destination of the 'file' type used in a service, that service must be restarted.

Basic settings tab

Advanced settings tab

Storage type

The storage type is used to transmit data to the storage.

Basic settings tab

Advanced settings tab

Correlator type

The correlator type is used to transmit data to the correlator.

Basic settings tab

Advanced settings tab

Predefined destinations

Destinations listed in the table below are included in the KUMA distribution kit.

Predefined destinations

Working with events

In the Events section of the KUMA web interface, you can inspect events received by the program to investigate security threats or create correlation rules. The events table displays the data received after the SQL query is executed.

Events can be sent to the correlator for a retroscan.

The event date format depends on the localization language selected in the application settings. Possible date format options:

• English localization: YYYY-MM-DD.
• Russian localization: DD.MM.YYYY.

Filtering and searching events

The Events section of the KUMA web interface does not show any data by default. To view events, you need to define an SQL query in the search field and click the button. The SQL query can be entered manually or it can be generated using a query builder.

Data aggregation and grouping is supported in SQL queries.

You can add filter conditions to an already generated SQL query in the window for viewing statistics, the events table, and the event details area:

• Changing a query from the Statistics window

  To change the filtering settings in the Statistics window:

  1. Open Statistics details area by using one of the following methods:
     • In the drop-down list in the top right corner of the events table select Statistics.
     • In the events table click any value and in the opened context menu select Statistics.

     The Statistics details area appears in the right part of the web interface window.

  2. Open the drop-down list of the relevant parameter and hover your mouse cursor over the necessary value.
  3. Use the plus and minus signs to change the filter settings by doing one of the following:
     • If you want the events selection to include only events with the selected value, click the icon.
     • If you want the events selection to exclude all events with the selected value, click the icon.

  As a result, the filter settings and the events table will be updated, and the new search query will be displayed in the upper part of the screen.

• Changing a query from the events table

  To change the filtering settings in the events table:

  1. In the Events section of the KUMA web interface, click any event parameter value in the events table.
  2. In the opened menu, select one of the following options:
     • If you want the table to show only events with the selected value, select Filter by this value.
     • If you want to exclude all events with the selected value from the table, select Exclude from filter.

  As a result, the filter settings and the events table are updated, and the new search query is displayed in the upper part of the screen.

• Changing a query from the Event details area

  To change the filter settings in the event details area:

  1. In the Events section of the KUMA web interface, click the relevant event.

     The Event details area appears in the right part of the window.

  2. Change the filter settings by using the plus or minus icons next to the relevant settings:
     • If you want the events selection to include only events with the selected value, click the icon.
     • If you want the events selection to exclude all events with the selected value, click the icon.

  As a result, the filter settings and the events table will be updated, and the new search query will be displayed in the upper part of the screen.

After modifying a query, all query parameters, including the added filter conditions, are transferred to the query builder and the search field.

When you switch to the query builder, the parameters of a query entered manually in the search field are not transferred to the builder, so you will need to create your query again. Also, the query created in the builder does not overwrite the query that was entered into the search string until you click the Apply button in the builder window.

In the SQL query input field, you can enable the display of control characters.

You can also filter events by time period. Search results can be automatically updated.

The filter configuration can be saved. Existing filter configurations can be deleted.

Filter functions are available for users regardless of their roles.

When accessing certain event fields with IDs, KUMA returns the corresponding names.

For more details on SQL, refer to the ClickHouse documentation. See also KUMA operator usage and supported functions.

Selecting Storage

Events that are displayed in the Events section of the KUMA web interface are retrieved from storage (from the ClickHouse cluster). Depending on the demands of your company, you may have more than one Storage. However, you can only receive events from one Storage at a time, so you must specify which one you want to use.

To select the Storage you want to receive events from,

In the Events section of the KUMA web interface, open the drop-down list and select the relevant storage cluster.

Now events from the selected storage are displayed in the events table. The name of the selected storage is displayed in the drop-down list.

The drop-down list displays only the clusters of tenants available to the user, and the cluster of the main tenant.

Generating an SQL query using a builder

In KUMA, you can use a query builder to generate an SQL query for filtering events.

To generate an SQL query using a builder:

1. In the Events section of the KUMA web interface, click the button.

   The filter constructor window opens.

2. Generate a search query by providing data in the following parameter blocks:

   SELECT—event fields that should be returned. The * value is selected by default, which means that all available event fields must be returned. To make viewing the search results easier, select the necessary fields in the drop-down list. In this case, the data only for the selected fields is displayed in the table. Note that Select * increases the duration of the request execution, but eliminates the need to manually indicate the fields in the request.

   When selecting an event field, you can use the field on the right of the drop-down list to specify an alias for the column of displayed data, and you can use the right-most drop-down list to select the operation to perform on the data: count, max, min, avg, sum.

   If you are using aggregation functions in a query, you cannot customize the events table display, sort events in ascending or descending order, or receive statistics.

   When filtering by alert-related events in alert investigation mode, you cannot perform operations on the data of event fields or assign names to the columns of displayed data.

   • FROM—data source. Select the events value.
   • WHERE—conditions for filtering events.

     Conditions and groups of conditions can be added by using the Add condition and Add group buttons. The AND operator value is selected by default in a group of conditions, but the operator can be changed by clicking on this value. Available values: AND, OR, NOT. The structure of conditions and condition groups can be changed by using the icon to drag and drop expressions.

     Adding filter conditions:

     1. In the drop-down list on the left, select the event field that you want to use for filtering.
     2. Select the necessary operator from the middle drop-down list. The available operators depend on the type of value of the selected event field.
     3. Enter the value of the condition. Depending on the selected type of field, you may have to manually enter the value, select it from the drop-down list, or select it on the calendar.

     Filter conditions can be deleted by using the button. Group conditions are deleted using the Delete group button.

   • GROUP BY—event fields or aliases to be used for grouping the returned data.

     If you are using data grouping in a query, you cannot customize the events table display, sort events in ascending or descending order, receive statistics, or perform a retroscan.

     When filtering by alert-related events in alert investigation mode, you cannot group the returned data.

   • ORDER BY—columns used as the basis for sorting the returned data. In the drop-down list on the right, you can select the necessary order: DESC—descending, ASC—ascending.
   • LIMIT—number of strings displayed in the table.

     The default value is 250.

     If you are filtering events by user-defined period and the number of strings in the search results exceeds the defined value, you can click the Show next records button to display additional strings in the table. This button is not displayed when filtering events by the standard period.

3. Click Apply.

   The current SQL query will be overwritten. The generated SQL query is displayed in the search field.

   If you want to reset the builder settings, click the Default query button.

   If you want to close the builder without overwriting the existing query, click the button.

4. Click the button to display the data in the table.

The table will display the search results based on the generated SQL query.

When switching to another section of the web interface, the query generated in the builder is not preserved. If you return to the Events section from another section, the builder will display the default query.

For more details on SQL, refer to the ClickHouse documentation. See also KUMA operator usage and supported functions.

Manually creating an SQL query

You can use the search string to manually create SQL queries of any complexity for filtering events.

To manually generate an SQL query:

1. Go to the Events section of the KUMA web interface.

   An input form opens.

2. Enter your SQL query into the input field.
3. Click the button.

You will see a table of events that satisfy the criteria of your query. If necessary, you can filter events by period.

Supported functions and operators

• SELECT—event fields that should be returned.

  For SELECT fields, the program supports the following functions and operators:

  • Aggregation functions: count, avg, max, min, sum.
  • Arithmetic operators: +, -, *, /, <, >, =, !=, >=, <=.

    You can combine these functions and operators.

    If you are using aggregation functions in a query, you cannot customize the events table display, sort events in ascending or descending order, or receive statistics.

• DISTINCT–removes duplicates from the result of a SELECT statement. You must use the following notation: SELECT DISTINCT SourceAddress as Addresses FROM <rest of the query>.
• FROM—data source.

  When creating a query, you need to specify the events value as the data source.

• WHERE—conditions for filtering events.
  • AND, OR, NOT, =, !=, >, >=, <, <=
  • IN
  • BETWEEN
  • LIKE
  • ILIKE
  • inSubnet
  • match (the re2 syntax of regular expressions is used in queries, special characters must be shielded with "\")
• GROUP BY—event fields or aliases to be used for grouping the returned data.

  If you are using data grouping in a query, you cannot customize the events table display, sort events in ascending or descending order, receive statistics, or perform a retroscan.

• ORDER BY—columns used as the basis for sorting the returned data.

  Possible values:

  • DESC—descending order.
  • ASC—ascending order.
• OFFSET—skip the indicated number of lines before printing the query results output.
• LIMIT—number of strings displayed in the table.

  The default value is 250.

  If you are filtering events by user-defined period and the number of strings in the search results exceeds the defined value, you can click the Show next records button to display additional strings in the table. This button is not displayed when filtering events by the standard period.

  Example queries: SELECT * FROM `events` WHERE Type IN ('Base', 'Audit') ORDER BY Timestamp DESC LIMIT 250 In the events table, all events with the Base and Audit type are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250. SELECT * FROM `events` WHERE BytesIn BETWEEN 1000 AND 2000 ORDER BY Timestamp ASC LIMIT 250 All events of the events table for which the BytesIn field contains a value of received traffic in the range from 1,000 to 2,000 bytes are sorted by the Timestamp column in ascending order. The number of strings that can be displayed in the table is 250. SELECT * FROM `events` WHERE Message LIKE '%ssh:%' ORDER BY Timestamp DESC LIMIT 250 In the events table, all events whose Message field contains data corresponding to the defined %ssh:% template in lowercase are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250. SELECT * FROM `events` WHERE inSubnet(DeviceAddress, '00.0.0.0/00') ORDER BY Timestamp DESC LIMIT 250 In the events table, all events for the hosts that are in the 00.0.0.0/00 subnet are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250. SELECT * FROM `events` WHERE match(Message, 'ssh.*') ORDER BY Timestamp DESC LIMIT 250 In the events table, all events whose Message field contains text corresponding to the ssh.* template are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250. SELECT max(BytesOut) / 1024 FROM `events` Maximum amount of outbound traffic (KB) for the selected time period. SELECT count(ID) AS "Count", SourcePort AS "Port" FROM `events` GROUP BY SourcePort ORDER BY Port ASC LIMIT 250 Number of events and port number. Events are grouped by port number and sorted by the Port column in ascending order. The number of strings that can be displayed in the table is 250. The ID column in the events table is named Count, and the SourcePort column is named Port.

If you want to use a special character in a query, you need to escape this character by placing a backslash (\) character in front of it.

When creating a normalizer for events, you can choose whether to retain the field values of the raw event. The data is stored in the Extra event field. This field is searched for events by using the LIKE operator.

When switching to the query builder, the query parameters that were manually entered into the search string are not transferred to the builder so you will need to create your query again. Also, the query created in the builder does not overwrite the query that was entered into the search string until you click the Apply button in the builder window.

Aliases must not contain spaces.

For more details on SQL, refer to the ClickHouse documentation. See also the supported ClickHouse functions.

Filtering events by period

In KUMA, you can specify the time period to display events from.

To filter events by period:

1. In the Events section of the KUMA web interface, open the Period drop-down list in the upper part of the window.
2. If you want to filter events based on a standard period, select one of the following:
   • 5 minutes
   • 15 minutes
   • 1 hour
   • 24 hours
   • In period

     If you select this option, use the opened calendar to select the start and end dates of the period and click Apply Filter. The date and time format depends on your operating system's settings. You can also manually change the date values if necessary.

3. Click the button.

When the period filter is set, only events registered during the specified time interval will be displayed. The period will be displayed in the upper part of the window.

You can also configure the display of events by using the events histogram that is displayed when you click the button in the upper part of the Events section. Events are displayed if you click the relevant data series or select the relevant time period and click the Show events button.

Displaying names instead of IDs

When accessing certain event fields with IDs, KUMA returns the corresponding names rather than IDs. This helps make the information more readable. For example, if you access the TenantID event field (which stores the tenant ID), you get the value of the TenantName event field (which stores the tenant name).

When exporting events, values of both fields are written to the file, the ID as well as the name.

The table below lists the fields that are substituted when accessed:

Substitution does not occur if an alias is assigned to the field in the SQL query. Examples:

• SELECT TenantID FROM `events` LIMIT 250 — in the search result, the name of the tenant is displayed in the TenantID field.
• SELECT TenantID AS Tenant_name FROM `events` LIMIT 250 — in the search result, the tenant ID will be displayed in the Tenant_name field.

Presets

You can use

presets

A preset is a predefined set of event fields that you can use to simplify working with queries.

To create a preset:

1. In the Events section, click the icon.
2. In the window that opens, on the Event field columns tab, select the required fields.

   To simplify your search, you can start typing the field name in the Search area. 

3. To save the selected fields, click Save current preset.

   The New preset window opens.

4. In the window that opens, specify the Name of the preset, and in the drop-down list, select the Tenant.
5. Click Save.

   The preset is created and saved.

To apply a preset:

1. In the query entry field, enter Select *.
2. In the Events section of the KUMA web interface, click the icon.
3. In the opened window, use the Presets tab to select the relevant preset and click the button.

   The fields from the selected preset are added to the SQL query field, and the columns are added to the table. No changes are made in Builder.

4. Click to execute the query.

   After the query execution completes, the columns are filled in.

Limiting the complexity of queries in alert investigation mode

When investigating an alert, the complexity of SQL queries for event filtering is limited if the Related to alert option is selected in the drop-down list. If this is the case, only the functions and operators listed below are available for event filtering.

If the All events option is selected from the drop-down list, these limitations are not applied.

• SELECT
  • The * character is used as a wildcard to represent any number of characters.
• WHERE
  • AND, OR, NOT, =, !=, >, >=, <, <=
  • IN
  • BETWEEN
  • LIKE
  • inSubnet

  Examples:

  • WHERE Type IN ('Base', 'Correlated')
  • WHERE BytesIn BETWEEN 1000 AND 2000
  • WHERE Message LIKE '%ssh:%'
  • WHERE inSubnet(DeviceAddress, '10.0.0.1/24')
• ORDER BY

  Sorting can be done by column.

• OFFSET

  Skip the indicated number of lines before printing the query results output.

• LIMIT

  The default value is 250.

  If you are filtering events by user-defined period and the number of strings in the search results exceeds the defined value, you can click the Show next records button to display additional strings in the table. This button is not displayed when filtering events by the standard period.

When filtering by alert-related events in alert investigation mode, you cannot group the returned data. When filtering by alert-related events in alert investigation mode, you cannot perform operations on the data of event fields or assign names to the columns of displayed data.

Saving and selecting events filter configuration

In KUMA, you can save a filter configuration and use it in the future. Other users can also use the saved filters if they have the appropriate access rights. When saving a filter, you are saving the configured settings of all the active filters at the same time, including the time-based filter, query builder, and the events table settings. Search queries are saved on the KUMA Core server and are available to all KUMA users of the selected tenant.

To save the current settings of the filter, query, and period:

1. In the Events section of the KUMA web interface, click the icon next to the filter expression and select Save current filter.
2. In the window that opens, enter the name of the filter configuration in the Name field. The name can contain up to 128 Unicode characters.
3. In the Tenant drop-down list, select the tenant that will own the created filter.
4. Click Save.

The filter configuration is now saved.

To select a previously saved filter configuration:

In the Events section of the KUMA web interface, click the icon next to the filter expression and select the relevant filter.

The selected configuration is active, which means that the search field is displaying the search query, and the upper part of the window is showing the configured settings for the period and frequency of updating the search results. Click the button to submit the search query.

You can click the icon near the filter configuration name to make it a default filter.

Deleting event filter configurations

To delete a previously saved filter configuration:

1. In the Events section of the KUMA web interface, click the icon next to the filter search query and click the icon next to the configuration that you need to delete.
2. Click OK.

The filter configuration is now deleted for all KUMA users.

Supported ClickHouse functions

The following ClickHouse functions are supported in KUMA:

• Arithmetic functions.
• Arrays—all functions except:
  • has
  • range
  • functions in which higher-order functions must be used (lambda expressions (->))
• Comparison functions: all operators except == and less.
• Logical functions: "not" function only.
• Type conversion functions.
• Date/time functions: all except date_add and date_sub.
• String functions.
• String search functions—all functions except:
  • position
  • multiSearchAllPositions, multiSearchAllPositionsUTF8, multiSearchFirstPosition, multiSearchFirstIndex, multiSearchAny
  • like and ilike
• Conditional functions: simple if operator only (ternary if and miltif operators are not supported).
• Mathematical functions.
• Rounding functions.
• Functions for splitting and merging strings and arrays.
• Bit functions.
• Functions for working with UUIDs.
• Functions for working with URLs.
• Functions for working with IP addresses.
• Functions for working with Nullable arguments.
• Functions for working with geographic coordinates.

In KUMA 2.1.3, errors with the operation of the DISTINCT operator have been fixed. In this case, you must use the following notation: SELECT DISTINCT SourceAddress as Addresses FROM <rest of the query>.

In KUMA 2.1.1, the SELECT DISTINCT SourceAddress / SELECT DISTINCT(SourceAddress) / SELECT DISTINCT ON (SourceAddress) operators work incorrectly.

Search and replace functions in strings, and functions from other sections are not supported.

For more details on SQL, refer to the ClickHouse documentation.

Viewing event detail areas

To view information about an event:

1. In the program web interface window, select the Events section.
2. Search for events by using the query builder or by entering a query in the search field.

   The event table is displayed.

3. Select the event whose information you want to view.

   The event details window opens.

The Event details area appears in the right part of the web interface window and contains a list of the event's parameters with values. In this area you can:

• Include the selected field in the search or exclude it from the search by clicking or next to the setting value.
• Clicking a file hash in the FileHash field opens a list in which you can select one of the following actions:
  • Show info from Threat Lookup.

    This is available when integrated with Kaspersky Threat Intelligence Portal.

  • Add to Internal TI of CyberTrace.
  • This is available when integrated with Kaspersky CyberTrace.
• Open a window containing information about the asset if it is mentioned in the event fields and registered in the program.
• You can click the link containing the collector name in the Service field to view the settings of the service that registered the event.

  You can also link an event to an alert if the program is in alert investigation mode and open the Correlation event details window if the selected event is a correlation event.

In the Event details area, the name of the described object is shown instead of its ID in the values of the following settings. At the same time, if you change the filtering of events by this setting (for example, by clicking ) to exclude events with a certain setting-value combination from search results), the object's ID, and not its name, is added to the SQL query:

• TenantID
• SeriviceID
• DeviceAssetID
• SourceAssetID
• DestinationAssetID
• SourceAccountID
• DestinationAccountID

Exporting events

In KUMA, you can export information about events to a TSV file. The selection of events that will be exported to a TSV file depends on filter settings. The information is exported from the columns that are currently displayed in the events table. The columns in the exported file are populated with the available data even if they did not display in the events table in the KUMA web interface due to the special features of the SQL query.

To export information about events:

1. In the Events section of the KUMA web interface, open the drop-down list and choose Export TSV.

   The new export TSV file task is created in the Task manager section.

2. Find the task you created in the Task manager section.

   When the file is ready to download, the icon will appear in the Status column of the task.

3. Click the task type name and select Upload from the drop-down list.

   The TSV file will be downloaded using your browser's settings. By default, the file name is event-export-<date>_<time>.tsv.

The file is saved based on your web browser's settings.

Configuring the table of events

Responses to user SQL queries are presented as a table in the Events section. The fields selected in the custom query appear at the end of the table, after the default columns. This table can be updated.

The following columns are displayed in the events table by default:

• Tenant.
• Timestamp.
• Name.
• DeviceProduct.
• DeviceVendor.
• DestinationAddress.
• DestinationUserName.

In KUMA, you can customize the displayed set of event fields and their display order. The selected configuration can be saved.

When using SQL queries with data grouping and aggregation for filtering events, statistics are not available and the order of displayed columns depends on the specific SQL query.

In the events table, in the event details area, in the alert window, and in the widgets, the names of assets, accounts, and services are displayed instead of the IDs as the values of the SourceAssetID, DestinationAssetID, DeviceAssetID, SourceAccountID, DestinationAccountID, and ServiceID fields. When exporting events to a file, the IDs are saved, but columns with names are added to the file. The IDs are also displayed when you point the mouse over the names of assets, accounts, or services.

Searching for fields with IDs is only possible using IDs.

To configure the fields displayed in the events table:

1. Click the icon in the top right corner of the events table.

   You will see a window for selecting the event fields that should be displayed in the events table.

2. Select the check boxes opposite the fields that you want to view in the table. You can search for relevant fields by using the Search field.

   You can configure the table to display any event field from the KUMA event data model. The Timestamp and Name parameters are always displayed in the table. Click the Default button to display only default event parameters in the events table.

   When you select a check box, the events table is updated and a new column is added. When a check box is cleared, the column disappears.

   You can also remove columns from the events table by clicking the column title and selecting Hide column from the drop-down list.

3. If necessary, change the display order of the columns by dragging the column headers in the event tables.
4. If you want to sort the events by a specific column, click its title and in the drop-down list select one of the available options: Ascending or Descending.

The selected event fields will be displayed as columns in the table of the Events section in the order you specified.

Refreshing events table

You can update the displayed event selection with the most recent entries by refreshing the web browser page. You can also refresh the events table automatically and set the frequency of updates. Automatic refresh is disabled by default.

To enable automatic refresh,

select the update frequency in the drop-down list:

• 5 seconds
• 15 seconds
• 30 seconds
• 1 minute
• 5 minutes
• 15 minutes

The events table now refreshes automatically.

To disable automatic refresh:

Select No refresh in the drop-down list:

Getting events table statistics

You can get statistics for the current events selection displayed in the events table. The selected events depend on the filter settings.

To obtain statistics:

Select Statistics from the drop-down list in the upper-right corner of the events table, or click on any value in the events table and select Statistics from the opened context menu.

The Statistics details area appears with the list of parameters from the current event selection. The numbers near each parameter indicate the number of events with that parameter in the selection. If a parameter is expanded, you can also see its five most frequently occurring values. Relevant parameters can be found by using the Search field.

In a fault-tolerant configuration, for all event fields that contain the FQDN of the Core, the Statistics section displays "core" instead of the FQDN.

The Statistics window allows you to modify the events filter.

When using SQL queries with data grouping and aggregation for filtering events, statistics are not available.

Viewing correlation event details

You can view the details of a correlation event in the Correlation event details window.

To view information about a correlation event:

1. In the Events section of the KUMA web interface, click a correlation event.

   You can use filters to find correlation events by assigning the correlated value to the Type parameter.

   The details area of the selected event will open. If the selected event is a correlation event, the Detailed view button will be displayed at the bottom of the details area.

2. Click the Detailed view button.

The correlation event window will open. The event name is displayed in the upper left corner of the window.

The Correlation event details section of the correlation event window contains the following data:

• Correlation event severity—the importance of the correlation event.
• Correlation rule—the name of the correlation rule that triggered the creation of this correlation event. The rule name is represented as a link that can be used to open the settings of this correlation rule.
• Correlation rule severity—the importance of the correlation rule that triggered the correlation event.
• Correlation rule ID—the identifier of the correlation rule that triggered the creation of this correlation event.
• Tenant—the name of the tenant that owns the correlation event.

The Related events section of the correlation event window contains the table of events related to the correlation event. These are base events that actually triggered the creation of the correlation event. When an event is selected, the details area opens in the right part of the web interface window.

The Find in events link to the right of the section header is used for alert investigation.

The Related endpoints section of the correlation event window contains the table of hosts related to the correlation event. This information comes from the base events related to the correlation event. Clicking the name of the asset opens the Asset details window.

The Related users section of the correlation event window contains the table of users related to the correlation event. This information comes from the base events related to the correlation event.

Normalizers

Normalizers are used for converting raw events that come from various sources in different formats to the KUMA event data model. Normalized events become available for processing by other KUMA resources and services.

A normalizer consists of the main event parsing rule and optional additional event parsing rules. By creating a main parsing rule and a set of additional parsing rules, you can implement complex event processing logic. Data is passed along the tree of parsing rules depending on the conditions specified in the

A normalizer is created in several steps:

1. Preparing to create a normalizer

   A normalizer can be created in the KUMA web interface:

   • In the Resources → Normalizers section.
   • When creating a collector, at the Event parsing step.

   Then parsing rules must be created in the normalizer.

2. Creating the main parsing rule for an event

   The main parsing rule is created using the Add event parsing button. This opens the Event parsing window, where you can specify the settings of the main parsing rule:

   • Specify event parsing settings.
   • Specify event enrichment settings.

   The main parsing rule for an event is displayed in the normalizer as a dark circle. You can view or modify the settings of the main parsing rule by clicking this circle. When you hover the mouse over the circle, a plus sign is displayed. Click it to add the parsing rules.

   The name of the main parsing rule is used in KUMA as the normalizer name.

3. Creating additional event parsing rules

   Clicking the plus icon that is displayed when you hover the mouse over the circle or the block corresponding to the normalizer opens the Additional event parsing window where you can specify the settings of the additional parsing rule:

   • Specify the conditions for sending data to the new normalizer.
   • Specify event parsing settings.
   • Specify event enrichment settings.

   The additional event parsing rule is displayed in the normalizer as a dark block. The block displays the triggering conditions for the additional parsing rule, the name of the additional parsing rule, and the event field. When this event field is available, the data is passed to the normalizer. Click the block of the additional parsing rule to view or modify its settings.

   If you hover the mouse over the additional normalizer, a plus button appears. You can use this button to create a new additional event parsing rule. To delete a normalizer, use the button with the trash icon.

4. Completing the creation of the normalizer

   To finish the creation of the normalizer, click Save.

In the upper right corner, in the search field, you can search for additional parsing rules by name.

For normalizer resources, you can enable the display of control characters in all input fields except the Description field.

If, when changing the settings of a collector resource set, you change or delete conversions in a normalizer connected to it, the edits will not be saved, and the normalizer itself may be corrupted. If you need to modify conversions in a normalizer that is already part of a service, the changes must be made directly to the normalizer under Resources → Normalizers in the web interface.

Event parsing settings

You can configure the rules for converting incoming events to the KUMA format when creating event parsing rules in the normalizer settings window, on the Normalization scheme tab.

Available settings:

• Name (required)—name of the parsing rules. Must contain 1 to 128 Unicode characters. The name of the main parsing rule is used as the name of the normalizer.
• Tenant (required)—name of the tenant that owns the resource.

  This setting is not available for extra parsing rules.

• Parsing method (required)—drop-down list for selecting the type of incoming events. Depending on your choice, you can use the preconfigured rules for matching event fields or set your own rules. When you select some parsing methods, additional parameter fields required for filling in may become available.

  Available parsing methods:

  • json

    This parsing method is used to process JSON data where each object, including its nested objects, occupies a single line in a file.

    When processing files with hierarchically arranged data, you can access the fields of nested objects by specifying the names of the parameters dividing them by a period. For example, the username parameter from the string "user": {"username": "system: node: example-01"} can be accessed by using the user.username query.

    Files are processed line by line. Multi-line objects with nested structures may be normalized incorrectly.

    In complex normalization schemes where additional normalizers are used, all nested objects are processed at the first normalization level, except for cases when the extra normalization conditions are not specified and, therefore, the event being processed is passed to the additional normalizer in its entirety.

    Newline characters can be \n and \r\n. Strings must be UTF-8 encoded.

  • cef

    This parsing method is used to process CEF data.

    When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button.

  • regexp

    This parsing method is used to create custom rules for processing data in a format using regular expressions.

    In the Normalization parameter block field, add a regular expression (RE2 syntax) with named capture groups. The name of a group and its value will be interpreted as the field and the value of the raw event, which can be converted into an event field in KUMA format.

    To add event handling rules:

    1. Copy an example of the data you want to process to the Event examples field. This is an optional but recommended step.
    2. In the Normalization parameter block field add a regular expression with named capture groups in RE2 syntax, for example "(?P<name>regexp)". The regular expression added to the Normalization parameter must exactly match the event. Also, when developing the regular expression, it is recommended to use special characters that match the starting and ending positions of the text: ^, $.

       You can add multiple regular expressions by using the Add regular expression button. If you need to remove the regular expression, use the button.

    3. Click the Copy field names to the mapping table button.

       Capture group names are displayed in the KUMA field column of the Mapping table. Now you can select the corresponding KUMA field in the column next to each capture group. Otherwise, if you named the capture groups in accordance with the CEF format, you can use the automatic CEF mapping by selecting the Use CEF syntax for normalization check box.

    Event handling rules were added.

  • syslog

    This parsing method is used to process data in syslog format.

    When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button.

  • csv

    This parsing method is used to create custom rules for processing CSV data.

    When choosing this method, you must specify the separator of values in the string in the Delimiter field. Any single-byte ASCII character can be used as a delimiter.

  • kv

    This parsing method is used to process data in key-value pair format.

    If you select this method, you must provide values in the following required fields:

    • Pair delimiter—specify a character that will serve as a delimiter for key-value pairs. You can specify any one-character (1 byte) value, provided that the character does not match the value delimiter.
    • Value delimiter—specify a character that will serve as a delimiter between the key and the value. You can specify any one-character (1 byte) value, provided that the character does not match the delimiter of key-value pairs.
  • xml

    This parsing method is used to process XML data in which each object, including its nested objects, occupies a single line in a file. Files are processed line by line.

    When this method is selected in the parameter block XML attributes you can specify the key attributes to be extracted from tags. If an XML structure has several attributes with different values in the same tag, you can indicate the necessary value by specifying its key in the Source column of the Mapping table.

    To add key XML attributes,

    Click the Add field button, and in the window that appears, specify the path to the required attribute.

    You can add more than one attribute. Attributes can be removed one at a time using the cross icon or all at once using the Reset button.

    If XML key attributes are not specified, then in the course of field mapping the unique path to the XML value will be represented by a sequence of tags.

    Tag numbering

    Tag numbering is available as of KUMA 2.1.3. This functionality allows automatically numbering tags in XML events, which lets you parse an event with identical tags or unnamed tags, such as <Data>.

    As an example, we will use the Tag numbering functionality to number the tags of the EventData attribute of Microsoft Windows PowerShell event ID 800.

    

    To parse such events, you must:

    • Configure tag numbering.
    • Configure data mapping for numbered tags with KUMA event fields.

    Simultaneous use of XML attributes and Tag numbering leads to incorrect operation of the normalizer. If an attribute contains unnamed tags or the identical tags, we recommend using the Tag numbering functionality. If the attribute contains only named tags, use XML attributes.

    To configure parsing of events with identically named or unnamed tags:

    1. Create a new normalizer or open an existing normalizer for editing.
    2. In the Basic event parsing window of the normalizer, in the Parsing method drop-down list, select 'xml' and in the Tag numbering field, click Add field.

       In the displayed field, enter the full path to the tag to whose elements you want to assign a number. For example, Event.EventData.Data. The first number to be assigned to a tag is 0. If the tag is empty, for example, <Data />, it is also assigned a number.

    3. To configure data mapping, under Mapping, click Add row and do the following:
       1. In the new row, in the Source field, enter the full path to the tag and its index. For the Microsoft Windows event from the example above, the full path with indices look like this:
          • Event.EventData.Data.0
          • Event.EventData.Data.1
          • Event.EventData.Data.2 and so on
       2. In the KUMA field drop-down list, select the field in the KUMA event that will receive the value from the numbered tag after parsing.
    4. To save changes:
       • If you created a new normalizer, click Save.
       • If you edited an existing normalizer, click Update configuration in the collector to which the normalizer is linked.

    Parsing is configured.

  • netflow5

    This parsing method is used to process data in the NetFlow v5 format.

    When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button.

    In mapping rules, the protocol type for netflow5 is not indicated in the fields of KUMA events by default. When parsing data in NetFlow format on the Enrichment normalizer tab, you should create a constant data enrichment rule that adds the netflow value to the DeviceProduct target field.

  • netflow9

    This parsing method is used to process data in the NetFlow v9 format.

    When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button.

    In mapping rules, the protocol type for netflow9 is not indicated in the fields of KUMA events by default. When parsing data in NetFlow format on the Enrichment normalizer tab, you should create a constant data enrichment rule that adds the netflow value to the DeviceProduct target field.

  • sflow5

    This parsing method is used to process data in sFlow5 format.

    When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button.

  • ipfix

    This parsing method is used to process IPFIX data.

    When choosing this method, you can use the preconfigured rules for converting events to the KUMA format by clicking the Apply default mapping button.

    In mapping rules, the protocol type for ipfix is not indicated in the fields of KUMA events by default. When parsing data in NetFlow format on the Enrichment normalizer tab, you should create a constant data enrichment rule that adds the netflow value to the DeviceProduct target field.

  • sql

    The normalizer uses this method to process data obtained by making a selection from the database.

• Keep raw event (required)—in this drop-down list, indicate whether you need to store the raw event in the newly created normalized event. Available values:
  • Don't save—do not save the raw event. This is the default setting.
  • Only errors—save the raw event in the Raw field of the normalized event if errors occurred when parsing it. This value is convenient to use when debugging a service. In this case, every time an event has a non-empty Raw field, you know there was a problem.

    If fields containing the names *Address or *Date* do not comply with normalization rules, these fields are ignored. No normalization error occurs in this case, and the values of the fields are not displayed in the Raw field of the normalized event even if the Keep raw event → Only errors option was selected.

  • Always—always save the raw event in the Raw field of the normalized event.

  This setting is not available for extra parsing rules.

• Keep extra fields (required)—in this drop-down list, you can choose whether you want to save fields and their values if no mapping rules have been configured for them (see below). This data is saved as an array in the Extra event field. Normalized events can be searched and filtered based on the data stored in the Extra field.

  Filtering based on data from the Extra event field

  Conditions for filters based on data from the Extra event field:

  • Condition—If.
  • Left operand—event field.
  • In this event field, you can specify one of the following values:
    • Extra field.
    • Value from the Extra field in the following format:

      Extra.<field name>

      For example, Extra.app.

      A value of this type is specified manually.

    • Value from the array written to the Extra field in the following format:

      Extra.<field name>.<array element>

      For example, Extra.array.0.

      The values in the array are numbered starting from 0.

      A value of this type is specified manually.

      To work with a value from the Extra field at depth 3 and below, use backquotes ``. For example, `Extra.lev1.lev2.lev3`.

  • Operator – =.
  • Right operand—constant.
  • Value—the value by which you need to filter events.

  By default, no extra fields are saved.

• Description—resource description: up to 4,000 Unicode characters.

  This setting is not available for extra parsing rules.

• Event examples—in this field, you can provide an example of data that you want to process.

  This setting is not available for the following parsing methods: netflow5, netflow9, sflow5, ipfix, sql.

  The Event examples field is populated with data obtained from the raw event if the event was successfully parsed and the type of data obtained from the raw event matches the type of the KUMA field.

  For example, the value "192.168.0.1" enclosed in quotation marks is not displayed in the SourceAddress field, in this case the value 192.168.0.1 is displayed in the Event examples field.

• Mapping settings block—here you can configure mapping of raw event fields to fields of the event in KUMA format:
  • Source—column for the names of the raw event fields that you want to convert into KUMA event fields.

    Clicking the button next to the field names in the Source column opens the Conversion window, in which you can use the Add conversion button to create rules for modifying the original data before they are written to the KUMA event fields. In the Conversion window, you can swap the added rules by dragging them by the icon; you can also delete them using the icon.

    Available conversions

    Conversions are changes that can be applied to a value before it gets written to the event field. The conversion type is selected from a drop-down list.

    Available conversions:

    • lower—is used to make all characters of the value lowercase
    • upper—is used to make all characters of the value uppercase
    • regexp – used to convert a value using the regular expression RE2. When this conversion type is selected, the field appears where regular expression should be added.
    • substring—is used to extract characters in the position range specified in the Start and End fields. These fields appear when this conversion type is selected.
    • replace—is used to replace specified character sequence with the other character sequence. When this type of conversion is selected, new fields appear:
      • Replace chars—in this field you can specify the character sequence that should be replaced.
      • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
    • trim—used to simultaneously remove the characters specified in the Chars field from the leading and end positions of the value. The field appears when this type of conversion is selected. For example, a trim conversion with the Micromon value applied to Microsoft-Windows-Sysmon results in soft-Windows-Sys.
    • append is used to add the characters specified in the Constant field to the end of the event field value. The field appears when this type of conversion is selected.
    • prepend—used to prepend the characters specified in the Constant field to the start of the event field value. The field appears when this type of conversion is selected.
    • replace with regexp—is used to replace RE2 regular expression results with the character sequence.
      • Expression—in this field you can specify the regular expression which results that should be replaced.
      • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
    • Converting encoded strings to text:
      • decodeHexString—used to convert a HEX string to text.
      • decodeBase64String—used to convert a Base64 string to text.
      • decodeBase64URLString—used to convert a Base64url string to text.

      When converting a corrupted string or if conversion error occur, corrupted data may be written to the event field.

      During event enrichment, if the length of the encoded string exceeds the size of the field of the normalized event, the string is truncated and is not decoded.

      If the length of the decoded string exceeds the size of the event field into which the decoded value is to be written, such a string is truncated to fit the size of the event field.

  • KUMA field—drop-down list for selecting the required fields of KUMA events. You can search for fields by entering their names in the field.
  • Label—in this column, you can add a unique custom label to event fields that begin with DeviceCustom* and Flex*.

  New table rows can be added by using the Add row button. Rows can be deleted individually using the button or all at once using the Clear all button.

  If you have loaded data into the Event examples field, the table will have an Examples column containing examples of values carried over from the raw event field to the KUMA event field.

  If the size of the KUMA event field is less than the length of the value placed in it, the value is truncated to the size of the event field.

Enrichment in the normalizer

When creating event parsing rules in the normalizer settings window, on the Enrichment tab, you can configure the rules for adding extra data to the fields of the normalized event using enrichment rules. These enrichment rules are stored in the settings of the normalizer where they were created.

Enrichments are created by using the Add enrichment button. There can be more than one enrichment rule. You can delete enrichment rules by using the button.

Settings available in the enrichment rule settings block:

• Source kind (required)—drop-down list for selecting the type of enrichment. Depending on the selected type, you may see advanced settings that will also need to be completed.

  Available Enrichment rule source types:

  • constant

    This type of enrichment is used when a constant needs to be added to an event field. Settings of this type of enrichment:

    • In the Constant field, specify the value that should be added to the event field. The value may not be longer than 255 Unicode characters. If you leave this field blank, the existing event field value will be cleared.
    • In the Target field drop-down list, select the KUMA event field to which you want to write the data.

  • dictionary

    This type of enrichment is used if you need to add a value from the dictionary of the Dictionary type.

    When this type is selected in the Dictionary name drop-down list, you must select the dictionary that will provide the values. In the Key fields settings block, you must use the Add field button to select the event fields whose values will be used for dictionary entry selection.

  • table

    This type of enrichment is used if you need to add a value from the dictionary of the Table type.

    When this enrichment type is selected in the Dictionary name drop-down list, select the dictionary for providing the values. In the Key fields group of settings, use the Add field button to select the event fields whose values are used for dictionary entry selection.

    In the Mapping table, configure the dictionary fields to provide data and the event fields to receive data:

    • In the Dictionary field column, select the dictionary field. The available fields depend on the selected dictionary resource.
    • In the KUMA field column, select the event field to which the value is written. For some of the selected fields (*custom* and *flex*), in the Label column, you can specify a name for the data written to them.

    New table rows can be added by using the Add new element button. Columns can be deleted using the button.

  • event

    This type of enrichment is used when you need to write a value from another event field to the current event field. Settings of this type of enrichment:

    • In the Target field drop-down list, select the KUMA event field to which you want to write the data.
    • In the Source field drop-down list, select the event field whose value will be written to the target field.
    • Clicking the button opens the Conversion window in which you can, using the Add conversion button, create rules for modifying the original data before writing them to the KUMA event fields.

      Available conversions

      Conversions are changes that can be applied to a value before it gets written to the event field. The conversion type is selected from a drop-down list.

      Available conversions:

      • lower—is used to make all characters of the value lowercase
      • upper—is used to make all characters of the value uppercase
      • regexp – used to convert a value using the regular expression RE2. When this conversion type is selected, the field appears where regular expression should be added.
      • substring—is used to extract characters in the position range specified in the Start and End fields. These fields appear when this conversion type is selected.
      • replace—is used to replace specified character sequence with the other character sequence. When this type of conversion is selected, new fields appear:
        • Replace chars—in this field you can specify the character sequence that should be replaced.
        • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
      • trim—used to simultaneously remove the characters specified in the Chars field from the leading and end positions of the value. The field appears when this type of conversion is selected. For example, a trim conversion with the Micromon value applied to Microsoft-Windows-Sysmon results in soft-Windows-Sys.
      • append is used to add the characters specified in the Constant field to the end of the event field value. The field appears when this type of conversion is selected.
      • prepend—used to prepend the characters specified in the Constant field to the start of the event field value. The field appears when this type of conversion is selected.
      • replace with regexp—is used to replace RE2 regular expression results with the character sequence.
        • Expression—in this field you can specify the regular expression which results that should be replaced.
        • With chars—in this field you can specify the characters sequence should be used instead of replaced characters.
      • Converting encoded strings to text:
        • decodeHexString—used to convert a HEX string to text.
        • decodeBase64String—used to convert a Base64 string to text.
        • decodeBase64URLString—used to convert a Base64url string to text.

        When converting a corrupted string or if conversion error occur, corrupted data may be written to the event field.

        During event enrichment, if the length of the encoded string exceeds the size of the field of the normalized event, the string is truncated and is not decoded.

        If the length of the decoded string exceeds the size of the event field into which the decoded value is to be written, such a string is truncated to fit the size of the event field.

  • template

    This type of enrichment is used when you need to write a value obtained by processing Go templates into the event field. Settings of this type of enrichment:

    • Put the Go template into the Template field.

      Event field names are passed in the {{.EventField}} format, where EventField is the name of the event field from which the value must be passed to the script.

      Example: Attack on {{.DestinationAddress}} from {{.SourceAddress}}.

    • In the Target field drop-down list, select the KUMA event field to which you want to write the data.
• Target field (required)—drop-down list for selecting the KUMA event field that should receive the data.

  This setting is not available for the enrichment source of the Table type.

Conditions for forwarding data to an extra normalizer

When creating additional event parsing rules, you can specify the conditions. When these conditions are met, the events are sent to the created parsing rule for processing. Conditions can be specified in the Additional event parsing window, on the Extra normalization conditions tab. This tab is not available for the basic parsing rules.

Available settings:

• Field to pass into normalizer—indicates the event field if you want only events with fields configured in normalizer settings to be sent for additional parsing.

  If this field is blank, the full event is sent to the extra normalizer for processing.

• Set of filters—used to define complex conditions that must be met by the events received by the normalizer.

  You can use the Add condition button to add a string containing fields for identifying the condition (see below).

  You can use the Add group button to add a group of filters. Group operators can be switched between AND, OR, and NOT. You can add other condition groups and individual conditions to filter groups.

  You can swap conditions and condition groups by dragging them by the icon; you can also delete them using the icon.

Filter condition settings:

• Left operand and Right operand—used to specify the values to be processed by the operator.

  In the left operand, you must specify the original field of events coming into the normalizer. For example, if the eventType - DeviceEventClass mapping is configured in the Basic event parsing window, then in the Additional event parsing window on the Extra normalization conditions tab, you must specify eventType in the left operand field of the filter. Data is processed only as text strings.

• Operators:
  • = – full match of the left and right operands.
  • startsWith – the left operand starts with the characters specified in the right operand.
  • endsWith – the left operand ends with the characters specified in the right operand.
  • match – the left operand matches the regular expression (RE2) specified in the right operand.
  • in – the left operand matches one of the values specified in the right operand.

The incoming data can be converted by clicking the button. The Conversion window opens, where you can use the Add conversion button to create the rules for converting the source data before any actions are performed on them. In the Conversion window, you can swap the added rules by dragging them by the icon; you can also delete them using the icon.

Available conversions

Supported event sources

KUMA supports the normalization of events coming from systems listed in the "Supported event sources" table. Normalizers for these systems are included in the distribution kit.

Supported event sources

Aggregation rules

Aggregation rules let you combine repetitive events of the same type and replace them with one common event. In this way, you can reduce the number of similar events sent to the storage and/or correlator, reduce the workload on services, conserve data storage space and licensing quota (EPS). An aggregation event is created when a time or number of events threshold is reached, whichever occurs first.

For aggregation rules, you can configure a filter and apply it only to events that match the specified conditions.

You can configure aggregation rules under Resources - Aggregation rules, and then select the created aggregation rule from the drop-down list in the collector settings. You can also configure aggregation rules directly in the collector settings.

Available aggregation rule settings

The KUMA distribution kit includes aggregation rules listed in the table below.

Predefined aggregation rules

Enrichment rules

Event enrichment involves adding information to events that can be used to identify and investigate an incident.

Enrichment rules let you add supplementary information to event fields by transforming data that is already present in the fields, or by querying data from external systems. For example, suppose that a user name is recorded in the event. You can use an enrichment rule to add information about the department, position, and manager of this user to the event fields.

Enrichment rules can be used in the following KUMA services and features:

• Collector.
• Correlator.
• Normalizer.

Available enrichment rule settings are listed in the table below.

Basic settings tab