Managing KUMA

This section describes KUMA general settings.

Logging in to the program web interface

To log in to the program web interface:

1. Enter the following address in your browser:

   https://<IP address or FQDN of KUMA Core server>:7220

   The web interface authorization page will open and prompt you to enter your login and password.

2. Enter the login of your account in the Login field.
3. Enter the password for the specified account in the Password field.
4. Click the Login button.

The main window of the program web interface opens.

In multitenancy mode, a user who is logging in to the program web interface for the first time will see the data only for those tenants that were selected for the user when their user account was created.

To log out of the program web interface,

open the KUMA web interface, click your user account name in the bottom-left corner of the window, and click the Logout button in the opened menu.

Viewing KUMA metrics

Comprehensive information about the performance of the KUMA Core, storage, collectors, and correlators is available in the Metrics section of the KUMA web interface. Selecting this section opens the Grafana portal deployed as part of KUMA Core installation and is updated automatically. If the Metrics section shows core: <port number>, this means that KUMA is deployed in a fault-tolerant configuration and the metrics were received from the host on which the Core was installed. In other configurations, the name of the host from which KUMA receives metrics is displayed.

The default Grafana user name and password are admin and admin.

Available metrics

Collector indicators:

• IO—metrics related to the service input and output.
  • Processing EPS—the number of processed events per second.
  • Processing Latency—the time required to process a single event (the median is displayed).
  • Output EPS—the number of events, sent to the destination per second.
  • Output Latency—the time required to send a batch of events to the destination and receive a response from it (the median is displayed).
  • Output Errors—the number or errors when sending event batches to the destination per second. Network errors and errors writing the disk buffer are displayed separately.
  • Output Event Loss—the number of lost events per second. Events can be lost due to network errors or errors writing the disk buffer. Events are also lost if the destination responded with an error code (for example, if the request was invalid).
• Normalization—metrics related to the normalizers.
  • Raw & Normalized event size—the size of the raw event and size of the normalized event (the median is displayed).
  • Errors—the number of normalization errors per second.
• Filtration—metrics related to the filters.
  • EPS—the number of events rejected by the Collector per second. The collector only rejects events if the user has added a filter into the collector service configuration.
• Aggregation—metrics related to the aggregation rules.
  • EPS—the number of events received and created by the aggregation rule per second. This metric helps determine the effectiveness of aggregation rules.
  • Buckets—the number of buckets in the aggregation rule.
• Enrichment—metrics related to the enrichment rules.
  • Cache RPS—the number requests to the local cache per second.
  • Source RPS—the number of requests to the enrichment source (for example, the Dictionary resource).
  • Source Latency—the time required to send a request to the enrichment source and receive a response from it (the median is displayed).
  • Queue—the enrichment requests queue size. This metric helps to find bottleneck enrichment rules.
  • Errors—the number of enrichment source request errors per second.

Correlator metrics

• IO—metrics related to the service input and output.
  • Processing EPS—the number of processed events per second.
  • Processing Latency—the time required to process a single event (the median is displayed).
  • Output EPS—the number of events, sent to the destination per second.
  • Output Latency—the time required to send a batch of events to the destination and receive a response from it (the median is displayed).
  • Output Errors—the number or errors when sending event batches to the destination per second. Network errors and errors writing the disk buffer are displayed separately.
  • Output Event Loss—the number of lost events per second. Events can be lost due to network errors or errors writing the disk buffer. Events are also lost if the destination responded with an error code (for example, if the request was invalid).
• Correlation—metrics related to the correlation rules.
  • EPS—the number of correlation events created per second.
  • Buckets—the number of buckets in the correlation rule (only for the standard kind of correlation rules).
• Active Lists—metrics related to the active lists.
  • RPS—the number of requests (and their type) to the Active list per second.
  • Records—the number of entries in the Active list.
  • WAL Size—the size of the Write-Ahead-Log. This metric helps determine the size of the Active list.

Storage indicators

• IO—metrics related to the service input and output.
  • RPS—the number of requests to the Storage service per second.
  • Latency—the time of proxying a single request to the ClickHouse node (the median is displayed).

Core service metrics

• IO—metrics related to the service input and output.
  • RPS—the number of requests to the Core service per second.
  • Latency—the time of processing a single request (the median is displayed).
  • Errors—the number of request errors per second.
• Notification Feed—metrics related to user activity.
  • Subscriptions—the number of clients, connected to the Core via SSE to receive server messages in real time. This number usually correlates with the number of clients using the KUMA web interface.
  • Errors—the number of message sending errors per second.
• Schedulers—metrics related to Core tasks.
  • Active—the number of repeating active system tasks. The tasks created by the user are ignored.
  • Latency—the time of processing a single request (the median is displayed).
  • Position—the position (timestamp) of the alert creation task. The next ClickHouse scan for correlation events will start from this position.
  • Errors—the number of task errors per second.

General metrics common for all services

• Process—general process metrics.
  • CPU—CPU usage.
  • Memory—RAM usage (RSS).
  • DISK IOPS—the number of disk read/write operations per second.
  • DISK BPS—the number of bytes read/written to the disk per second.
  • Network BPS—the number of bytes received/sent per second.
  • Network Packet Loss—the number of network packets lost per second.
  • GC Latency—the time of the GO Garbage Collector cycle (the median is displayed).
  • Goroutines—the number of active goroutines. This number differs from the thread count.
• OS—metrics related to the operating system.
  • Load—the average load.
  • CPU—CPU usage.
  • Memory—RAM usage (RSS).
  • Disk—disk space usage.

Metrics storage period

KUMA operation data is saved for 3 months by default. This storage period can be changed.

To change the storage period for KUMA metrics:

1. Log in to the OS of the server where the KUMA Core is installed.
2. In the file /etc/systemd/system/multi-user.target.wants/kuma-victoria-metrics.service, in the ExecStart parameter, edit the --retentionPeriod=<metrics storage period, in months> flag by inserting the necessary period. For example, --retentionPeriod=4 means that the metrics will be stored for 4 months.
3. Restart KUMA by running the following commands in sequence:
   1. systemctl daemon-reload
   2. systemctl restart kuma-victoria-metrics

The storage period for metrics has been changed.

Managing KUMA tasks

When working in the program web interface, you can use tasks to perform various operations. For example, you can import assets or export KUMA event information to a TSV file.

Viewing the tasks table

The tasks table contains a list of created tasks and is located in the Task manager section of the program web interface window. You can view the tasks that were created by you (current user).

A user with the General Administrator role can view the tasks of all users.

The tasks table contains the following information:

• State—the state of the task. One of the following statuses can be assigned to a task:
  • Green dot blinking—the task is active.
  • Completed—the task is complete.
  • Cancel—the task was canceled by the user.
  • Error—the task was not completed because of an error. The error message is displayed if you hover the mouse over the exclamation mark icon.
• Task—the task type. The program provides the following types of tasks:
  • Events export—export KUMA events.
  • Threat Lookup—request data from the Kaspersky Threat Intelligence Portal.
  • Retroscan—task for replaying events.
  • KSC assets import—imports asset data from Kaspersky Security Center servers.
  • Accounts import—imports user data from Active Directory.
  • KICS for Networks assets import—imports asset data from KICS for Networks.
  • Repository update—updates the KUMA repository to receive the resource packages from the source specified in settings.
• Created by—the user who created the task. If the task was created automatically, the column will show Scheduled task.

  This column is displayed only for users with the General Administrator and Administrator roles.

• Created—task creation time.
• Updated—time when the task was last updated.
• Tenant—the name of the tenant in which the task was started.

The task date format depends on the localization language selected in the application settings. Possible date format options:

• English localization: YYYY-MM-DD.
• Russian localization: DD.MM.YYYY.

Configuring the display of the tasks table

You can customize the display of columns and the order in which they appear in the tasks table.

To customize the display and order of columns in the tasks table:

1. In the KUMA web interface, select the Task manager section.

   The tasks table is displayed.

2. In the table header, click the button.
3. In the opened window, do the following:
   • If you want to enable display of a column in the table, select the check box next to the name of the parameter that you want to display in the table.
   • If you do not want the parameter to be displayed in the table, clear the check box.

   At least one check box must be selected.

4. If you want to reset the settings, click the Default link.
5. If you want to change the order in which the columns are displayed in the table, move the mouse cursor over the name of the column, hold down the left mouse button and drag the column to the necessary position.

The display of columns in the tasks table will be configured.

Viewing task run results

To view the results of a task:

1. In the KUMA web interface, select the Task manager section.

   The tasks table is displayed.

2. Click the link containing the task type in the Task column.

   A list of the operations available for this task type will be displayed.

3. Select Show results.

The task results window opens.

Restarting a task

To restart a task:

1. In the KUMA web interface, select the Task manager section.

   The tasks table is displayed.

2. Click the link containing the task type in the Task column.

   A list of the operations available for this task type will be displayed.

3. Select Restart.

The task will be restarted.

Proxies

Proxy servers are used to store proxy server configuration settings, for example, in destinations. The http type is supported.

Available settings:

• Name (required)—unique name of the proxy server. Must contain 1 to 128 Unicode characters.
• Tenant (required)—name of the tenant that owns the resource.
• Use URL from the secret (required)—drop-down list to select a secret resource that stores URLs of proxy servers. If required, a secret can be created in the proxy server creation window by using the button. The selected secret can be changed by clicking on the button.
• Do not use for domains—one or more domains that require direct access.
• Description—up to 4,000 Unicode characters.

Connecting to an SMTP server

KUMA can be configured to send email notifications using an SMTP server. Users will receive notifications if the Receive email notifications check box is selected in their profile settings.

Only one SMTP server can be added to process KUMA notifications. An SMTP server connection is managed in the KUMA web interface under Settings → General → SMTP server settings.

To configure SMTP server connection:

1. Open the KUMA web interface and select Settings → General.
2. In the SMTP server settings block, change the relevant settings:
   • Disabled—select this check box if you want to disable connection to the SMTP server.
   • Host (required)—SMTP host in one of the following formats: hostname, IPv4, IPv6.
   • Port (required)—SMTP port. The value must be an integer from 1 to 65,535.
   • From (required)—email address of the message sender. For example, kuma@company.com.
   • Alias for KUMA Core server—name of the KUMA Core server that is used in your network. Must be different from the FQDN.
   • If necessary, use the Secret drop-down list to select a secret of the credentials type that contains the account credentials for connecting to the SMTP server.

     Add secret

     1. If you previously created a secret, select it from the Secret drop-down list.

        If no secret was previously added, the drop-down list shows No data.

     2. If you want to add a new secret, click the button on the right of the Secret list.

        The Secret window opens.

     3. In the Name field, enter the name that will be used to display the secret in the list of available secrets.
     4. In the User and Password fields, enter the credentials of the user account that the Agent will use to connect to the connector.
     5. If necessary, add any other information about the secret in the Description field.
     6. Click the Save button.

     The secret will be added and displayed in the Secret list.

   • Select the necessary frequency of notifications in the Monitoring notifications interval drop-down list.

     Notifications from the source about a monitoring policy triggering are repeated after the selected period until the status of the source becomes green again.

     If the Notify once setting is selected, you receive a notification about monitoring policy activation only once.

   • Turn on the Disable monitoring notifications toggle button if you do not want to receive notifications about the state of event sources. The toggle switch is turned off by default.
3. Click Save.

The SMTP server connection is now configured, and users can receive email messages from KUMA.

Working with Kaspersky Security Center tasks

You can connect Kaspersky Security Center assets to KUMA and download database and application module updates to these assets, or run an anti-virus scan on them by using Kaspersky Security Center tasks. Tasks are started in the KUMA web interface.

To run Kaspersky Security Center tasks on assets connected to KUMA, it is recommended to use the following script:

1. Creating a user account in the Kaspersky Security Center Administration Console

   The credentials of this account are used when creating a secret to establish a connection with Kaspersky Security Center, and can be used to create a task.

   For more details about creating a user account and assigning permissions to a user, please refer to the Kaspersky Security Center Help Guide.

2. Creating KUMA tasks in Kaspersky Security Center
3. Configuring KUMA integration with Kaspersky Security Center
4. Importing asset information from Kaspersky Security Center into KUMA
5. Assigning a category to the imported assets

   After import, the assets are automatically placed in the Uncategorized devices group. You can assign one of the existing categories to the imported assets, or create a category and assign it to the assets.

6. Running tasks on assets

   You can manually start tasks in the asset information or configure tasks to start automatically.

Creating KUMA tasks in Kaspersky Security Center

You can run the anti-virus database and application module update task, and the virus scan task on Kaspersky Security Center assets connected to KUMA. The assets must have Kaspersky Endpoint Security for Windows or Linux installed. The tasks are created in Kaspersky Security Center Web Console.

For more details about creating the Update and Virus scan tasks on the assets with Kaspersky Endpoint Security for Windows, refer to the Kaspersky Endpoint Security for Windows Help Guide.

For more details about creating the Update and Virus scan tasks on the assets with Kaspersky Endpoint Security for Linux, refer to the Kaspersky Endpoint Security for Linux Help Guide.

Task names must begin with "kuma" (not case-sensitive and without quotations). For example, KUMA antivirus check. Otherwise, the task is not displayed in the list of available tasks in the KUMA web interface.

Starting Kaspersky Security Center tasks manually

You can manually run the anti-virus database, application module update task, and the anti-virus scan task on Kaspersky Security Center assets connected to KUMA. The assets must have Kaspersky Endpoint Security for Windows or Linux installed.

First, you need to configure the integration of Kaspersky Security Center with KUMA and create tasks in Kaspersky Security Center.

To manually start a Kaspersky Security Center task:

1. In the Assets section of the KUMA web interface, select the asset that was imported from Kaspersky Security Center.

   The Asset details window opens.

2. Click the KSC response button.

   This button is displayed if the connection to the Kaspersky Security Center that owns the selected asset is enabled.

3. In the opened Select task window, select the check boxes next to the tasks that you want to start, and click the Start button.

Kaspersky Security Center starts the selected tasks.

Some types of tasks are available only for certain assets.

You can obtain vulnerability and software information only for assets running a Windows operating system.

Starting Kaspersky Security Center tasks automatically

You can configure the automatic start of the anti-virus database and application module update task and the virus scan task for Kaspersky Security Center assets connected to KUMA. The assets must have Kaspersky Endpoint Security for Windows or Linux installed.

First, you need to configure the integration of Kaspersky Security Center with KUMA and create tasks in Kaspersky Security Center.

Configuring automatic start of Kaspersky Security Center tasks includes the following steps:

Step 1. Adding a correlation rule

To add a correlation rule:

1. In the KUMA web interface, select the Resources section.
2. Select Correlation rules and click the Add correlation rule button.
3. On the General tab, define the following settings:
   1. In the Name field, define the rule name.
   2. In the Tenant drop-down list, select the tenant that owns the resource.
   3. In the Type drop-down list, select simple.
   4. In the Propagated fields field, add the following fields: DestinationAssetID.
   5. If required, define the values for the following fields:
      • In the Rate limit field, define the maximum number of times per second that the rule will be triggered.
      • In the Severity field, define the severity of alerts and correlation events that will be created as a result of the rule being triggered.
      • In the Description field, provide any additional information.
4. On the Selectors → Settings tab, do the following:
   1. In the Filter drop-down list, select Create new.
   2. In the Conditions field, click the Add group button.
   3. In the operator field for the group you added, select AND.
   4. Add a condition for filtering by the DeviceProduct field value:
      1. In the Conditions field, click the Add condition button.
      2. In the condition field, select If.
      3. In the Left operand field, select event field.
      4. In the 'Event field' field, select DeviceProduct.
      5. In the Operator field, select =.
      6. In the Right operand field, select constant.
      7. In the value field, enter KSC.
   5. Add a condition for filtering by the Name field value:
      1. In the Conditions field, click the Add condition button.
      2. In the condition field, select If.
      3. In the Left operand field, select event field.
      4. In the event field, select Name.
      5. In the Operator field, select =.
      6. In the Right operand field, select constant.
      7. In the value field, enter the name of the event. When this event is detected, the task is started automatically.

         For example, if you want the Virus scan task to start when Kaspersky Security Center registers the Malicious object detected event, specify this name in the Value field.

         You can view the event name in the Name field of the event details.

5. On the Actions tab, define the following settings:
   1. In the Actions section, open the On every event drop-down list.
   2. Select the Output check box.

      You do not need to fill in other fields.

6. Click the Save button.

The correlation rule will be created.

Step 2. Creating a correlator

You need to launch the correlator installation wizard. At step 3 of the wizard, you are required to select the correlation rule that you added by following this guide.

The DeviceHostName field must display the domain name (FQDN) of the asset. If it is not displayed, create a DNS record for this asset and create a DNS enrichment rule at Step 4 of the wizard.

Step 3. Adding a filter

To add a filter:

1. In the KUMA web interface, select the Resources section.
2. Select Filters and click the Add filter button.
3. In the Name field, specify the filter name.
4. In the Tenant drop-down list, select the tenant that owns the resource.
5. In the Conditions field, click the Add group button.
6. In the operator field for the group you added, select AND.
7. Add a condition for filtering by the DeviceProduct field value:
   1. In the Conditions field, click the Add condition button.
   2. In the condition field, select If.
   3. In the Left operand field, select event field.
   4. In the 'Event field' field, select Type.
   5. In the Operator field, select =.
   6. In the Right operand field, select constant.
   7. In the Value field, enter 3.
8. Add a condition for filtering by the Name field value:
   1. In the Conditions field, click the Add condition button.
   2. In the condition field, select If.
   3. In the Left operand field, select event field.
   4. In the event field, select Name.
   5. In the Operator field, select =.
   6. In the Right operand field, select constant.
   7. In the Value field, enter the name of the correlation rule created at Step 1.

Step 4. Adding a response rule

To add a response rule:

1. In the KUMA web interface, select the Resources section.
2. Select Response rules and click the Add response rule button.
3. In the Name field, define the rule name.
4. In the Tenant drop-down list, select the tenant that owns the resource.
5. In the Type drop-down list, select Response via KSC.
6. In the Kaspersky Security Center task drop-down list, select the Kaspersky Security Center task you want to start.
7. In the Event field drop-down list, select the DestinationAssetID.
8. In the Workers field, specify the number of processes that the service can run simultaneously.

   By default, the number of work processes is the same as the number of virtual processors on the server where the correlator service is installed.

• In the Description field, you can add up to 4,000 Unicode characters.
• In the Filter drop-down list, select the filter added at Step 3 of this instruction.

To send requests to Kaspersky Security Center, you must ensure that Kaspersky Security Center is available over the UDP protocol.

If a response rule is owned by the shared tenant, the displayed Kaspersky Security Center tasks that are available for selection are from the Kaspersky Security Center server that the main tenant is connected to.

If a response rule has a selected task that is absent from the Kaspersky Security Center server that the tenant is connected to, the task is not performed for assets of this tenant. This situation could arise when two tenants are using a common correlator, for example.

Step 5. Adding a response rule to the correlator

To add a response rule to the correlator:

1. In the KUMA web interface, select the Resources section.
2. Select Correlators.
3. In the list of correlators, select the correlator added at Step 2 of this instruction.
4. In the steps tree, select Response rules.
5. Click Add.
6. In the Response rule drop-down list, select the rule added at step 4 of these instructions.
7. In the steps tree, select Setup validation.
8. Click the Save and restart services button.
9. Click the Save button.

The response rule will be added to the correlator.

The automatic start will be configured for the anti-virus database and application module update task and the virus scan task on Kaspersky Security Center assets connected to KUMA. The tasks are started when a threat is detected on the assets and KUMA receives the corresponding events.

Checking the status of Kaspersky Security Center tasks

In the KUMA web interface, you can check whether a Kaspersky Security Center task was started or whether a search for events owned by the collector listening for Kaspersky Security Center events was completed.

To check the status of Kaspersky Security Center tasks:

1. In KUMA, select Resources → Active services.
2. Select the collector that is configured to receive events from the Kaspersky Security Center server and click the Go to Events button.

A new browser tab will open in the Events section of KUMA. The table displays events from the Kaspersky Security Center server. The status of the tasks can be seen in the Name column.

Kaspersky Security Center event fields:

• Name—status or type of the task.
• Message—message about the task or event.
• FlexString<number>Label—name of the attribute received from Kaspersky Security Center. For example, FlexString1Label=TaskName.
• FlexString<number>—value of the FlexString<number>Label attribute. For example, FlexString1=Download updates.
• DeviceCustomNumber<number>Label—name of the attribute related to the task state. For example, DeviceCustomNumber1Label=TaskOldState.
• DeviceCustomNumber<number>—value related to the task state. For example, DeviceCustomNumber1=1 means the task is executing.
• DeviceCustomString<number>Label—name of the attribute related to the detected vulnerability: for example, a virus name, affected application.
• DeviceCustomString<number>—value related to the detected vulnerability. For example, the attribute-value pairs DeviceCustomString1Label=VirusName and DeviceCustomString1=EICAR-Test-File mean that the EICAR test virus was detected.

KUMA logs

By default, only errors are logged for all KUMA components. To receive detailed data in logs, configure Debug mode in the component settings.

The log is updated for 30 days or until its size reaches 1 GB. After 30 days, the log is archived and new events are written to a new log. Archives are stored in the logs folder. A maximum of three archived logs are stored on the server at a time. When a new log archive appears, if there are more than three archives, the oldest one is deleted. Archives containing logs older than 365 days are also deleted.

Debug mode is available for the following components:

Viewing Core logs in Kubernetes

When Core log files reach 100 MB, they are archived and a new log is written. No more than five files are stored at a time. If there are more than five files when a new log appears, the oldest file is deleted.

On worker nodes, you can view the logs of containers and pods residing on these nodes in the file system of the node.
For example:
/var/log/pods/kuma_core-deployment-<UID>/core/*.log
/var/log/pods/kuma_core-deployment-<UID>/mongodb/*.log

To view the logs of all containers in the Core pod:

k0s kubectl logs -l app=core --all-containers -n kuma

To view the log of a specific container:

k0s kubectl logs -l app = core -c <container_name> -n kuma

To enable real-time log viewing, add the -f switch:

k0s kubectl logs -f -l app=core --all-containers -n kuma

To view the logs of the previous pod that was overwritten by a new one (for example, when recovering from a critical error or after redeployment), add the --previous switch:

k0s kubectl logs -l app=core -c core -n kuma --previous

To access the logs from other hosts that are not included in the cluster, you need the k0s-kubeconfig.yml file containing the access credentials created during KUMA installation, and the locally installed kubectl cluster management utility.
The cluster controller or traffic balancer specified in the server parameter of the k0s-kubeconfig.yml file must be accessible over the network.

The file path must be exported to a variable: 
export KUBECONFIG=/<file path>/k0s-kubeconfig.yml

You can use kubeclt to view the logs. For example:

kubectl logs -l app=core -c mongodb -n kuma

KUMA notifications

Standard notifications

KUMA can be configured to send email notifications using an SMTP server. To do so, configure a connection to an SMTP server and select the Receive email notifications check box for users who should receive notifications.

KUMA automatically notifies users about the following events:

• A report was created (the users listed in the report template receive a notification).
• An alert was created (all users receive a notification).
• An alert was assigned to a user (the user to whom the alert was assigned receives a notification).
• A task was performed (the users who created the task receive a notification).
• New resource packages are available. They can be obtained by updating the KUMA repository (the users whose email address is specified in the task settings are notified).

Custom notifications

Instead of the standard KUMA notifications about the alert generation, you can send notifications based on custom templates. To configure custom notifications instead of standard notifications, take the following steps:

• Create an email template.
• Create a notification rule that specifies the correlation rules and email addresses.

When an alert is created based on the selected correlation rules, notifications created based on custom email templates will be sent to the specified email addresses. Standard KUMA notifications about the same event will not be sent to the specified addresses.