Monitoring event sources

This section provides information about monitoring event sources.

Source status

In KUMA, you can monitor the state of the sources of data received by collectors. There can be multiple sources of events on one server, and data from multiple sources can be received by one collector.

You can configure automatic identification of event sources using one of the following sets of fields:

• Custom set of fields. You can specify from 1 to 9 fields in the order you want.
• Apply default mapping — DeviceProduct, DeviceHostName, DeviceAddress, DeviceProcessName. The field order cannot be changed.

  Sources are identified if the following fields in events are not empty: the DeviceProduct field, the DeviceAddress and/or DeviceHostname field, and the TenantID field (you do not need to specify the TenantID field, it is determined automatically). The DeviceProcessName field can be empty. If the DeviceProcessName field is not empty, and the other required fields are filled, a new source is identified.

  Identification of event sources depending on non-empty event fields

  DeviceProduct	DeviceHostName	DeviceAddress	DeviceProcessName	TenantID (detected automatically)
  +	+			+	Source 1 identified
  +		+		+	Source 2 identified
  +	+	+		+	Source 3 identified
  +	+		+	+	Source 4 identified
  +		+	+	+	Source 5 identified
  +	+	+	+	+	Source 6 identified
  	+	+		+	Source not identified
  	+		+	+	Source not identified
  		+	+	+	Source not identified
  +			+	+	Source not identified

Only one set of fields is applied for the entire installation. When upgrading to a new KUMA version, the default set of fields is applied. Only a user with the Main administrator role can configure the set of fields for identifying an event source. After you save changes to the set of fields, previously identified event sources are deleted from the KUMA Console and from the database. If necessary, you can revert to using a set of fields to determine default event sources. For the edited settings to take effect and KUMA to begin identifying sources based on the new settings, you must restart the collectors.

To identify event sources:

1. In the KUMA Console, go to the Source status section.
2. This opens the Source status window; in that window, click the wrench button.
3. This opens the Settings of event source detection window; in that window, in the Grouping fields for source detection drop-down list, select the event fields by which you want to identify event sources.

   You can specify from 1 to 9 fields in the order you want. In a custom configuration, KUMA identifies sources in which the TenantID field is filled (you do not need to specify this field separately, it is determined automatically) and at least one field from the Identical fields for source identification is filled. For numeric fields, 0 is considered an empty value. If a single numeric field is selected for source identification, and the value of the numeric field is 0, the source is not detected.

   After you save the modified set of fields, an audit event is created and all previously identified sources are deleted from the KUMA Console and from the database; assigned policies are disabled.

4. If you want to go back to the list of fields for identifying the default event source, click Apply default mapping. The default field order cannot be changed. If you manually specify the fields in the wrong order, an error is displayed and the save settings button becomes unavailable. The correct default sequence of fields is DeviceProduct, DeviceHostName, DeviceAddress, DeviceProcessName. Minimum configuration for identifying event sources using the default set of events: non-empty values in the DeviceProduct field, the DeviceAddress and/or DeviceHostName field, and the TenantID field (TenantID is determined automatically).
5. Click Save.
6. Restart the collectors to apply the changes and begin identifying event sources by the specified list of fields.

Source identification is configured.

To view events that are associated with an event source:

1. In the KUMA Console, go to the Source status section.
2. This opens the Event sources window; in that window, select your event source in the list, and in the Name column, expand the menu for the selected event source, click the Events for <number> days button.

   KUMA takes you to the Threat Hunting section, where you can view a list of events for the selected source over the last 5 minutes. Values of fields configured in the event source identification settings are automatically specified in the query. If necessary, in the Threat Hunting section, you can change the time period in the query and click Run query again to view the queried data for the specified time period.

Limitations

1. In a configuration with the default field set, KUMA registers the event source only if the raw event contains the DeviceProduct field and the DeviceAddress and/or DeviceHostName fields.

   If the raw event does not contain the DeviceProduct field and the DeviceAddress and/or DeviceHostName fields, you can:

   • Configure enrichment in the normalizer: on the Enrichment tab of the normalizer, select the Event data type, specify the Source field setting, and for the Target field, select the DeviceProduct + DeviceAddress and/or DeviceHostName and click OK.
   • Use an enrichment rule: select the Event data source type, specify the Source field setting, and as the Target field, select DeviceProduct + DeviceAddress and/or DeviceHostName, then click Create. The created enrichment rule must be linked to the collector at the Event enrichment step.

   KUMA will perform enrichment and register the event source.

2. If KUMA receives events with identical values of the fields that identify the source, KUMA registers different sources if the following conditions are satisfied:
   • The values of the required fields are identical, but different tenants are determined for the events.
   • The values of the required fields are identical, but one of the events has an optional DeviceProcessName field specified.
   • The values of the required fields are identical, but the data in these fields have different character case.

If you want KUMA to log such events under the same source, you can further configure the fields in the normalizer.

Lists of sources are generated in collectors, merged in the KUMA Core, and displayed in the program web interface under Source status on the List of event sources tab. Data is updated every minute.

The rate and number of incoming events serve as an important indicator of the state of the observed system. You can configure monitoring policies such that changes are tracked automatically and notifications are automatically created when indicators reach specific boundary values. Monitoring policies are displayed in the KUMA Console under Source status on the Monitoring policies tab.

When monitoring policies are triggered, monitoring events are created and include data about the source of events.

List of event sources

Sources of events are displayed in the table under Source status → List of event sources. One page can display up to 250 sources. You can sort the table by clicking the column header of the relevant setting. Clicking on a source of events opens an incoming data graph.

You can use the Search field to search for event sources. The search is performed using regular expressions (RE2).

If necessary, you can configure the interval for updating data in the table. Available update periods: 1 minute, 5 minutes, 15 minutes, 1 hour. The default value is No refresh. You may need to configure the update period to track changes made to the list of sources.

The following columns are available:

• Status—status of the event source:
  • Green—events are being received within the limits of the assigned monitoring policy.
  • Red—the frequency or number of incoming events go beyond the boundaries defined in the monitoring policy.
  • Gray—a monitoring policy has not been assigned to the source of events.

  The table can be filtered by this setting.

  • Name—name of the event source. The name is generated automatically from the values of fields configured in the event source identification settings.

    You can change the name of an event source. The name can contain no more than 128 Unicode characters.

• Host name or IP address—name or IP address of the host from which the events originate if the DeviceHostName or DeviceAddress fields are specified in the event source identification settings.
• Monitoring policy—name of the monitoring policy assigned to the event source.
• Stream—frequency at which events are received from the event source.
• Lower limit—lower boundary of the permissible number of incoming events as indicated in the monitoring policy.
• Upper limit—upper boundary of the permissible number of incoming events as indicated in the monitoring policy.
• Tenant—the tenant that owns the events received from the event source.

  By default, no more than 250 event sources are displayed on the page and are available for selection. If there are more event sources, to select them you must load additional event sources by clicking the Show next 250 button in the lower part of the window.

If you select sources of events, the following buttons become available:

• Save to CSV—you can click this button to export data of the selected event sources to a file named event-source-list.csv in UTF-8 encoding.
• Apply policy and Disable policy—you can click these buttons to enable or disable a monitoring policy for a source of events. When enabling a policy, you must select the policy from the drop-down list. When disabling a policy, you must select how long you want to disable the policy: temporarily or forever.

  If there is no policy for the selected event source, the Apply policy button is inactive. This button will also be inactive if sources from different tenants are selected, but the user has no available policies in the shared tenant.

  In some rare cases, the status of a disabled policy may change from gray to green a few seconds after it is disabled due to overlapping internal processes of KUMA. If this happens, you need to disable the monitoring policy again.

• Remove event source from the list—you can click this button to remove an event source from the table. The statistics on this source will also be removed. If a collector continues to receive data from the source, the event source will re-appear in the table but its old statistics will not be taken into account.

Monitoring policies

The rate and number of incoming events serve as an important indicator of the state of the system. For example, you can detect when there are too many events, too few, or none at all. Monitoring policies are designed to detect such situations. In a policy, you can specify a lower threshold, an optional upper threshold, and the way the events are counted: by frequency or by total number.

The policy must be applied to the event source. After applying the policy, you can monitor the status of the source: green means everything is OK, red means the stream is outside the configured threshold. If the status is red, an event of the Monitoring type generated. You can also configure notifications to be sent to an arbitrary email address. Policies for monitoring the sources of events are displayed in the table under Source status → Monitoring policies. You can sort the table by clicking the column header of the relevant setting. Clicking a policy opens the data area with policy settings. The settings can be edited.

To add a monitoring policy:

1. In the KUMA Console, under Source status → Monitoring policies, click Add policy and specify the settings in the opened window:
   1. In the Policy name field, enter a unique name for the policy you are creating. The name must contain 1 to 128 Unicode characters.
   2. In the Tenant drop-down list, select the tenant that will own the policy. Your tenant selection determines the specific sources of events that can covered by the monitoring policy.
   3. In the Policy type drop-down list, select one of the following options:
      • byCount—by the number of events over a certain period.
      • byEPS—by the number of events per second over a certain period. The average value over the entire period is calculated. You can additionally track spikes during specific periods.
   4. In the Lower limit and Upper limit fields, set the boundaries representing normal behavior. Deviations from these boundaries will trigger the monitoring policy, create alerts, and forward notifications.
   5. In the Count interval field, specify the period during which the monitoring policy must take into account the data from the monitoring source. The maximum value is 14 days.
   6. If necessary, specify the email addresses to which notifications about the activation of the KUMA monitoring policy should be sent. To add each address, click the Email button.

      To forward notifications, you must configure a connection to the SMTP server.

2. Click Add.

The monitoring policy will be added.

To remove a monitoring policy,

select one or more policies, then click Delete policy and confirm the action.

You cannot remove preinstalled monitoring policies or policies that have been assigned to data sources.