Contents
Starting the Collector Installation Wizard
A collector consists of two parts: one part is created inside the KUMA web interface, and the other part is installed on the network infrastructure server intended for receiving events. The Installation Wizard creates the first part of the collector.
To start the Collector Installation Wizard:
- In the KUMA web interface, in the Resources section, click Add event source.
- In the KUMA web interface in the Resources → Collectors section click Add collector.
Follow the instructions of the Wizard.
Aside from the first and last steps of the Wizard, the steps of the Wizard can be performed in any order. You can switch between steps by using the Next and Previous buttons, as well as by clicking the names of the steps in the left side of the window.
After the Wizard completes, a resource set for a collector is created in the KUMA web interface under Resources → Collectors, and a collector service is added under Resources → Active services.
Step 1. Connecting event sources
This is a required step of the Installation Wizard. At this step, you specify the main settings of the collector: its name and the tenant that will own it.
To specify the basic settings of the collector:
- In the Name field, enter a unique name for the service you are creating. The name must contain from 1 to 128 Unicode characters.
- In the Tenant drop-down list, select the tenant that will own the collector. The tenant selection determines what resources will be available when the collector is created.
If you return to this window from another subsequent step of the Installation Wizard and select another tenant, you will have to manually edit all the resources that you have added to the service. Only resources from the selected tenant and shared tenant can be added to the service.
- If required, specify the number of processes that the service can run concurrently in the Workers field. By default, the number of worker processes is the same as the number of vCPUs on the server where the service is installed.
- If necessary, use the Debug drop-down list to enable logging of service operations.
- You can optionally add up to 256 Unicode characters describing the service in the Description field.
The main settings of the collector are specified. Proceed to the next step of the Installation Wizard.
Page topStep 2. Transport
This is a required step of the Installation Wizard. On the Transport tab of the Installation Wizard, select or create a connector resource with the settings indicating from where the collector service should receive events.
To add an existing connector to a resource set:
Select the name of the required connector from the Connector drop-down list.
The Transport tab of the Installation Wizard will display the settings of the selected connector. You can open the selected resource for editing in a new browser tab using the button.
To create a new connector:
- Select Create from the Connector drop-down list.
- In the Type drop-down list, select the connector type and define its settings on the Basic settings and Advanced settings tabs. The available settings depend on the selected type of connector:
When using the tcp or upd connector type at the normalization stage, IP addresses of the assets from which the events were received will be written in the DeviceAddress event field if it is empty.
When using a wmi or wec connector, agents will be automatically created for receiving Windows events.
It is recommended to use the default encoding (UTF-8), and to apply other settings only if bit characters are received in the fields of events.
The connector resource has been added to the resource set of the collector. The created resource is only available in this resource set and is not displayed in the web interface Resources → Connectors section.
Proceed to the next step of the Installation Wizard.
Page topStep 3. Event parsing
This is a required step of the Installation Wizard. On the Event parsing tab of the Installation Wizard, select or create a normalizer resource whose settings will define the rules for converting raw events into normalized events. You can add more than one normalizer to implement complex processing logic.
When creating a new normalizer in the Installation Wizard, it will be saved in the set of resources for the collector and cannot be used in other collectors. If you want to use the same normalizer in different services, it is recommended to create it as an individual resource.
Adding a normalizer
To add an existing normalizer to a resource set:
- Click the Add event parsing button.
The Event parsing window will open with the normalizer settings and an active Normalization scheme tab.
- In the Normalizer drop-down list, select the required normalizer.
The Event parsing window will display the parameters of the selected normalizer. You can open the selected resource for editing in a new browser tab using the
button.
- Click OK.
The normalizer is displayed as a dark circle on the Event parsing tab of the Installation Wizard. Clicking on the circle will open the normalizer options for editing. When you hover over the circle, a plus sign is displayed: click on it to add more normalizers (see below).
To create a new normalizer:
- Select Create from the Normalizer drop-down list.
The Event parsing window will open with the normalizer settings and an active Normalization scheme tab.
- In the Name field, enter a unique name for the normalizer. The name must contain from 1 to 128 Unicode characters.
- In the Parsing method drop-down list, select the type of events to receive. Depending on your choice, you can use the preconfigured rules for matching event fields or set your own rules. When you select some parsing methods, additional parameter fields required for filling in may become available.
Available parsing methods:
- In the Keep raw log drop-down list, specify whether the original raw event should be stored in the newly created normalized event. Available values:
- Never—do not save the raw event This is the default setting.
- Only errors—save the raw event in the
Raw
field of the normalized event if errors occurred when parsing it. This value is convenient to use when debugging a service: in this case, every time an event has a non-emptyRaw
field, you know there was a problem. - Always—always save the raw event in the
Raw
field of the normalized event.
- In the Save extra fields drop-down list, choose whether you want to store the raw event fields in the normalized event if no mapping rules have been configured for them (see below). The data is stored in the Extra event field. By default, fields are not saved.
- Copy an example of the data you want to process to the Event examples field. This is an optional but recommended step.
Event examples can also be loaded from a TSV, CSV, or TXT file by using the Load from file button.
- Configure the mapping of the raw event fields to event fields in KUMA format In the Mapping table:
- In the Source column, provide the name of the raw event field that you want to convert into the KUMA event field.
Clicking the
button next to the field names in the Source column opens the Conversion window, in which you can use the Add conversion button to create rules for modifying the original data before they are written to the KUMA event fields.
- In the KUMA field column, select the required KUMA event field from the drop-down list. You can search for fields by entering their names in the field.
- If the name of the KUMA event field selected at the previous step begins with DeviceCustom*, you can add a unique custom label in the Label field if necessary.
New table rows can be added by using the Add row button. Rows can be deleted individually using the
button or all at once using the Clear all button.
If you have loaded data into the Event examples field, the table will have an Examples column containing examples of values carried over from the raw event field to the KUMA event field.
- In the Source column, provide the name of the raw event field that you want to convert into the KUMA event field.
- Click OK.
The normalizer is displayed as a dark circle on the Event parsing tab of the Installation Wizard. Clicking on the circle will open the normalizer options for editing. When you hover over the circle, a plus sign is displayed: click on it to add more normalizers (see below).
Enriching normalized events with additional data
You can add additional data to the newly created normalized events by creating enrichment rules in the normalizer similar to those in enrichment rule resources. These enrichment rules are stored in the normalizer resource where they were created. There can be more than one enrichment rule.
To add enrichment rules to the normalizer:
- Select the normalizer and go to the Enrichment tab in the Event parsing window.
- Click the Add enrichment button.
The enrichment rule parameter block appears. Close the parameter block using the
button.
- Select the enrichment type from the Source kind drop-down list. Depending on the selected type, you may see advanced settings that will also need to be completed.
Available Enrichment rule source types:
- In the Target field drop-down list, select the KUMA event field to which you want to write the data.
- Click OK.
Enrichment rules are added to the normalizer, and the Event parsing window is closed.
Creating a structure of normalizers
You can create several extra normalizers within a normalizer. This allows you to customize complex event handling logic.
The sequence in which normalizers are created matters: events are processed sequentially and their path is shown using arrows.
To create an extra normalizer:
- Create the initial normalizer (see above).
The created normalizer will be displayed in the window as a dark circle.
- Hover over the initial normalizer and click the plus sign button that appears.
- In the Add normalizer to normalization scheme window, specify the conditions under which the data will be sent to the extra normalizer:
- If you want to send only events with specific fields to the extra normalizer, list them in the Fields to pass into normalizer field.
- If you want to send only events in which certain fields have been assigned specific values to the extra normalizer, specify the name of the event field in the Use normalizer for events with specific event field values field and the value that should match it in the Condition value field.
The data processed by these conditions can be preconverted by clicking the
button. This opens the Conversion window, in which you can use the Add conversion button to create rules for modifying the original data before it is written to the KUMA event fields.
- Click OK.
This will open the Event parsing window, in which you can configure the rules for processing events as you did in the initial normalizer (see above). The Keep raw log parameter is not available. The Event examples field displays the values specified when the initial normalizer was created.
- Specify the extra normalizer settings similar to the initial normalizer
- Click OK.
The extra normalizer is displayed as a dark block that indicates the conditions under which this normalizer will be used. The conditions can be changed by moving your mouse cursor over the extra normalizer and clicking the button showing the pencil image. If you hover the mouse pointer over the extra normalizer, a plus button appears, which you can use to create a new extra normalizer. To delete a normalizer, use the button with the trash icon.
Proceed to the next step of the Installation Wizard.
Page topStep 4. Filtering events
This is an optional step of the Installation Wizard. The Event filtering tab of the Installation Wizard allows you to select or create a filter resource whose settings specify the conditions for filtering out irrelevant events. You can add more than one filter to a collector. You can swap the filters by dragging them by the icon as well as delete them. Filters are combined by the AND operator.
To add an existing filter to a collector resource set,
Click the Add filter button and select the required filter from the Filter drop-down menu.
To add a new filter to the collector resource set:
- Click the Add filter button and select Create from the Filter drop-down menu.
- If you want to keep the filter as a separate resource, set the Save filter toggle switch. This can be useful if you decide to reuse the same filter across different services. The toggle switch is turned off by default.
- If you toggle the Save filter switch on, enter a name for the created filter resource in the Name field. The name must contain from 1 to 128 Unicode characters.
- In the conditions section, specify the conditions that the events must meet:
- The Add condition button is used to add filtering conditions. You can select two values (two operands, left and right) and assign the operation you want to perform with the selected values. The result of the operation is either True or False.
- In the operator drop-down list, select the function to be performed by the filter.
- In the Left operand and Right operand drop-down lists, select where the data to be filtered will come from. As a result of the selection, Advanced settings will appear. Use them to determine the exact value that will be passed to the filter. For example, when choosing active list you will need to specify the name of the active list, the entry key and the entry key field.
- You can use the Match case drop-down list to choose whether the values passed to the filter should be case sensitive.
- You can use the If drop-down list to choose whether you want to create a negative filter condition.
Conditions can be deleted using the
button.
- The Add group button is used to add groups of conditions. Operator AND can be switched between AND, OR, and NOT values.
A condition group can be deleted using the
button.
- Using the Add filter button you can add existing filter resources selected in the Select filter drop-down list to the conditions. You can navigate to a nested filter resource using the
button.
A nested filter can be deleted using the
button.
- The Add condition button is used to add filtering conditions. You can select two values (two operands, left and right) and assign the operation you want to perform with the selected values. The result of the operation is either True or False.
The filter has been added.
Proceed to the next step of the Installation Wizard.
Page topStep 5. Event aggregation
This is an optional step of the Installation Wizard. The Event aggregation tab of the Installation Wizard allows you to select or create an aggregation rule resource whose settings specify the conditions for aggregating events of the same type. More than one aggregation rule can be added to a collector.
To add an existing aggregation rule to a set of collector resources:
Click the Add aggregation rule button and select the required resource from the Aggregation rule drop-down menu.
To add a new aggregation rule to a set of collector resources:
- Click the Add aggregation rule button and select Create new from the Aggregation rule drop-down menu.
- Enter the name of the newly created filter in the Name field. The name must contain from 1 to 128 Unicode characters.
- In the Threshold field, specify the number of events that should be received before the aggregation rule triggers and the events are aggregated. The default value is 100.
- In the Lifetime field, indicate how long the program should receive events for aggregation. On the timeout, the aggregation rule is triggered and a new event is created. The default value is 60.
- In the Identical fields section, use the Add field button to select the fields that will be used to identify the same types of events. Selected events can be deleted using the buttons with a cross icon.
- In the Unique fields section, you can use the Add field button to select the fields that will disqualify events from aggregation even if they have fields listed in the Identical fields section. Selected events can be deleted using the buttons with a cross icon.
- In the Sum fields section, you can use the Add field button to select the fields whose values will be summed during the aggregation process. Selected events can be deleted using the buttons with a cross icon.
- In the Filter section you can specify conditions to identify events that will be processed by the aggregation rule resource. You can select an existing filter resource from the drop-down list, or select Create new to create a new filter.
Aggregation rule added. You can delete it using the button.
Proceed to the next step of the Installation Wizard.
Page topStep 6. Event enrichment
This is an optional step of the Installation Wizard. On the Event enrichment tab of the Installation Wizard, you can specify which data from which sources should be added to events processed by the collector. You can enrich events with data received using LDAP or via enrichment rules.
LDAP enrichment
To enable enrichment using LDAP:
- Click Add enrichment with LDAP data.
This opens the settings block for LDAP enrichment.
- In the LDAP accounts mapping settings block, use the New domain button to specify the domain of the user accounts. You can specify multiple domains.
- In the LDAP enrichment mapping table, define the rules for mapping KUMA requests to LDAP responses:
- In the KUMA field column, indicate the KUMA event field whose data should be sent to LDAP.
- In the LDAP attribute to receive column, indicate the type of data to send to LDAP.
- In the KUMA event field to write to column, indicate which field of a KUMA event should receive data obtained from LDAP.
You can use the New line button to add a string to the table, and can use the
button to remove a string. You can use the Apply default mapping button to fill the mapping table with standard values.
Event enrichment rules for data received from LDAP were added to the group of resources for the collector.
If you add an enrichment to an existing collector using LDAP, then you must stop and restart the service.
Rule-based enrichment
There can be more than one enrichment rule. You can add them by clicking the Add enrichment button and can remove them by clicking the button. You can use existing resources of enrichment rules or create rules directly in the Installation Wizard.
To add an existing enrichment rule to a set of resources:
- Click Add enrichment.
This opens the response rule settings block.
- In the Enrichment rule drop-down list, select the relevant resource.
The enrichment rule is added to the set of resources for the collector.
To create a new enrichment rule in a set of resources:
- Click Add enrichment.
This opens the response rule settings block.
- In the Enrichment rule drop-down list, select Create.
- In the Source kind drop-down list, select the source of data for enrichment and define its corresponding settings:
- In the Target field drop-down list, select the KUMA event field to which you want to write the data.
- Use the Debug drop-down list to indicate whether or not to enable logging of service operations. Logging is disabled by default.
- In the Filter section, you can specify conditions to identify events that will be processed by the enrichment rule resource. You can select an existing filter resource from the drop-down list, or select Create new to create a new filter.
The new enrichment rule was added to the set of resources for the collector.
Proceed to the next step of the Installation Wizard.
Page topStep 7. Routing
This is an optional step of the Installation Wizard. On the Routing tab of the Installation Wizard, you can select or create destination resources with parameters indicating where the events processed by the collector should be redirected. Typically, events from the collector are routed to two points: to the correlator to analyze and search for threats; and to the storage, both for storage and so that processed events can be viewed later. Events can be sent to other locations as needed. There can be more than one destination point.
To add an existing destination to a collector resource set:
- In the Add destination drop-down list, select the type of destination resource you want to add:
- Select Storage if you want to configure forwarding of processed events to the storage.
- Select Correlator if you want to configure forwarding of processed events to a correlator.
- Select Other if you want to send events to other locations.
This type of resource includes correlator and storage services that were created in previous versions of the program.
The Add destination window opens where you can specify parameters for events forwarding.
- In the Destination drop-down list, select the necessary destination.
The window name changes to Edit destination, and it displays the settings of the selected resource. The resource can be opened for editing in a new browser tab using the
button.
- Click Save.
The selected destination is displayed on the Installation Wizard tab. A destination resource can be removed from the resource set by selecting it and clicking Delete in the opened window.
To add a new destination resource to a collector resource set:
- In the Add destination drop-down list, select the type of destination resource you want to add:
- Select Storage if you want to configure forwarding of processed events to the storage.
- Select Correlator if you want to configure forwarding of processed events to a correlator.
- Select Other if you want to send events to other locations.
This type of resource includes correlator and storage services that were created in previous versions of the program.
The Add destination window opens where you can specify parameters for events forwarding.
- Specify the settings on the Basic settings tab:
- In the Destination drop-down list, select Create.
- In the Name field, enter a unique name for the destination resource. The name must contain from 1 to 128 Unicode characters.
- Use the Disabled toggle button to specify whether events will be sent to this destination. By default, sending events is enabled.
- Select the Type for the destination resource:
- Select storage if you want to configure forwarding of processed events to the storage.
- Select correlator if you want to configure forwarding of processed events to a correlator.
- Select nats, tcp, http, kafka, or file if you want to configure sending events to other locations.
- Specify the URL to which events should be sent in the hostname:<API port> format.
You can specify multiple destination URLs using the URL button for all types except nats and file, if your KUMA license includes High Level Availability module.
If you have selected storage or correlator as the destination type, the URL field can be populated automatically using the Copy service URL drop-down list that displays active services of the selected type.
- For the nats and kafka types, use the Topic field to specify which topic the data should be written to. The topic name must contain from 1 to 255 Unicode characters.
- If required, define the settings on the Advanced settings tab. The available settings vary based on the selected destination resource type.
- Compression is a drop-down list where you can enable Snappy compression. By default, compression is disabled.
- Proxy is a drop-down list for proxy server resource selection.
- Buffer size field is used to set buffer size (in bytes) for the destination resource. The default value is 1 MB, and the maximum value is 64 MB.
- Timeout field is used to set the timeout (in seconds) for another service or component response. The default value is
30
. - Disk buffer size limit field is used to specify the size of the disk buffer in bytes. The default size is 10 GB.
- Storage ID is a NATS storage identifier.
- TLS mode is a drop-down list where you can specify the conditions for using TLS encryption:
- Disabled (default)—do not use TLS encryption.
- Enabled—encryption is enabled, but without verification.
- With verification—use encryption with verification that the certificate was signed with the KUMA root certificate. The root certificate and key of KUMA are created automatically during program installation and are stored on the KUMA Core server in the folder /opt/kaspersky/kuma/core/certificates/.
When using TLS, it is impossible to specify an IP address as a URL.
- URL selection policy is a drop-down list in which you can select a method for determining which URL to send events to if several URLs have been specified:
- Any
- Prefer first
- Round robin
- Delimiter is used to specify the character delimiting the events. By default,
\n
is used. - Path—the file path if the file destination type is selected.
- Flush interval sets the time (in seconds) between sending data to the destination resource. The default value is
100
. - Workers—this field is used to set the number of services processing the queue. By default, this value is equal to the number of vCPUs of the KUMA Core server.
- You can set health checks using the Health check path and Health check timeout fields. You can also disable health checks by selecting the Health Check Disabled check box.
- Debug—a drop-down list where you can specify whether resource logging should be enabled. By default it is Disabled.
- The Disk buffer disabled drop-down list is used to enable or disable the use of a disk buffer. By default, the disk buffer is disabled.
- In the Filter section you can specify conditions to identify events that will be processed by the aggregation rule resource. You can select an existing filter resource from the drop-down list, or select Create new to create a new filter.
- Click Save.
The created destination resource is displayed on the Installation Wizard tab. A destination resource can be removed from the resource set by selecting it and clicking Delete in the opened window.
Proceed to the next step of the Installation Wizard.
Page topStep 8. Checking the settings
This is the required, final step of the Installation Wizard. In this step, KUMA creates a set of service resources, and based on this set, the Services are created automatically.
- The set of resources for the collector is displayed under Resources → Collectors. It can be used to create new collector services. When this set of resources changes, all services that operate based on this set of resources will start using the new parameters after the services restart. To do so, you can use the Save and restart services and Save and reload services buttons.
A set of resources can be modified, copied, moved from one folder to another, deleted, imported, and exported, like other resources.
- Services are displayed in Resources → Active services. The services created using the Installation Wizard perform functions inside the KUMA program. To communicate with external parts of the network infrastructure, you need to install similar external services on the servers and assets intended for them. For example, an external collector service should be installed on a server intended as an events recipient, external storage services should be installed on servers that have a deployed ClickHouse service, and external agent services should be installed on the Windows assets that must both receive and forward Windows events.
To finish the Installation Wizard:
- Click Create and save service.
The Setup validation tab of the Installation Wizard displays a table of services created based on the set of resources selected in the Installation Wizard. The lower part of the window shows examples of commands that you must use to install external equivalents of these services on their intended servers and assets.
For example:
/opt/kaspersky/kuma/kuma collector --core https://kuma-example:<port used for communication with the KUMA Core> --id <service ID> --api.port <port used for communication with the service> --install
The port for communication with the KUMA Core, the service ID, and the port for communication with the service are added to the command automatically. You should also ensure the network connectivity of the KUMA system and open the ports used by its components if necessary.
- Close the Wizard by clicking Save collector.
The collector service is created in KUMA. Now you will install a similar service on the server intended for receiving events.
If a wmi or wec connector was selected for collectors, you must also install the automatically created KUMA agents.
Page top