Managing the dynamics of data accumulation

The operation of the Kaspersky Container Security components results in accumulation of a large amount of data, which require considerable disk space resources for storage. You can manage the dynamics of data accumulation by limiting the storage period and cleaning the database.

The following components of the solution carry the greatest load on the disk space:

• PostgreSQL DBMS
• S3 compatible file storage
• ClickHouse DBMS

PostgreSQL DBMS

We recommend that you contact the solution deployment engineer or Technical Support to connect to the PostgreSQL DBMS using the port forward option and the vacuum command.

You can reduce the risk of full database filling by adjusting the storage period of the most resource-intensive data: scan results and event logs. For this purpose, during the middleware (kcs-middleware) deployment you must specify the values of the following variables:

• EVENT_LIFETIME_HOURS is a variable that defines the storage time for records in the event log.
• SCAN_LIFETIME_HOURS is a variable that determines the storage period for scan results.

Variable values are indicated in hours, the minimum allowable value is 1 hour. The default value is 2160 hours (90 days).

Before the middleware deployment with the adjusted values of the specified variables, you must stop the operation of the middleware (kcs-middleware) and the agent broker (kcs-ab). Otherwise, these components continue to process data while cleaning is conducted and may block this process.

In test (pilot) infrastructures where data integrity and consistency is not required or data can even be lost, you can use a faster cleaning method. To do this, you must delete PostgreSQL PV and create PostgreSQL PV again without data. The created Persistent Volume can be of the same size or larger.

S3-compatible file storage

The solution uses s3-compatible file storage only to store report files.

To clean up the storage when it is full, the cluster administrator must perform the following:

1. Connect to the Minio file storage component (kcs-s3) using the port forward option.
2. Download all reports and, if necessary, save them in another place for further storage.
3. Delete data.

If necessary, you can increase Persistent Volume by using standard cluster tools.

ClickHouse DBMS

The settings of the tables in the Clickhouse database of the solution require their constant clearing. If the load in the infrastructure is very high, resources may not have time to be cleaned. In this case, you can increase Persistent Volume by using standard cluster tools.

You must organize monitoring of free disk space and the dynamics of its use independently using third-party tools approved within the framework of the solution infrastructure.