Index Log Retention

Overview

This process allows users to delete indexes in their cluster according to the name of the index and the specific range of days (+ filter specification)

Activate log retention process

# /opt/configurator/configure
Display only the mandatory entries? (Yn): n
Configure all the modules when finished? (Yn): n

UBIqube SOC Configuration Menu

1: System configuration
2: Web Portal configuration
3: JEntreprise configuration
4: SEC Engine configuration
5: Reports configuration
6: Database configuration
7: Alarms and events notifications
8: SOC Customisation
9: Zero Touch Deployment
10: OSS BSS third party tools integration
11: ElasticSearch configuration

0: save & exit (use CTRL-C to exit without saving)
11

ElasticSearch configuration

1: cluster configuration
2: Web portal (SES and JENTREPRISE)
3: SEC Engine

0: back
1

cluster configuration

1: general
2: network settings
3: retention policy
4: discovery
5: advanced config: breakers and gateway

0: back
3

retention policy

1: manage multiple SYSLOGS retention period with custom ES search query (*|1d)
2: manage multiple NETFLOW retention period with custom ES search query (*|1d)
3: cache retention period (1w)
4: retention policy application frequency (0 0 * * *)
5: activate the log retention policy script (true/false) (true)
6: prefix of the index name that will be processed by the log retention policy script (ubilogs*)
7: prefix of the netflow index name that will be processed by the log retention policy script (ubiflows*)


Index name and filter specification (days, "field":"value",...)

# /opt/configurator/configure
Display only the mandatory entries? (Yn): n
Configure all the modules when finished? (Yn): n

UBIqube SOC Configuration Menu

1: System configuration
2: Web Portal configuration
3: JEntreprise configuration
4: SEC Engine configuration
5: Reports configuration
6: Database configuration
7: Alarms and events notifications
8: SOC Customisation
9: Zero Touch Deployment
10: OSS BSS third party tools integration
11: ElasticSearch configuration

0: save & exit (use CTRL-C to exit without saving)
11

ElasticSearch configuration

1: cluster configuration
2: Web portal (SES and JENTREPRISE)
3: SEC Engine

0: back
1

cluster configuration

1: general
2: network settings
3: retention policy
4: discovery
5: advanced config: breakers and gateway

0: back
3

retention policy

1: manage multiple SYSLOGS retention period with custom ES search query (*|1d)
2: manage multiple NETFLOW retention period with custom ES search query (*|1d)

3: cache retention period (1w)
4: retention policy application frequency (0 0 * * *)
5: activate the log retention policy script (true/false) (true)
6: prefix of the index name that will be processed by the log retention policy script (ubilogs*)
7: prefix of the netflow index name that will be processed by the log retention policy script (ubiflows*)

Note : 

For ES search query (*|1d), combination can be applied.

Example: ES search query (type:traffic|7d,*|90d) => delete logs from now - 7 days which contain "type:traffic" + all logs from now - 90 days

Concerned file: /opt/ubi-elasticsearch/log_retention_management.php