Setup

Overview

Specific packages are provided by UBiqube to run Elasticsearch with MSA.

Requirements:

  • RPM packages:
    • Elasticsearch, the RESTful search and analytics engine itself (available online on https://www.elastic.co/downloads/past-releases)
    • ubi-dms, contains the ;SNMP monitoring configuration tool and UBIqube base MIB
    • ubi-elasticsearch, specific configuration of the ES nodes (mapping templates, cluster configuration, administration scripts)
    • ubi-configurator, contains all variables used by ubi-elasticsearch package
  • Java version: Java 8
  • Each Elasticsearch is running on a Centos 6.8 64bits

ES Standalone MSA and ES D-MSA

The 2 main jobs to be achieved are indexing and searching.

For a D-MSA, the system should be able to provide dynamic scaling and HA.

In this case, the architecture will be set with 3 types of node in different servers:

  • Data node: this node stores only the data. When it receives a request from a client, it searches data from shards or creates an index.
  • Master node: this node maintains a cluster and requests indexing or search to data nodes.
  • Client node: the client node takes the search requests from the MSA web portal as well as from the MSA SecEngine event-based notification system. The client node is also used by the SecEngine to index syslogs when the amount of data to index is small.

For a standard standalone MSA, a single Elasticsearch node architecture will be used. 

It means that the log collection, indexing and searching are running on the same server run as data and master node type.