Indexing and Performance

Your Elasticsearch indexing time and system resources required may vary significantly based on the objects selected to be indexed. Likewise, custom fields should only be indexed if they will be used in search queries for your instance. Unnecessary objects and fields can result in poor indexing performance, e.g., memo fields containing large volumes of text will be indexed exponentially slower than numeric-type fields. As such, it is recommended to limit indexing to objects you plan to leverage for global search. Limiting the index to these objects (as opposed to a blanket index of the instance) will result in quicker indexing times while also reducing the load on the memory and required disk space.
 
Important: Users upgrading from previous versions of Elasticsearch (namely 1.7.4 to 5.3) must re-index their instances. Preliminary testing finds that it takes roughly one hour per 4GB for indexing. This number will vary based on the types of files present in the index. 

 

Allocated Memory

 

Never allocate all available memory to Elasticsearch. A best practice for this is to only ever allocate 50% of your available memory to Elasticsearch. The other half should be reserved for Lucene caching, which uses ANY free memory on the machine. It loads segments (inverted indices) into memory for faster searching, so definitely keep that in mind when calculating Elasticsearch memory requirements.

General Best Practices

•If multicast is disabled (which is suggested), it is a good practice to make the Master Eligible Nodes the discovery nodes.

 

•For TeamConnect instances that are 100GB, it usually suffices to have a single node cluster where the node acts as both a master and data node.

 

•For instances that require more nodes, or if the client wishes to utilize Elasticsearch's shard replication for backup and high availability, then a larger cluster is usually preferred.

 

In order to utilize shard replication, you simply need to start up another node within the same cluster (don't forget to specify the entry point into the cluster for unicast).

Please read the Master Eligible Nodes details here for more information.