eG Monitoring
 

Measures reported by ESIndexingTest

This test auto-discovers the indexes in the Elasticsearch cluster, and reports the current health and size of each index. In addition, this test also reveals the count of refresh/merge/flush operations performed in each index and the time taken for performing those operations. With the help of these metrics, administrators can figure out how quickly the refresh/merge/flush operations were performed, detect the slowness in time and proactively prevent the bottleneck. Besides, this test sheds light on the number of documents added to/deleted from each index using which the administrators can find out the growing/declining size of each index. In the process, the details on the shards in each index are also reported.

Output of the test: One set of results for each index in the target Elasticsearch cluster.

This test also reports metrics for the Summary descriptor.

The measures made by this test are as follows:

Measurement Description Measurement Unit Interpretation
Index_Health Indicates the current health of this index.   This measure reveals the health of this index at shard level. The numeric values that correspond to the measure values mentioned above are as follows:

Measure Value Numeric Value Description
Red 0 Indicates that the specific shard is not allocated to any node in the cluster
Yellow 1 Indicates that the primary shard is allocated but replicas are not assigned to any node.
Green 2 Indicates that all shards in the cluster are allocated to the nodes.

Note:

This test typically reports the Measure Values listed in the table above to indicate the current health of each index. However, the graph of this measure is represented using the numeric equivalents only.

This measure is not applicable for Summary descriptor.
Indexing_Rate Indicates the rate at which the documents were indexed in this index.

For Summary descriptor, this measure indicates the rate at which the documents were indexed in the indexes of the cluster. 		Documents/sec A gradual/sudden increase in the value of this measure indicates that the index size growing as well. In such cases, administrators may need to allocate more space to the index.
Index_Size Indicates the current size of this index.

For Summary descriptor, this measure indicates the total size of the indexes of the cluster. 		MB  
Indexed_Document_Count Indicates the number of documents that were newly added to this index during last measurement period.

For Summary descriptor, this measure indicates the number of documents that were newly added to the indexes of the cluster. 		Number A gradual/sudden increase in the value of this measure indicates the increase in the size of each index.
Total_Indexed_Document_Count Indicates the total number of documents that have been added to this index since this index was created.

For Summary descriptor, this measure indicates the number of documents that are added to the indexes of the cluster. 		Number  
Deleted_Document_Count Indicates the number of documents that were deleted from this index during last measurement period.

For Summary descriptor, this measure indicates the number of documents that were deleted from the indexes. 		Number  
Total_Deleted_Document_Count Indicates the total number of documents that are deleted from this index since the index was created.

For Summary descriptor, this measure indicates the total number of documents that are deleted from the indexes. 		Number  
Refresh_Count Indicates the number of refresh operations performed for this index.

For Summary descriptor, this measure indicates the total number of refresh operations performed for the indexes in the cluster. 		Number An elasticsearch refresh makes the documents available for search whenever the new documents added to the index. The refresh operation is performed at regular intervals. A very low value of this measure denotes that the newly added documents may not be available for the search.
Refreh_Time Indicates the time taken for performing refresh operations for this index.

For Summary descriptor, this measure indicates the time taken for performing refresh operations in the cluster. 		Seconds Compare the value of this measure across the indexes to know the index that took maximum time to refresh.
Time_Spent_Per_Refresh Indicates the average time taken for performing a single refresh operation for this index.

For Summary descriptor, this measure indicates the time taken for performing a single refresh operation in the cluster. 		Seconds A low value is desired for this measure.
Merge_Count Indicates the number of merge operations performed in this index.

For Summary descriptor, this measure indicates the number of merge operations performed in the cluster. 		Number A shard in the index is broken down into segments. Segments are internal storage elements in the index where the index data is stored, and are immutable. Smaller segments are periodically merged into larger segments to keep the index size at bay and to expunge deletes.
Merge_Time Indicates the average time taken for performing the merge operations for this index.

For Summary descriptor, this measure indicates the average time taken for performing the merge operations in the cluster.		 Seconds Comparing the value of this measure across the indexes will reveal the index that took maximum time to perform merge operations.
Time_Spent_Per_Merge Indicates the time taken for performing a single merge operation in this index.

For Summary descriptor, this measure indicates the time taken for perperforming a single merge operation in the cluster		 Seconds A low value is desired for this measure.
Flush_Count Indicates the number of flush operations performed in this index.

For Summary descriptor, this measure indicates the number of flush operations performed in the cluster.		 Number The flush process is performed to ensure that any data that is currently only persisted in the transaction log is also permanently persisted in Lucene search library. This reduces recovery times as that data doesn't need to be reindexed from the transaction logs after the Lucene indexed is opened. By default, Elasticsearch uses heuristics in order to automatically trigger flushes as required. A very low value of this measure may indicate the increase in recovery time of the index.
Time_Spent_Per_Flush Indicates the time taken to perform a single flush operation in this index.

For Summary descriptor, this measure indicates the time taken for performing a single flush operation in the cluster.		 Seconds  
Flush_Time Indicates the average time taken for performing the flush operations in this index.

For Summary descriptor, this measure indicates the average time taken for performing the flush operations in the cluster. 		Seconds  
Index_Primary_Shards Indicates the number of primary shards in this index. Number This measure will not be reported for the Summary descriptor.
Index_Replica_Shards Indicates the number of replica shards created for each primary shard in this index. Number This measure will not be reported for the Summary descriptor.
Index_Active_Primary_Shards Indicates the number of primary that are currently active in this index. Number This measure will not be reported for the Summary descriptor.
Index_Active_Shards Indicates the total number of active shards in this index. Number This measure will not be reported for the Summary descriptor.
Index_Relocating_Shards Indicates the number of relocating shards in this index. Number This measure will not be reported for the Summary descriptor.
Index_Initializing_Shards Indicates the number of shards that are currently in initializing state in this index. Number This measure will not be reported for the Summary descriptor.
Index_Unassigend_Shards Indicates the number of unassigned shards in this index. Number This measure will not be reported for the Summary descriptor.
Index_Green_Count Indicates the number of indexes in the normal state. Number This measure is only applicable for the Summary descriptor.

The detailed diagnosis of this measure reveals the name of the indexes that are in the normal state.
Index_Warning_Count Indicates the number of indexes in the warning state. Number This measure is only applicable for the Summary descriptor.

The detailed diagnosis of this measure reveals the name of the indexes that are in the warning state.
Index_Red_Count Indicates the number of indexes in the critical state. Number This measure is only applicable for the Summary descriptor.

The value of this measure should be very low. The detailed diagnosis of this measure reveals the name of the indexes that are in the critical state.