eG Monitoring
 

Measures reported by FSIndPartionTest

The overall index created by an Indexer has two levels of partitioning: index columns and index partitions.

When the complete index is too large to reside on one server, you can split it into multiple disjoint index columns. The query matching component will then evaluate the queries against all index columns within the search cluster, and merge the results from each index column into the final query hit list. Within each index column the indexer uses a partitioning of the index in order to handle large number of indexed items with low indexing and query latency. This partitioning is dynamic and handled internally on each index server. When the query matching component evaluates a query, each partition runs in a separate thread. The default number of partitions is 5. In order to handle more than 15 million items per server (column), you must configure a larger number of partitions.

To check whether sufficient indexer partitions are up and running to serve the query load effectively, you need to do the following:

  • Run periodic checks on the state of the individual partitions to identify the unavailable partitions;
  • Continuously monitor the load on the partitions and how quickly each partition processes the load, so that probable overload conditions and processing bottlenecks can be identified and fixed early.

The FSIndPartionTest test helps achieve all of the above.

The measures made by this test are as follows:

Measurement Description Measurement Unit Interpretation
FSActivDocs Indicates the number of active items on this index partition. Number The Index partitions 0 and 1 should have less than 1 million items each, preferably even less in order to keep indexing latency low. In periods with high item throughput, indexing latency will be reduced and these partitions will be larger, as this is more optimal for overall throughput. Items will although automatically be rearranged into the higher numbered partitions during periods with lighter load.
FSIndPerSecs Indicates the number of items that are indexed per second for this partition. Number A consistent decrease in the value of this measure could indicate a processing bottleneck.
FSPenExcuList Indicates the number of items that are to be excluded from indexing in this index partition. Number This translates to items that are in the index but that should not be searchable (deleted, present in other partitions, etc).
FSStatCode Indicates the current state of this partition. Number  
FSTotDocs Indicates the total number of items in this partition including the items on the excluded lists. Number This measure is a good indicator of the total load on a partition. Compare the value of this measure across partitions to isolate the overloaded partitions. If one/more partitions are found to be handling abnormally high load than the rest, it could indicate that load is unevenly distributed across the partitions. You may hence have to configure more partitions to handle the load.