eG Monitoring
 

Measures reported by MongoJournalStTest

With MMAPv1, when a write operation occurs, MongoDB updates the in-memory view. With journaling enabled, MongoDB writes the in-memory changes first to on-disk journal files. If MongoDB should terminate or encounter an error before committing the changes to the data files, MongoDB can use the journal files to apply the write operation to the data files and maintain a consistent state.

With journaling, MongoDB's storage layer has two internal views of the data set: the private view, used to write to the journal files, and the shared view, used to write to the data files:

  1. MongoDB first applies write operations to the private view.

  2. MongoDB then applies the changes in the private view to the on-disk journal files in the journal directory roughly every 100 milliseconds. MongoDB records the write operations to the on-disk journal files in batches called group commits. Grouping the commits help minimize the performance impact of journaling since these commits must block all writers during the commit. Writes to the journal are atomic, ensuring the consistency of the on-disk journal files.

  3. Upon a journal commit, MongoDB applies the changes from the journal to the shared view.

  4. Finally, MongoDB applies the changes in the shared view to the data files. More precisely, at default intervals of 60 seconds, MongoDB asks the operating system to flush the shared view to the data files.

If the mongod instance were to crash without having applied the writes to the data files, the journal could replay the writes to the shared view for eventual write to the data files.

However, if MongoDB takes too long to write changes to the journal, then, in the event that a mongod instance crashes, not all the changes will be available in the journal to enable a complete recovery. In such situations therefore, data loss is inevitable. This is why, it is imperative that administrators detect potential delays in journaling, isolate its root-cause, and resolve the bottleneck before it impacts data recovery. This is where the MongoJournalStTest test helps!

This test monitors writes to the journals and data files, reports the time taken for these writes, and proactively alerts administrators to delays in the journaling process. Using this test, administrators can also figure out at what step of the process the delay may have occurred - when writing to the journal? when committing the writes? or when writing to the data files? This way, the test enables administrators to accurately diagnose the root-cause of journaling slowness, so that they can fix it before it results in data loss.

Note:

This test reports metrics only for those MongoDB instances that use the MMAPv1 storage engine.

Outputs of the test : One set of results for the target Mongo database server being monitored.

The measures made by this test are as follows:

Measurement Description Measurement Unit Interpretation
No_of_commits Indicates the number of transactions written to the journal during the last time a group commit was performed on the journal. Number  
Journal_data_write Indicates the amount of data written to the journal during the last time a group commit was performed on the journal. MB  
Journal_prepare_time Indicates the amount of time spent preparing to write to the journal. Seconds Smaller values indicate better journal performance.
Journal_write_time The amount of time, in milliseconds, spent actually writing to the journal. Seconds Ideally, the value of this measure should be low. A consistent increase in this value is a cause for concern, as it indicates a bottleneck when writing to the journal. File system speeds and device interfaces increase the value of this measure.
Compression_ratio Indicates the compression ratio of the data written to the journal Percent A high value is desired for this measure.
Data_file_write Indicates the amount of data written from journal to data files during the last group commit. MB  
Data_file_write_time Indicates the amount of time spent writing to data files after journaling. Seconds Ideally, the value of this measure should be low. A consistent increase in this value is a cause for concern, as it indicates a bottleneck when writing to the data files. File system speeds and device interfaces increase the value of this measure.
Early_commits Indicates the number of times MongoDB requested a commit before the scheduled journal group commit interval. Seconds For the MMAPv1 storage engine, you can set the group commit interval using the --journalCommitInterval command line option. The allowed range is 2 to 300 milliseconds. Lower values increase the durability of the journal at the expense of disk performance. Use the value of this measure ensure that your journal group commit interval is not too long for your deployment.
Commits_time Indicates the amount of time spent for commits. Seconds  
Commits_in_write_lock Indicates the count of the commits that occurred while a write lock was held. Seconds Commits in a write lock indicate a MongoDB node under a heavy write load and call for further diagnosis.
Commits_time_write_locks Indicates the amount of time spent for commits that occurred while a write lock was held. Seconds