eG Monitoring
 

Measures reported by DatadedupStatusTest

Once the Data Deduplication feature is enabled for a volume, the Data Deduplication engine can potentially process all of the data on the selected volume (except a file size less than 32 KB, files in folders that are excluded, or files that have age settings applied). The deduplication engine involves finding and removing duplication within volume data without compromising its fidelity or integrity. After the volume is enabled for deduplication and the data is optimized, the volume contains the following:

  • Unoptimized files - For example, unoptimized files could include files that do not meet the selected file-age policy setting, system state files, alternate data streams, encrypted files, files with extended attributes, files smaller than 32 KB, other reparse point files, or files in use by other applications (the “in use” limit is removed in Windows Server 2012 R2).

  • Optimized files - Files that are stored as reparse points that contain pointers to a map of the respective chunks in the chunk store that are needed to restore the file when it is requested.

  • Chunk store - Location for the optimized file data.

  • Additional free space - The optimized files and chunk store occupy much less space than they did prior to optimization.

Using this test, administrators can find out the statistics related to the above-said files. This test reveals the space utilization on each volume and the size of the optimized files, datastores and chunks.

This test is disabled by default. To enable the test, select the ENABLE / DISABLE option from the Tests menu of the Agents tile in the Admin tile menu. Select Microsoft Windows as the Component type, and pick Performance as the Test type. From the list of DISABLED TESTS, pick this test and click the < button to enable it. Finally, click Update.

Outputs of the Test: One set of results for each deduplication volume on the target host.

The measures made by this test are as follows:

Measurement Description Measurement Unit Interpretation
Capacity Indicates the total capacity of this volume. GB  
Free_space Indicates the amount of space available for use on this volume. GB A high value is desired for this measure.
Used_space Indicates the amount of space utilized on this volume. GB  
Unoptimized_space Indicates the total logical size of all (optimized and non-optimized) files on this volume. GB  
Saved_space Indicates the difference between the logical size of the optimized files and the logical size of the chunk store (i.e. the sum of the deduplicated user data and deduplication metadata). GB  
Savings_rate Indicates the percentage of deduplication saved space on this volume. Percent  
Optimized_files_count Indicates the number of the optimized files on this volume. Number  
Optimized_files_size Indicates the total size of the all optimized files on this volume. GB  
Optimized_file_save_rate Indicates the percentage of space utilized for saving the optimized files on this volume. Percent  
Inpolicy_files_count Indicates the number of files that are currently qualifies for optimization. Number  
Inpolicy_files_size Indicates the total size of files that are currently qualifies for optimization. GB  
Last_optimization_result Indicates the result of an optimization job that was run last on this volume.   The values that this measure can report and their corresponding numeric values are discussed in the table above:

Numeric Value Measure Value
0 Success
1 Failure

Note:

By default, this measure reports the Measure Values listed in the table above. In the graph of this measure however, the value of this measure is represented using their numeric equivalents only.
Last_garb_collect_result Indicates the result of an garbage collection job that was run last on this volume.   The values that this measure can report and their corresponding numeric values are discussed in the table above:

Numeric Value Measure Value
0 Success
1 Failure

Note:

By default, this measure reports the Measure Values listed in the table above. In the graph of this measure however, the value of this measure is represented using their numeric equivalents only.
Last_scrubbing_resul Indicates the result of an scrubbing job that was run last on this volume.   The values that this measure can report and their corresponding numeric values are discussed in the table above:

Numeric Value Measure Value
0 Success
1 Failure

Note:

By default, this measure reports the Measure Values listed in the table above. In the graph of this measure however, the value of this measure is represented using their numeric equivalents only.
Usage_type Indicates the type of data to be stored in this volume.   The values that this measure can report and their corresponding numeric values are discussed in the table above:

Numeric Value Measure Value
1 Defalt
2 HyperV
3 Backup

Note:

By default, this measure reports the Measure Values listed in the table above. In the graph of this measure however, the value of this measure is represented using their numeric equivalents only.
Minimum_files_ages Indicates the minimum number of days since users have accessed a file before the deduplication engine optimizes the file. Number  
Minimum_files_size Specifies the minimum size threshold for files that are to be optimized. GB The deduplication engine optimizes the files that meet the minimum size threshold.
No_compress Indicates whether/not the data is compressed after deduplication on this volume.   The values that this measure can report and their corresponding numeric values are discussed in the table above:

Numeric Value Measure Value
0 No
1 Yes

Note:

By default, this measure reports the Measure Values listed in the table above. In the graph of this measure however, the value of this measure is represented using their numeric equivalents only.
Chunk_redundancy_thresh Indicates the chunk redundancy threshold set for this volume.   This measure specifies that if the data deduplication engine discovers 50 chunks of identical data, it makes one redundant copy as a safeguard.
Verify Indicates whether/not byte-by-byte verification is performed for each duplicated chunk.   The values that this measure can report and their corresponding numeric values are discussed in the table above:

Numeric Value Measure Value
0 No
1 Yes

Note:

By default, this measure reports the Measure Values listed in the table above. In the graph of this measure however, the value of this measure is represented using their numeric equivalents only.
Optimized_in_use_files Indicates whether the files in this volume are optimized or not.   The values that this measure can report and their corresponding numeric values are discussed in the table above:

Numeric Value Measure Value
0 No
1 Yes

Note:

By default, this measure reports the Measure Values listed in the table above. In the graph of this measure however, the value of this measure is represented using their numeric equivalents only.
Optimized_partial_files Indicates whether the files in this volume are partially optimized.   The values that this measure can report and their corresponding numeric values are discussed in the table above:

Numeric Value Measure Value
0 No
1 Yes

Note:

By default, this measure reports the Measure Values listed in the table above. In the graph of this measure however, the value of this measure is represented using their numeric equivalents only.
Data_chunk_count Indicates the number of data chunks in a container. Number  
Data_container_count Indicates the number of containers in the data store. Number  
Data_chunk_average_size Indicates the average size of data chunk in the data store. GB  
Data_chunk_median_size Indicates the number of data streams in a container. GB  
Data_store_uncompact_fre Indicates the amount of uncompacted space that is available for use on this volume. GB  
Stream_map_chunk_count Indicates the number of stream map chunks in a container. Number  
Stream_map_contain_count Indicates the number of containers in the stream map store. Number  
Stm_map_avg_data_chnk_ct Indicates the stream map store size divided by the total number of streams in the store. GB  
Stm_map_med_data_chnk_ct Indicates the number of median stream chunks stored in this volume. Number  
Stm_map_max_data_chnk_ct Indicates the maximum number of stream map chunks that can be stored in this volume. Number  
Hotspot_chunk_count Indicates the number of hotspots in a container. Number  
Hotspot_container_count Indicates the number of hotspots containers in the stream map store. Number  
Hotspot_median_refer_cnt Indicates the number of median hotspot references. Number  
Corruption_log_entry_cnt Indicates the number of log entries on data corruptions on this volume. Number

Some of the most common causes for deduplication to report corruption are:

  • Incompatible Robocopy options used when copying data

  • Incompatible Backup/Restore program used on a dedup volume

  • Migrating a deduplicated volume to a down-level Windows Server version

  • Enabling compression on volume roots also enabled with deduplication

  • Hardware issues

  • File System corruption

Ideally, a low value is desired for this measure. A sudden/gradul increase in the value of this measure indicates decrease in data integrity of the volume.
Total_chunk_store_size Indicates total chunk store size on this volume. GB The chunk store is an organized series of container files in the System Volume Information folder that Data Deduplication uses to uniquely store chunks.