eG Monitoring
 

Measures reported by AzrStorageTest

Azure Storage is a Microsoft-managed cloud service that provides storage that is highly available, secure, durable, scalable and redundant.

An Azure storage account is an access point to all the elements that compose the Azure storage realm. Once the user creates the storage account, they can select the level of resilience needed and Azure will take care of the rest. A single storage account can store up to 500TB of data.

With an Azure storage account, you have access to the following types of storage:

  • Azure Blob Storage: Blob Storage is Microsoft Azure's service for storing binary large objects or blobs which are typically composed of unstructured data such as text, images, and videos, along with their metadata.

  • Azure Table Storage: Azure Table Storage is a scalable, NoSQL, key-value data storage system that can be used to store large amounts of data in the cloud. This storage offering has a schemaless design, and each table has rows that are composed of key-value pairs.

  • Azure Queue Storage: Azure Queue Storage is a service that allows users to store high volumes of messages, process them asynchronously and consume them when needed while keeping costs down by leveraging a pay-per-use pricing model.

  • Azure File Storage: Azure Files is a shared network file storage service that provides administrators a way to access native SMB file shares in the cloud. The Azure File service provides a way for applications running on cloud VMs to share files among them by using standard protocols like WriteFile or ReadFile.

Also, by configuring geo-redundant storage accounts, you can have data in your primary storage accounts copied to a second region. This way, if the primary region is unavailable, you can initiate a fail over to the secondary region, thereby ensuring the uninterrupted delivery of storage services.

In summary, what makes Azure Storage the most coveted storage platform is its large storage capacity, reliability, versatility (ability to handle heterogeneous storage types), and cost effectiveness. Additionally, because Azure storage eases cloud application development, it has found favor with cloud application developers as well. This means that any issue that undermines the performance of a storage account can result in unexpected application outages / poor application performance. Such issues can include, the reduced availability of the Azure storage service, the lack of adequate storage space in an account, poor responsiveness of an account to requests, etc. If such applications front-end your critical business services, then service quality will suffer, causing SLA violations, loss of revenue, and a rise in penalities. To avoid this, it is imperative that administrators monitor the availability, usage, and processing ability of each Azure storage account that is configured for a target subscription, quickly capture abnormalities, and rapidly initiate measures to right the wrongs before application performance degrades. This is where the AzrStorageTest helps!

This test automatically discovers the Azure storage accounts configured for the monitored Azure subscription. For each discovered account, the test then reports the provisioning state of that account. Accounts that are in an Error state are highlighted in the process. The test also evaluates how resilient each storage account is by monitoring the availability of the primary and secondary locations; alerts are sent out if both locations are unavailable. The test also measures how quickly/otherwise each storage account processes requests, thus leading you to the latent accounts. Furthermore, the test provides real-time insights into usage and processing ability ofeach storage type. In the process, when unusual space usage and abnormal slowness is observed in an account, administrators will be able to accurately tell which type of storage is at the root of the problem - blob? table? file? or queue?

This way, the test not only leads administrators to the problem areas of a storage account, it also reveals the probable cause of these problems, thereby enabling administrators to act quickly and eliminate the issues.

Outputs of the test :One set of results for each Azure Storage account configured for every resource group in the target Azure Subscription

The measures made by this test are as follows:

Measurement Description Measurement Unit Interpretation
Prvsng_state Indicates the current provisioning status of this account.   The values reported by this measure and its numeric equivalents are mentioned in the table below:

Measure Value Numeric Value
Succeeded 1
Updating 2
Error 3
Unknown 0


Note:

By default, this measure reports the Measure Values listed in the table above to indicate the current provisioning status of a storage account. In the graph of this measure however, the same is represented using the numeric equivalents only.

Use the detailed diagnosis of this measure to know the location of the storage, the storage type, when it was created, and more.
Prmry_status Indicates whether/not the primary location of this storage account is available.   The values reported by this measure and its numeric equivalents are mentioned in the table below:

Measure Value Numeric Value
Available 1
Unavailable 0


Note:

By default, this measure reports the Measure Values listed in the table above to indicate whether/not the storage account is available in its primary location. In the graph of this measure however, the same is represented using the numeric equivalents only.

If the value of this measure is Unavailable for any storage account, it is a cue to administrators to initiate a fail over to the secondary location.
Scndry_status Indicates whether/not the secondary location of this storage account is available.   The values reported by this measure and its numeric equivalents are mentioned in the table below:

Measure Value Numeric Value
Available 1
Unavailable 0


Note:

By default, this measure reports the Measure Values listed in the table above to indicate whether/not the secondary location of the storage account is available. In the graph of this measure however, the same is represented using the numeric equivalents only.

If both the Primary status and Secondary status measures report the value Unavailable for a storage account, it is worrisome. This is because, it implies that all copies of the storage account are inaccessible to applications. This can bring application operations to a stand-still!
Usd_cpcty Indicates the amount of storage space in this account that is in use currently. MB Compare the value of this measure across accounts to identify the resource-hungry storage accounts. Make sure that such accounts are sized with sufficient storage space to fulfill the space requirements of dependent applications.

If a storage account is found to be space-hungry, then compare the value of the Used file capacity, Used blob capacity, Used table capacity, and Used queue capacity measures of that storage account, to know what type of storage is hogging the storage space and is contributing to the abnormal usage.
File_capacity Indicates the amount of storage space in this account that is used by file storage. MB If the Storage account used capacity measure reports an unusually high value for an account, then compare the value of this measure with that of the Used blob capacity, Used table capacity, and Used queue capacity measures to accurately identify the type of storage that is responsible for the abnormal usage - is it file storage? blob storage? table storage? or queue storage?
Blob_capacity Indicates the amount of storage space in this account that is used by blob storage. MB If the Storage account used capacity measure reports an unusually high value for an account, then compare the value of this measure with that of the Used file capacity, Used table capacity, and Used queue capacity measures to accurately identify the type of storage that is responsible for the abnormal usage - is it file storage? blob storage? table storage? or queue storage?
Table_capacity Indicates the amount of storage space in this account that is used by table storage. MB If the Storage account used capacity measure reports an unusually high value for an account, then compare the value of this measure with that of the Used file capacity, Used blob capacity, and Used queue capacity measures to accurately identify the type of storage that is responsible for the abnormal usage - is it file storage? blob storage? table storage? or queue storage?
Queue_capacity Indicates the amount of storage space in this account that is used by queue storage. MB If the Storage account used capacity measure reports an unusually high value for an account, then compare the value of this measure with that of the Used blob capacity, Used table capacity, and Used file capacity measures to accurately identify the type of storage that is responsible for the abnormal usage - is it file storage? blob storage? table storage? or queue storage?
Trnsctn Indicates the number of requests made to this storage account across all storage types. Number  
Ingress Indicates the amount of ingress data consumed by this storage account across all storage types. MB A default ingress limit is set for different types of storage accounts. The ingress limit refers to the maximum amount of data that a storage type can receive. If the value of this measure is unusually high for any storage account, then compare the value of the File share ingress, Blob ngress, Table ingress, and Queue ingress measures to know which storage type is receiving the maximum amount of data.
Egress Indicates the amount of egress data consumed by this storage account across all storage types. MB A default egress limit is set fordifferent types of storage accounts. The egress limit refers to the maximum amount of data that a storage type can send.

If the value of this measure is unusually high for any storage account, then compare the value of the File share egress, Blob egress, Table egress, and Queue egress measures to know which storage type is sending the maximum amount of data.
Succss_srvr_ltncy Indicates the average time taken by the storage types in this storage account to process requests successfully. Seconds Server latency is the interval from when an Azure storage account receives the last packet of the request until the first packet of the response is returned from that account. In simpler terms, it means the time taken by the Azure storage account to process any given request.

End-to-end latency is the interval from when the Azure storage account receives the first packet of the request until that storage account receives a client acknowledgment on the last packet of the response. In simpler terms it means the round trip of any operation starting at the client application, plus the time taken for processing the request at the storage account and then coming back to the client application.

If you find that end-to-end latency is significantly higher than server latency, then investigate and address the source of the additional latency.

The main factor influencing end-to-end latency is operation size. It takes longer to complete larger operations, due to the amount of data being transferred over the network and processed by Azure Storage. Client configuration factors such as concurrency and threading also affect latency. Client resources including CPU, memory, local storage, and network interfaces can also affect latency.

If the server latency is abnormally high, then it could be because the storage sub-system is experiencing serious processing deficiencies. In this case, you may want to know which specific storage type is unable to service requests quickly - is it blob storage? table storage? file storage? or queue storage? For this, compare the value of the File share success server latency, Blob success server latency, Table success server latency, and Queue success server latency measures.
Succss_e2e_ltncy Indicates that average end-to-end latency of successful requests made to this storage account. Seconds
Availblty Indicates whether/not this storage account is available currently. Percent Ideally, the value of this measure should be 100%. This means that this storage account is available. Any value lower than 100% is a cause for concern, as it implies reduced availability of the storage service. All unexpected errors result in reduced availability.
File_cntr_count Indicates the number of file shares in this storage account. Number Use the detailed diagnosis of this measure to know which file shares are in this account. For each file share, the detailed diagnostics further reveals when that file share was last modified, the lease state and duration, the access tier and its status, the protocols enabled, the share size quota, the file share usage (in bytes), and when a snapshot of that share was taken.
File_count Indicates the number of files in this storage account. Number  
Snapshot_count Indicates the number of snapshots present on the share in this storage account's Azure Files service. Number Azure Files provides the capability to take share snapshots of file shares. Share snapshots capture the share state at that point in time.
Capacity_quota Indicates the upper limit on the amount of storage that can be used by this storage account's Azure Files Service. MB If the value of the Used file capacity measure is equal or close to the value of this measure, it means that the capacity quota has been / is about to be exhausted. In other words, it implies that the Azure Files service is about to run out of free storage space. To avoid this, you may either want to delete unwanted files and make room, or increase the storage capacity quota.
File_trnsctn Indicates the number of requests made to this storage account's Azure Files Service. Number  
File_ingress Indicates the amount of ingress data consumed by this storage account's Azure Files Service. MB A default ingress limit is set for File storage. The ingress limit refers to the maximum amount of data that File storage can receive.

By comparing the value of this measure with the ingress limit, you can be forewarned if the capacity limits set for your File storage need to be increased to ensure peak storage performance.
File_egress Indicates the amount of egress data consumed by this storage account's Azure Files Service. MB A default egress limit is set for File storage. The egress limit refers to the maximum amount of data that File storage can send.

By comparing the value of this measure with the egress limit, you can be forewarned if the capacity limits set for File storage need to be increased to ensure peak storage performance.
File_srvr_latncy Indicates the average time taken by the File storage in this account to process requests successfully. Seconds Server latency of File storage is the interval from when File storage receives the last packet of the request until the first packet of the response is returned from that storage. In simpler terms, it means the time taken by File storage to process any given request.

End-to-end latency of File storage is the interval from when the File storage receives the first packet of the request until that storage receives a client acknowledgment on the last packet of the response. In simpler terms it means the round trip of any operation starting at the client application, plus the time taken for processing the request at the File storage and then coming back to the client application.

If you find that end-to-end latency is significantly higher than server latency, then investigate and address the source of the additional latency.

The main factor influencing end-to-end latency is operation size. It takes longer to complete larger operations, due to the amount of data being transferred over the network and processed by File storage. Client configuration factors such as concurrency and threading also affect latency.

Client resources including CPU, memory, local storage, and network interfaces can also affect latency.

If the server latency is abnormally high, then it could imply that the File storage is experiencing serious processing deficiencies.
File_e2e_latncy Indicates that average end-to-end latency of successful requests made to the File storage in this account. Seconds
File_availability Indicates the percentage availability of this storage account's Azure Files Service. Percent Ideally, the value of this measure should be 100%. This means that the File storage service is available. Any value lower than 100% is a cause for concern, as it implies reduced availability of the service. All unexpected errors result in reduced availability.
Blob_cntr_count Indicates the number of containers in this storage account. Number Use the detailed diagnosis of this measure to know which blobs are in this account. For each blob, the detailed diagnostics further reveals when that file blob was last modified, the lease state and duration, the configured retention period, the version, and more.
Blob_count Indicates the number of blob objects stored in this storage account. Number  
Index_capacity Indicates the total of blob storage used in this storage account. MB  
Blob_trnsctn Indicates the number of requests made to the Blob storage in this storage account. Number  
Blob_ingress Indicates the amount of ingress data consumed by this storage account's Blob storage. MB A default ingress limit is set for Blob storage. The ingress limit refers to the maximum amount of data that Blob storage can receive.

By comparing the value of this measure with the ingress limit, you can be forewarned if the capacity limits set for your Blob storage need to be increased to ensure peak storage performance.
Blob_egress Indicates the amount of egress data consumed by this storage account's Blob storage. MB A default egress limit is set for Blob storage. The egress limit refers to the maximum amount of data that Blob storage can send.

By comparing the value of this measure with the egress limit, you can be forewarned if the capacity limits set for Blob storage need to be increased to ensure peak storage performance.
Blob_srvr_latncy Indicates the average time taken by the Blob storage in this account to process requests successfully. Seconds Server latency of Blob storage is the interval from when Blob storage receives the last packet of the request until the first packet of the response is returned from that storage. In simpler terms, it means the time taken by Blob storage to process any given request.

End-to- end latency of Blob storage is the interval from when the Blob storage receives the first packet of the request until that storage receives a client acknowledgment on the last packet of the response. In simpler terms it means the round trip of any operation starting at the client application, plus the time taken for processing the request at the Blob storage and then coming back to the client application.

If you find that end-to-end latency is significantly higher than server latency, then investigate and address the source of the additional latency.

The main factor influencing end-to-end latency is operation size. It takes longer to complete larger operations, due to the amount of data being transferred over the network and processed by Blob storage. Client configuration factors such as concurrency and threading also affect latency. Client resources including CPU, memory, local storage, and network interfaces can also affect latency.

If the server latency is abnormally high, then it could imply that the Blob storage is experiencing serious processing deficiencies.
Blob_e2e_latncy Indicates that average end-to-end latency of successful requests made to the File storage in this account. Seconds
Blob_availability Indicates the percentage availability of Blob storage containers in this storage account. Percent Ideally, the value of this measure should be 100%. This means that the Blob storage is available. Any value lower than 100% is a cause for concern, as it implies reduced availability of the storage. All unexpected errors result in reduced availability
Table_cntr_count Indicates the number of tables in this storage account. Number  
Table_entity_count Indicates the number of table entities in this storage account. Number An entity is a set of properties, similar to a database row. A property is a name-value pair. Each entity can include up to 252 properties to store data.
Table_trnsctn Indicates the number of requests to the Table storage in this storage account. Number  
Table_ingress Indicates the amount of ingress data consumed by this storage account's Table storage. MB A default ingress limit is set for Table storage. The ingress limit refers to the maximum amount of data that Table storage can receive.

By comparing the value of this measure with the ingress limit, you can be forewarned if the capacity limits set for your Table storage need to be increased to ensure peak storage performance.
Table_egress Indicates the amount of egress data consumed by this storage account's Table storage. MB A default egress limit is set for Table storage. The egress limit refers to the maximum amount of data that Table storage can send.

By comparing the value of this measure with the egress limit, you can be forewarned if the capacity limits set for Table storage need to be increased to ensure peak storage performance.
Table_srvr_latncy Indicates the average time taken by the Table storage in this account to process requests successfully. Seconds Server latency of Table storage is the interval from when Table storage receives the last packet of the request until the first packet of the response is returned from that storage. In simpler terms, it means the time taken by Table storage to process any given request.

End-to-end latency of Table storage is the interval from when the Table storage receives the first packet of the request until that storage receives a client acknowledgment on the last packet of the response. In simpler terms it means the round trip of any operation starting at the client application, plus the time taken for processing the request at the Table storage and then coming back to the client application.

If you find that end-to-end latency is significantly higher than server latency, then investigate and address the source of the additional latency.

The main factor influencing endto-end latency is operation size. It takes longer to complete larger operations, due to the amount of data being transferred over the network and processed by Table storage. Client configuration factors such as concurrency and threading also affect latency. Client resources including CPU, memory, local storage, and network interfaces can also affect latency.

If the server latency is abnormally high, then it could imply that the Table storage is experiencing serious processing deficiencies.
Table_e2e_latncy Indicates that average end-to-end latency of successful requests made to the File storage in this account. Seconds
Table_availability Indicates the percentage availability of this storage account's Table storage. Percent Ideally, the value of this measure should be 100%. This means that the Table storage service is available. Any value lower than 100% is a cause for concern, as it implies reduced availability of the service. All unexpected errors result in reduced availability.
Queue_cntr_count Indicates the number of queues in this storage account. Number Use the detailed diagnosis of this measure to know which queues are in this account.
Queue_msg_count Indicates the number of unexpired queue messages in this storage account. Number  
Queue_trnsctn Indicates the number of requests to the Queue storage in this storage account. Number  
Queue_ingress Indicates the amount of ingress data consumed by this storage account's Queue storage. MB A default ingress limit is set for Queue storage. The ingress limit refers to the maximum amount of data that Queue storage can receive.

By comparing the value of this measure with the ingress limit, you can be forewarned if the capacity limits set for your Queue storage need to be increased to ensure peak storage performance.
Queue_egress Indicates the amount of egress data consumed by this storage account's Queue storage. MB A default egress limit is set for Queue storage. The egress limit refers to the maximum amount of data that Queue storage can send.

By comparing the value of this measure with the egress limit, you can be forewarned if the capacity limits set for your Queue storage need to be increased to ensure peak storage performance.
Queue_srvr_latncy Indicates the average time taken by the Queue storage in this account to process requests successfully. Seconds Indicates the average time taken by the Queue storage in this account to process requests successfully.

End- to-end latency of Queue storage is the interval from when the Queue storage receives the first packet of the request until that storage receives a client acknowledgment on the last packet of the response. In simpler terms it means the round trip of any operation starting at the client application, plus the time taken for processing the request at the Queue storage and then coming back to the client application.

If you find that end-to-end latency is significantly higher than server latency, then investigate and address the source of the additional latency.

The main factor influencing end-to-end latency is operation size. It takes longer to complete larger operations, due to the amount of data being transferred over the network and processed by Queue storage. Client configuration factors such as concurrency and threading also affect latency. Client resources including CPU, memory, local storage, and network interfaces can also affect latency.

If the server latency is abnormally high, then it could imply that the Queue storage is experiencing serious processing deficiencies.
Queue_e2e_latncy Indicates that average end-to-end latency of successful requests made to the Queue storage in this account. Seconds
Queue_availability Indicates the percentage availability of this storage account's Queue storage. Percent Ideally, the value of this measure should be 100%. This means that the Queue storage service is available. Any value lower than 100% is a cause for concern, as it implies reduced availability of the service. All unexpected errors result in reduced availability.