eG Monitoring
 

Measures reported by DockContPerfTest

This test monitors each container available in the Docker and reports the CPU utilization, I/O processing, memory related statistics such as memory utilization, paging in/paging outs, errors that were detected etc. Using this test, administrators can easily figure out processing/memory bottlenecks and rectify the same before the users complain of slow responsiveness of the containers.

Note:

This test will be supported only for the Docker server version 1.5 and above.

Outputs of the test : One set of results for each container available in the Docker server being monitored

The measures made by this test are as follows:

Measurement Description Measurement Unit Interpretation
size Indicates the amount of disk space utilized by this container. MB The detailed diagnosis of this measure if enabled, lists the containers in this server.
rx_bytes Indicates the rate at which the data was received by this container. Mbps A sudden increase or decrease in the value of this measure could be a cause for concern.
rx_packets Indicates the rate at which packets were received by this container. Pkts/sec A significant increase or decrease in the value of this measure may alter traffic condition in the Docker server.
rx_errors Indicates the number of errors occurred while data was received by this container. Number Ideally, the value of this measure should be zero.
rx_dropped Indicates the number of packets dropped by this container during reception. Number Ideally, the value of this measure should be zero.
tx_bytes Indicates the rate at which the data was transmitted by this container. Mbps  
tx_packets Indicates the rate at which packets were transmitted by this container. Pkts/sec  
tx_errors Indicates the number of errors occurred while data was transmitted by this container. Number Ideally, the value of this measure should be zero.
tx_dropped Indicates the number of packets dropped by this container during transmission. Number Ideally, the value of this measure should be zero.
percpu_usage Indicates the percentage of CPU that is currently utilized by this container. Percent Comparing the value of this measure across the containers will enable you to accurately identify the container on which CPU-intensive applications are executing.
usage_in_kernelmode Indicates the percentage of CPU utilized by this container in kernel mode. Percent A processor in a server has two different modes: user mode and kernel mode. The processor switches between the two modes depending on what type of code is running on the processor. Applications run in user mode, and core operating system components run in kernel mode.

In Kernel mode, the executing code has complete and unrestricted access to the underlying hardware. It can execute any CPU instruction and reference any memory address. Kernel mode is generally reserved for the lowest-level, most trusted functions of the operating system. Crashes in kernel mode are catastrophic; hence they will halt the entire container.

A high value for this measure indicates that the container is taking too much of CPU resources to execute processes in kernel mode.
usage_in_usermode Indicates the percentage of CPU utilized by this container in user mode. Percent In User mode, the executing code has no ability to directly access hardware or reference memory. Code running in user mode must delegate to system APIs to access hardware or memory. Due to the protection afforded by this sort of isolation, crashes in user mode are always recoverable. Most of the code running on the containers will execute in user mode. User mode processes communicate and use the kernel through the Kernel API and system calls.
usage Indicates the amount of memory that is currently utilized by this container. MB  
limit Indicates the maximum amount of memory that is allocated to this container. MB  
permem_used Indicates the percentage of memory utilized by this container. Percent A high value for this measure indicates that the memory resources of the container is depleting alarmingly.
max_usage Indicates the maximum amount of memory utilized by this container. MB  
total_active_anon Indicates the amount of anonymous memory that has been identified as active by the kernel. MB Anonymous memory is the large, zero-filled block of memory that is directly mapped by kernel from the anonymous memory region when large memory with ideally multiples of page sizes is required for the containers. The pages of anonymous memory do not link to any file on the disk, and are the part of program's data area or stack.

All anonymous pages are initially active. In that, some pages will be tagged as inactive when the kernel sweeps over the memory at regular intervals. Whenever the inactive pages are accessed, they are immediately retagged as active. The inactive pages will be swapped when the kernel is almost out of memory, and time comes to swap out to disk.
total_inactive_anon Indicates the amount of anonymous memory that has been identified as inactive by the kernel. MB
total_cache Indicates the amount of memory used by processes of the control group that can be associated precisely with a storage block on this container. MB When you read from and write to files on the disk, the amount of cache memory will increase. Size of the cache memory depends on the number of read operations and write operations performed on this container.
total_active_file Indicates the amount of cache memory that has been identified as active by the kernel. MB Pages in the cache memory can be swapped between active and inactive states similar to the anonymous memory but the exact rules used by the kernel to move memory pages between active and inactive sets are different from the rules used for the anonymous memory. The cache memory pages can be immediately retrieved in a cheaper way when the kernel needs to reclaim memory while the anonymous pages and dirty/modified pages have to be written to disk first before retrieving process.
total_inactive_file Indicates the amount of cache memory that has been identified as inactive by the kernel. MB
total_mapped_file Indicates the amount of memory mapped by the processes in the control group. MB Docker on Linux also makes use of another technology called cgroups or control groups. A key to running applications in isolation is to have them only use the resources you want. This ensures containers are good multi-tenant citizens on a host. Control groups allow Docker to share available hardware resources to containers and, if required, set up limits and constraints. For example, limiting the memory available to a specific container.
total_pgfault Indicates the number of times a process in this container triggered a page fault. Number A page fault occurs when a process accesses a part of its virtual memory space which is nonexistent or protected. The former can happen if the process is buggy and tries to access an invalid address (it will then be sent a SIGSEGV signal, typically killing it with the famous Segmentation fault message). The latter can happen when the process reads from a memory zone which has been swapped out, or which corresponds to a mapped file: in that case, the kernel will load the page from disk, and let the CPU complete the memory access. It can also happen when the process writes to a copy-on-write memory zone: likewise, the kernel will preempt the process, duplicate the memory page, and resume the write operation on the process` own copy of the page.

The major page faults are regular faults occur when the kernel actually has to read the data from disk, duplicate an existing page, or allocate an empty page.
total_pgmajfault Indicates the number of times a process in this container triggered a major page fault. Number
total_pgpgin Indicates the number of pages that were added to the control group of this container. Number  
total_pgpgout Indicates the number of pages that were not billed to the control group of this container. Number  
total_rss Indicates the amount of memory that doesn't correspond to anything on disk such as stacks and heaps, and anonymous memory maps of this container. MB  
total_rss_huge Indicates the amount of maximum memory that doesn't correspond to anything on disk such as stacks and heaps, and anonymous memory maps of this container. MB  
total_swap Indicates the amount of swap memory that is currently utilized by the processes in the control group of this container. MB An unusually high value for the swap memory can indicate a memory bottleneck.
total_unevictable Indicates the amount of memory that cannot be reclaimed by this container. MB Generally, the unevictable memory has been locked with mlock, and is often used by crypto frameworks to make sure that secret keys and other sensitive material never gets swapped out to disk.
total_writeback Indicates the amount of memory that was written back in this container. MB A high value is preferred for this measure. Normally, the data are first written into the cache before writing it into the memory or disk that supports caching. During idle machine cycles, the data are written from the cache into the memory or onto the disk at high speed. In this way, the Write back caches improve performance and writing speed than the normal RAM or disk.
failcnt Indicates the number of memory failures that occurred in this container. Number Ideally, the value of this measure should be low.
read Indicates the rate at which the data was read from this container. Mbps Compare the values of these measures across the containers to identify the slowest container in terms of processing read and write operations (respectively).
write Indicates the rate at which the data were written into this container. Mbps
sync Indicates the rate at which the synchronous I/O operations were performed on this container. Mbps In synchronous I/O file, a thread starts a I/O operation and immediately enters a wait state until the I/O request has completed such that the I/O operations of this container are performed in one-by-one manner so as to prevent unwanted data traffic.
Async Indicates the rate at which the asynchronous I/O operations were performed on this container. Mbps A thread performing asynchronous file I/O sends an I/O request to the kernel by calling an appropriate function. If the request is accepted by the kernel, the calling thread continues processing another job until the kernel signals to the thread that the I/O operation is complete. Therefore, speed of the I/O operations will be increased.
total Indicates the rate at which the total I/O operations were performed on this container. Mbps