eG Monitoring
 

Measures reported by ClusterResPoolTest

The ClusterResPoolTest auto-discovers the resource pools and/or vApps under ESX clusters, and reports critical statistics pertaining to each pool. To distinguish between vApps and resource pools, the eG monitoring console tags every discovered vApp as [vApp]. The measures made by this test are as follows:

Measurement Description Measurement Unit Interpretation
Rp_status Indicates the current status of this resource pool.   If the status cannot be determined, then Unknown will be displayed here. If the pool is functioning normally, then this measure will return the value Normal. If the root resource pool does not have the capacity to meet the reservation of its children, then the status will be displayed as Overcommitted. One or more pools in the tree may have children whose reservations are greater than the pool is configured to support, then the status will be Inconsistent.

The numeric values that correspond to these measures are as follows:

State Value
Unknown 0
Normal 1
Overcommitted 2
Inconsistent 3

Child_resource_pools Indicates the total number of child resource pools under this resource pool. Number  
All_virtual_machines Indicates the total number of VMs in this resource pool. Number This number includes both the directly and indirectly assigned VMs.
All_Vms_powered_on Indicates the total number of powered-on VMs in this resource pool. Number This number includes both the directly and indirectly assigned VMs.
Vms_powered_on Indicates the number of directly assigned virtual machines under this resource pool that is currently powered on. Number If the value of this measure is equal to the value of the Total_virtual_machines measure, it indicates that all VMs in the pool are in a powered-on state. If the values do not match, it indicates that one/more VMs are currently in a powered-off state.
Total_virtual_machines Indicates the number of virtual machines that were directly assigned to this resource pool. Number  
Cpu_usage Indicates the CPU usage of the resource pool in Mhz. Mhz The percent CPU usage measure serves as an effective indicator of how resource-intensive a particular resource pool is on a specific cluster. However, for performing capacity planning or what-if analysis, the CPU usage of a pool measured in absolute terms would be more useful. For instance, the physical CPU usage of a pool could be 30% on a particular cluster - this means that the resource pool is consuming 30% of the total physical CPU capacity of that cluster. If you are planning to migrate the resource pool to another cluster, then it would be unwise to assume that the pool will only consume 30% of CPU on the other cluster as well, as the percentage will vary depending upon the physical CPU resources that are available to the other cluster. The absolute measure however will remain unchanged across clusters. Therefore, to decide which cluster a VM/resource pool is to be moved to, and to analyze the impact of this movement on the CPU resources of the new ESX host, you would require an absolute measure of CPU usage.
Physical_cpu_used Indicates the percentage of physical CPU resources used by the VMs in this pool. Percent A high value of this measure could indicate excessive CPU usage by the child resource pools/virtual machines configured under this resource pool. By comparing the value of this measure across resource pools, you can quickly identify that resource pool which is draining the CPU resources of the ESX host.

The detailed diagnosis of this measure, if enabled, reveals the names and CPU usage of each VM in this pool, thus revealing which VM is CPU-intensive.

Cpu_reserved_by_vms Indicates the total amount of CPU reserved by VMs in this pool. Mhz Use the detailed diagnosis of this measure to know how much CPU has been reserved by each VM.
Cpu_reservation Indicates the amount of CPU resources that is guaranteed available to the virtual machine or resource pool. Mhz Typically, the Reservation setting for a resource pool specifies the minimum acceptable amount of CPU or memory - not the amount you would like to have available.

Follow the guidelines mentioned below to define the Reservation setting:

  • Do not set the Reservation too high. A very high reservation can limit the number of virtual machines you can power on in a resource pool.
  • Always leave some headroom while reserving resource. Do not commit all resources. As you move closer to fully reserving all capacity in the system, it becomes increasingly difficult to make changes to reservations and to the resource pool hierarchy without violating admission control.
  • If you expect frequent changes to the total available resources, use Shares, not Reservation, to allocate resources fairly across resource pools.

Reserved resources are not wasted if they are not used. If the utilization is less than the reservation, the resources can be utilized by other running virtual machines.

Cpu_reservation_used Indicates the total amount of CPU resources that have been used to satisfy the reservation requirements of all descendants of this resource pool (includes both resource pools and virtual machines). Mhz CPU resources are typically reserved for powered-on virtual machines in a resource pool only. Virtual machines that are not currently running do not use any CPU resources. Reserved resources are not wasted if they are not used. If the utilization is less than the reservation, the resources can be utilized by other running virtual machines.
Cpu_unreserved Indicates the total amount of resources available to satisfy a reservation for a child resource pool. Mhz In the undercommitted state, this value is limited by the capacity at the root node. In the overcommitted case, this could be higher since we do not perform the dynamic capacity checks. Consider a resource pool with reservation=2GHz that istotally idle. It has 2GHz reserved, but it is not actually using any of its reservation. In such a case:

  • Other resource pools cannot reserve these 2GHz.
  • Other resource can use these 2GHz, that is, idle CPU reservations are not wasted.
Cpu_allocation_used Indicates the percentage of allocated CPU resources that are being used by this resource pool. Percent A high value of this measure indicates that one/more VMs in the pool are consuming the allocated CPU resources excessively.
Cpu_limit Indicates the limit beyond which this resource pool should not use CPU resources, even if there are resources available. Mhz If the Unlimited flag is enabled for a resource pool, then this measure will report the value Unlimited. This setting helps avoid wastage of idle resources.

This is typically used to ensure a consistent performance of virtual machines / resource pools independent of available resources. If set to -1, then there is no fixed limit on resource usage (only bounded by available resources and shares).

Memory_used Indicates the amount of memory used by running VMs in this resource pool. MB  
Memory_reservation Indicates the amount of memory resources that is guaranteed available to the virtual machine or resource pool. MB Typically, the Reservation setting for a resource pool specifies the minimum acceptable amount of CPU or memory - not the amount you would like to have available. Follow the guidelines mentioned below to define the Reservation setting:

  • Do not set the Reservation too high. A very high reservation can limit the number of virtual machines you can power on in a resource pool.
  • Always leave some headroom while reserving resource. Do not commit all resources. As you move closer to fully reserving all capacity in the system, it becomes increasingly difficult to make changes to reservations and to the resource pool hierarchy without violating admission control.
  • If you expect frequent changes to the total available resources, use Shares, not Reservation, to allocate resources fairly across resource pools.

Reserved resources are not wasted if they are not used. If the utilization is less than the reservation, the resources can be utilized by other running virtual machines.

Mem_reserved_by_vms Indicates the total amount of memory reserved by VMs in this pool. MB Use the detailed diagnosis of this measure to know how much memory has been reserved by each VM in the pool.
Mem_consumed Indicates the total amount of memory consumed by the VMs in this pool. MB Use the detailed diagnosis of this measure to know how much memory has been consumed by each VM in the pool. This way, you can quickly identify memory-intensive VMs.
Memory_reservation_used Indicates the total amount of memory resources that have been used to satisfy the reservation requirements of all descendants of this resource pool (includes both resource pools and virtual machines). MB Memory resources are typically reserved for powered-on virtual machines in a resource pool only. Virtual machines that are not currently running do not use any memory resources.

Reserved resources are not wasted if they are not used. If the utilization is less than the reservation, the resources can be utilized by other running virtual machines.

Memory_unreserved Indicates the total amount of resources available to satisfy a reservation for a child resource pool.l MB In the undercommitted state, this value is limited by the capacity at the root node. In the overcommitted case, this could be higher since we do not perform the dynamic capacity checks. If the resource pool does not use the reserved memory, then remember the following:

  • Other resource pools cannot reserve these memory resources
  • Other resource pools can use these memory resources, that is, idle memory reservations are not wasted
Memory_limit Indicates the limit beyond which this resource pool should not use memory resources, even if there are resources available. MB If the Unlimited flag is enabled for a resource pool, then this measure will report the value Unlimited. This setting helps avoid wastage of idle resources.

This is typically used to ensure a consistent performance of virtual machines / resource pools independent of available resources. If set to -1, then there is no fixed limit on resource usage (only bounded by available resources and shares).

Active_memory Indicates the amount of memory resources actively used from this pool. MB High memory consumption by a resource pool, besides eroding the physical memory resources of the host, can also impact the memory allocations to other resource pools/virtual machines; the lack of adequate memory to a VM can significantly strain its performance.
Active_memory_used Indicates the percentage of total configured or available memory in this pool. Percent
Zero_memory Indicates the amount of memory that is zeroed out. MB  
Swapped_memory Indicates the amount of memory that is swapped. MB ESX Server hosts use swapping to forcibly reclaim memory from a virtual machine when no vmmemctl driver is available because the vmmemctl driver:
  • Was never installed
  • Has been explicitly disabled
  • Is not running (for example, while the guest operating system is booting
  • Is temporarily unable to reclaim memory quickly enough to satisfy current system demands

Standard demand-paging techniques swap pages back in when the virtual machine needs them. Swap space must be reserved on disk for any unreserved virtual machine memory. This swap reservation is required to ensure the system is able to preserve virtual machine memory under any circumstances. In practice, only a small fraction of the swap space may actually be used. Typically, swap space usage for each VM should be low. Since access from RAM is much faster than access from physical disk, excessive usage of swap memory will slow down the performance of a VM. Watch for VMs that are seeing higher swap usage and more swap reads and writes.

Memory_overhead Indicates the memory overhead incurred by this resource pool. MB Virtual machines can incur two kinds of memory overhead:
  • The additional time to access memory within a virtual machine.
  • The extra space needed by the ESX Server host for its own code and data structures, beyond the memory allocated to each virtual machine.

The ESX server's memory virtualization ensures that the time overhead to memory accesses is minimal. The memory space overhead, on the other hand, is composed of two other components:

  • A fixed system-wide overhead for the service console (in the case of ESX 3/3.5) and the VMkernel.
  • Additional overhead for each virtual machine

In addition, the space reserved for the virtual machine frame buffer and various virtualization data structures, also add to the memory overhead. Addition of more virtual CPUs, allocation of more memory to the guests, and choosing to configure 32-bit guest operating systems instead of 64-bit ones, can go a long way in reducing this overhead. The ESX server also provides optimizations such as memory sharing to save up memory.

Balloon_memory Indicates the amount of memory used by memory control. MB The vmmectl driver that is installed on a virtual machine, emulates an increase or decrease inmemory pressure on the guest operating system; this way, it forces the guest OS to placememory pages into its local swap file. This driver differs from theVMware swap file method as it forces the operating system to determinewhat memory it wishes to page. Once the memory is pagedlocally on the guest operating system, the free physical pages of memory may be reallocated to other guests. As the ESX hosts sees thatmemory demand has been reduced, it will instruct vmmemctl to &/ldquo;deflate” the balloon and reduce pressure on the guest OS to pagememory. The maximum amount of memory that can be reclaimed from a guest may be configured by modifying the “sched.mem.maxmemctl” advanced option.

If the memory reclaimed from a guest (i.e., the value of this measure) is very low, it indicates excessive memory usage by the guest. Under such circumstances, you might want to consider allocating more memory to the guest.

Memory_granted Indicates the amount of memory that is granted to this resource pool. MB  
Shared_memory Indicates the amount of memory in the resource pool that is shared. MB  
Memory_avail_perc Indicates the percentage of free memory in this resource pool. Percent A consistent decrease in the value of this measure could indicate a potential memory contention in the pool. Compare the value of this measure across resource pools to isolate the resource pool with the least amount of free memory.