• It cordons the node: Cordoning a node means that it will be marked unschedulable, so new pods can no longer be scheduled to the node.

• It evicts or deletes the Pods on that node: After the node is made unschedulable, the drain command will try to evict the pods that are already running on that node. If eviction is supported on the cluster (from Kubernetes version 1.7) the drain command will use the Eviction API that takes disruption budgets into account, if it's not supported it will simply delete the pods on the node.

Use the detailed diagnosis of this measure to know which which nodes wereremoved/drained from which namespace, and why.

Deleting the node object from Kubernetes causes all the Pod objects running on the node to be deleted from the apiserver, and frees up their names.

Use the detailed diagnosis of this measure to know which nodes were deleted from which namespace.

You can use the detailed diagnosis of this measure to know which Pods were evicted and when eviction occurred.

Kubernetes assigns each node a range of IP addresses, a CIDR block, so that each Pod can have a unique IP address.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

nodeSelector is the simplest recommended form of node selection constraint. nodeSelector is a field of PodSpec. It specifies a map of key-value pairs. For the pod to be eligible to run on a node, the node must have each of the indicated key-value pairs as labels (it can have additional labels as well). The most common usage is one key-value pair.

If, when attempting to schedule a Pod, scheduler finds that that Pod's nodeSelector does not have any matching node, the NodeSelectorMismatching event is triggered. In this case, the Pod in question will remain in the Pending state until a matching node is found.

In this case you can try several things:

• Add more nodes to the cluster.

• Terminate unneeded pods to make room for pending pods.

• Check that the pod is not larger than your nodes. For example, if all nodes have a capacity of cpu:1, then a pod with a request of cpu: 1.1 will never be scheduled.

You can use the detailed diagnosis of these measures to identify the nodes on which the events occurred, when it occurred, which Pods were impacted, and why.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, which image was being pulled, and which nodes and Pods were impacted by the event.

The default pull policy is IfNotPresent which causes the Kubelet to skip pulling an image if it already exists. If you would like to always force a pull, you can do one of the following:

• set the imagePullPolicy of the container to Always.

• omit the imagePullPolicy and use :latest as the tag for the image to use.

• omit the imagePullPolicy and the tag for the image to use.

• enable the AlwaysPullImages admission controller.

• Network connectivity issues

• Incorrect image tag

• The image does not exist

• Kubernetes does not have permission to pull the image

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

This pull policy should be used if you want or need to have a full control on which images are used. It is a good choice for containers that are dedicated to a project where only specific images can be used.

If, when attempting to pull an image, the kubelet finds that the image is not present locally, then this policy is violated.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

• There is an invalid container image tag;

• Kubernetes does not have permissions to access the image;

• The image does not exist;

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

A container is healthy, by the most general definition, if its main process is running. If the container’s main process is terminated unexpectedly, then the container is considered unhealthy.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Typically, if a ReplicationController cannot create Pods, you may have to debug the Pods. The first step in debugging a Pod is taking a look at it. Check the current state of the Pod and recent events: Look at the state of the containers in the Pod. Are they all running? Have there been recent restarts? Then, continue debugging depending on the state of the Pods. Are Pods stuck in a Pending state? Then, check for resource inadequacies. Are the Pods in Waiting state? Then, check if there are any issues in image pulling. Are the Pods crashing? Then, study the container logs for troubleshooting the same.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

The FailedMount event typically follows the FailedAttachVolume event because the mount operation happens after the attach operation and because the attach has already failed, it means that the mount operation is not possible.

The FailedMount and FailedAttachVolume events can help you debug issues with storage. These events will prevent Pods from starting correctly. You may think that your Pods are just slow to start, but if there are permissions or networking issues when creating network volumes, you will need to rectify them to get your Pods working again.

Use the detailed diagnosis of these measures to know when these events occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of these measures to know when these events occurred, why, on which nodes, and which Pods were impacted.

Allocatable on a Kubernetes node is defined as the amount of compute resources that are available for Pods.

To properly enforce node allocatable constraints on the node, you must enable the new cgroup hierarchy via the -cgroups -per -qos flag. This flag is enabled by default. When enabled, the kubelet will parent all end-user pods under a cgroup hierarchy managed by the kubelet.

Following is the recommended cgroup configuration for Kubernetes nodes. All OS system daemons are expected to be placed under a top level SystemReserved cgroup. Kubelet and Container Runtime are expected to be placed under KubeReserved cgroup.

kube-reserved is meant to capture resource reservation for kubernetes system daemons like the kubelet, container runtime, node problem detector, etc.

system-reserved is meant to capture resource reservation for OS system daemons like sshd, udev, etc

To optionally enforce system-reservedon system daemons, specify the parent control group for OS system daemons as the value for -system-reserved-cgroup kubelet flag. If this specification includes an invalid cgroup, then Kubelet will fail to enforce system-reserved, and will trigger the FailedNodeAllocatableEnforcement event.

A Pod is composed of a group of application containers in an isolated environment with resource constraints. In CRI, this environment is called PodSandbox.

Before starting a Pod, kubelet calls RuntimeService.RunPodSandbox to create the environment. This includes setting up networking for a pod (e.g., allocating an IP). If kubelet is unable to create the environment for running a Pod, the FailedCreatePodSandBox event is triggered.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

The kubelet can optionally perform and react to three kinds of probes on running Containers:

• livenessProbe: Indicates whether the Container is running. If the liveness probe fails, the kubelet kills the Container, and the Container is subjected to its restart policy.

• readinessProbe: Indicates whether the Container is ready to service requests. If the readiness probe fails, the endpoints controller removes the Pod’s IP address from the endpoints of all Services that match the Pod.

• startupProbe: Indicates whether the application within the Container is started. All other probes are disabled if a startup probe is provided, until it succeeds. If the startup probe fails, the kubelet kills the Container, and the Container is subjected to its restart policy.

A ContainerProbeWarning event is triggered when any of these probes fail.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

There are two hooks that are exposed to Containers:

• PostStart: This hook executes immediately after a container is created. However, there is no guarantee that the hook will execute before the container ENTRYPOINT. No parameters are passed to the handler.

• PreStop: This hook is called immediately before a container is terminated due to an API request or management event such as liveness probe failure, preemption, resource contention and others. A call to the preStop hook fails if the container is already in terminated or completed state. It is blocking, meaning it is synchronous, so it must complete before the call to delete the container can be sent. No parameters are passed to the handler.

If a hook handler fails, it broadcasts an event. While failure of the PostStart hook handler triggers the FailedPostStartHook event, the failure of the PreStop hook handler triggers the FailedPreStopHook event.

Use the detailed diagnosis of these measures to know when these events occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, on which nodes, and which Pods were impacted.

At configured intervals, the controller manager queries the resource utilization against the metrics specified in each HorizontalPodAutoscaler definition. The controller manager obtains the metrics from either the resource metrics API (for per-pod resource metrics), or the custom metrics API (for all other metrics). Typically, metrics are fetched from a series of aggregated APIs - metrics.k8s.io, custom.metrics.k8s.io, and external.metrics.k8s.io. The controller then calculates the actual utilization value of the resource, considers the target/desired utilization value that is set, and computes the ratio between the desired and actual metric value. The autoscaler then scales the desired number of replicas up or down based on this ratio.

One of the common reasons for the failure of auto-scaling is the inability of the controller to fetch the resource metrics from the API. Without the metrics, scales cannot be computed, and consequently, the count of replicas cannot be scaled up/down. The FailedGetResourceMetric is broadcast everytime the controller fails to get resource metrics.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, on which nodes, and which Pods were impacted.

If a PV belonging to a StorageClass needs to be dynamically provisioned for a PVC, then a key field that your StorageClass definition should contain is the Provisioner. A Provisioner determines what volume plugin is to be used for provisioning PVs dynamically. Likewise, the definition should also include mountOptions. In this case, if the Provisioner - i.e., volume plugin - in use does not support mount options, then volume provisioning will fail. Where multiple mountOptions are provided, provisioning failures will also occur if even one of the mount options is found to be invalid.

Provisioning failures may also occur if:

• The volume plugin does not match any of the supported plugins;

• The application is requesting more storage space than is available in the underlying volumes that have been provisioned.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Common causes for this are:

• The application inside the container keeps crashing

• Some type of parameters of the pod or container have been configured incorrectly

• An error has been made when deploying Kubernetes

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

This event is triggered every time a Job completes.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes, and which Pods were impacted.

Use the detailed diagnosis of this measure to know when this event occurred, why, on which nodes.

If a Deployment is successfully canceled, then the DeploymentCancelled event is triggered. When canceled, the deployment configuration will be automatically rolled back by scaling up the previous running replication controller.

On the other hand, if cancellation of a Deployment fails, then the DeploymentCancellationFailed event is triggered.

You can use the detailed diagnosis of these measures to know when each of these events occurred, why, and on which nodes.

You can use the detailed diagnosis of this measure to know when this event occurred, on which nodes, and which Pods were created in the process.

An Ingress can be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An Ingress controller is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.

When an Ingress is created, typically, an IP address is allocated by the Ingress Controller to satisfy the Ingress. Ingress controllers and load balancers may take a minute or two to allocate an IP address. Until that time, you often see the address listed as .

Sometimes, the Ingress Controller may not find any IP address to allocate to the service for which the Ingress was created. In this case, the Ingress will fail with the event IngressIPRangeFull.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

A Persistent Volume that cannot be detached poses a problem if you try to create another Pod using the same PVC.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

If a PVC is created, but no PersistentVolumes are available for the PVC to be bound to, then the FailedBinding event gets fired. In such cases, Pods that use unbound PVCs will stay in the Pending state, until the problem is resolved.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Typically, a user creates a PersistentVolumeClaim (PVC) with a specific amount of storage requested and with certain access modes. A control loop in the master watches for new PVCs, checks if any static PV (a PV manually created by the administrator) exactly matches the new PVC, and binds them together. Claims will remain unbound indefinitely if a matching volume does not exist. Claims will be bound as matching volumes become available. For example, a cluster provisioned with many 50Gi PVs would not match a PVC requesting 100Gi. The PVC can be bound when a 100Gi PV is added to the cluster. Until a 100Gi PV becomes available, the cluster will not bind the PVC with any of the existing PVs; instead, it will fail binding with the event VolumeMismatch.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Typically, when a Pod is deleted and the PV has to be freed up, the recycler Pod comes in and tries to make the PV available. But, sometimes, due to certain errors, the recycler POD may switch to the 'failed' state. For instance, if the recycler Pod fails to remove the .snapshot folder, the Pod will fail to be created. As a result, the PV too fails to become available - i.e., volume recycling fails. n such a situation, the VolumeFailedRecycle event is fired.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

This can happen if the path specification in your PV does not match with the actual path of the volume being deleted.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Many load balancer issues around creating, updating, and deleting the load balancer can also be traced to a permissions issue with your cloud provider. Ensure that your Kubernetes nodes have the ability to create and modify load balancers in your cloud provider to avoid these issues. If your cloud provider provides Identity & Access Management (IAM) double-check the permissions that your nodes and pods have.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.

Use the detailed diagnosis of this measure to know when this event occurred, why, and which nodes and Pods were impacted by the event.