Kubernetes Troubleshooting

Kubernetes Pending Pods Troubleshooting

Learn how to troubleshoot Kubernetes Pods stuck in Pending state. This production-focused guide covers Kubernetes scheduling, insufficient CPU or memory, taints and tolerations, node selectors, affinity, PVCs, ResourceQuotas and scheduler events.

Quick Diagnosis

When a Pod is Pending, do not immediately restart or delete it. First determine why the Kubernetes scheduler cannot place it.

01

Check the Pod status

Confirm that the Pod is actually Pending and identify the namespace where it is running.

kubectl get pod <pod-name> -n <namespace>
02

Describe the Pod

The Events section is usually the most important source of information when a Pod cannot be scheduled.

kubectl describe pod <pod-name> -n <namespace>
03

Check recent events

Look for scheduler messages such as insufficient CPU, insufficient memory, untolerated taint or node selector mismatch.

kubectl get events -n <namespace> --sort-by=.lastTimestamp
04

Check available nodes

Inspect node readiness, capacity, allocatable resources and scheduling restrictions.

kubectl get nodes -o wide
05

Check resource usage

Compare the Pod's CPU and memory requests with what the cluster can currently allocate.

kubectl top nodes

What Does Pending Mean in Kubernetes?

A Pod in Pending state has been accepted by the Kubernetes API server but has not been successfully scheduled onto a node.

The most important distinction is that Pending is not the same as Running with an application failure. A Pending Pod usually has a scheduling or prerequisite problem that must be investigated before the container can start.

First command to remember:

kubectl describe pod <pod-name> -n <namespace>

Look at the bottom of the output for the Events section. In many scheduling incidents, the scheduler message directly identifies the constraint preventing placement.

Common Causes of Pending Pods

Insufficient CPU

The scheduler cannot find a node with enough allocatable CPU to satisfy the Pod's CPU request.

kubectl describe pod <pod-name>

Insufficient Memory

The requested memory cannot be satisfied by any eligible node.

kubectl describe pod <pod-name>

Node Taints

The Pod does not tolerate a taint on otherwise eligible nodes.

kubectl describe node <node-name>

Node Selector

The Pod requires a node label that no available node currently has.

kubectl get nodes --show-labels

Node Affinity

Required node affinity rules may prevent the scheduler from selecting any node.

kubectl get pod <pod-name> -o yaml

PVC Not Bound

A Pod using a PersistentVolumeClaim may remain Pending when the PVC itself cannot be bound.

kubectl get pvc -n <namespace>

ResourceQuota

Namespace resource quotas can prevent new workloads from being admitted or scheduled.

kubectl get resourcequota -n <namespace>

Node Not Ready

Pods may not be scheduled onto nodes that are unavailable or marked NotReady.

kubectl get nodes

Kubernetes Scheduler Events to Watch

Scheduler events are often the fastest way to identify why a Pod cannot be placed.

0/3 nodes are available: 3 Insufficient cpu.

The Pod's requested CPU cannot currently be satisfied by the available nodes.

0/3 nodes are available: 3 Insufficient memory.

The Pod's requested memory is greater than the schedulable memory available on eligible nodes.

0/3 nodes are available: 3 node(s) had untolerated taint.

The available nodes have taints that the Pod does not tolerate.

0/3 nodes are available: 3 node(s) didn't match Pod's node affinity/selector.

The Pod's placement rules do not match the labels of available nodes.

kubectl get events -A --sort-by=.lastTimestamp

Real-World Pending Pod Troubleshooting

Scenario 1: Pod Pending because of insufficient CPU

Start with the Pod events to confirm the scheduler reported insufficient CPU. Then compare the Pod's CPU request with node allocatable capacity and existing workload requests.

1.kubectl get pod <pod-name>
2.kubectl describe pod <pod-name>
3.kubectl get nodes
4.kubectl describe nodes
5.kubectl top nodes

Scenario 2: Pod Pending because of a taint

If the scheduler reports an untolerated taint, inspect the node taints and compare them with the Pod's tolerations. Do not remove production taints simply to force scheduling.

1.kubectl describe pod <pod-name>
2.kubectl get nodes
3.kubectl describe node <node-name>

Scenario 3: Pod Pending because of PVC

A workload may appear Pending because its required PersistentVolumeClaim is not Bound. Investigate the PVC events, StorageClass and available PersistentVolumes before changing the workload.

1.kubectl get pod <pod-name>
2.kubectl get pvc
3.kubectl describe pvc <pvc-name>
4.kubectl get pv
5.kubectl get storageclass

Scenario 4: Pod Pending because of node selector

Compare the Pod's nodeSelector or affinity requirements with the labels actually present on cluster nodes.

1.kubectl get pod <pod-name> -o yaml
2.kubectl get nodes --show-labels
3.kubectl describe pod <pod-name>

Troubleshooting Insufficient CPU or Memory

Kubernetes scheduling is based on resource requests and the allocatable resources available on eligible nodes. A node can show apparently low real-time CPU usage while still being unable to accept another Pod because existing workload requests consume its allocatable capacity.

Check Pod requests

kubectl get pod <pod-name> -o yaml

Check node allocatable capacity

kubectl describe nodes

Troubleshooting Taints and Tolerations

Taints allow nodes to repel Pods that do not explicitly tolerate the configured taint. If a Pending Pod reports an untolerated taint, compare the node's taints with the Pod's tolerations.

Inspect node taints

kubectl describe node <node-name>

Inspect Pod tolerations

kubectl get pod <pod-name> -o yaml

Production caution: Do not remove a node taint simply to make a workload schedule. Taints are often intentional controls for dedicated, system or restricted nodes.

Troubleshooting PVC and Storage Issues

Storage dependencies are another common reason for workloads to remain Pending. Start by checking whether the PersistentVolumeClaim is Bound and inspect its events.

kubectl get pvc -n <namespace>kubectl describe pvc <pvc-name> -n <namespace>kubectl get pvkubectl get storageclass

Production-Safe Troubleshooting

Pending Pods are usually best investigated before any disruptive action is taken.

Start with read-only commands: get, describe, events and top.

Confirm the active cluster context before troubleshooting production.

Check Pod requests against node allocatable resources rather than only looking at current CPU usage.

Inspect scheduler events before changing node labels, taints or workload manifests.

Do not remove taints or relax affinity rules without understanding why they were configured.

Do not delete and recreate workloads simply because they are Pending.

For PVC-related Pending Pods, investigate the storage layer before changing the application.

After making a change, verify scheduling and rollout status.

How to Prevent Pending Pods

Set realistic CPU and memory requests and limits.

Monitor node capacity and workload resource requests.

Use cluster autoscaling where appropriate for workloads with variable capacity requirements.

Document intentional node taints and corresponding tolerations.

Keep node labels consistent with nodeSelector and affinity rules.

Monitor PersistentVolume and PersistentVolumeClaim health.

Review ResourceQuota and LimitRange configuration.

Use Kubernetes events and scheduler-related alerts as part of operational monitoring.

Related Kubernetes Guides

Kubernetes Pending Pods FAQ

What does Pending mean in Kubernetes?+

Pending means the Pod has been accepted by the Kubernetes API but has not yet been successfully scheduled onto a node. The scheduler may be unable to find a node that satisfies the Pod's resource, placement, storage or other scheduling requirements.

What is the first command to run when a Pod is Pending?+

Start with kubectl get pod <pod-name> -n <namespace> to confirm the state, then run kubectl describe pod <pod-name> -n <namespace>. The Events section often explains why the scheduler could not place the Pod.

Why does a Kubernetes Pod stay Pending because of insufficient CPU?+

Kubernetes schedules based primarily on resource requests. Even when a node appears to have unused CPU according to a monitoring tool, its allocatable CPU may not be sufficient to satisfy the new Pod's requested CPU.

Can a taint cause a Pod to remain Pending?+

Yes. A node taint can prevent a Pod from being scheduled unless the Pod has a matching toleration. The scheduler event commonly reports an untolerated taint when this is the reason.

Can a PVC cause a Pod to remain Pending?+

Yes. If the Pod requires a PersistentVolumeClaim and the PVC cannot be bound to suitable storage, the workload may remain Pending. Check the PVC status and events with kubectl get pvc and kubectl describe pvc.

How do I check which nodes can run a Pending Pod?+

Inspect the Pod's resource requests, nodeSelector, affinity and tolerations, then compare those requirements with node labels, taints, capacity and allocatable resources.

This Kubernetes Pending Pods troubleshooting guide is intended for Kubernetes administrators, DevOps engineers, SREs and production support teams. Always verify commands and configuration changes against your Kubernetes version and environment before applying them to production.