DevOps • SRE • Production Support

DevOps Troubleshooting Guide

Find the right commands, understand common failure symptoms and follow practical investigation workflows for DevOps, application support, SRE and production environments.

KubernetesLinuxDockerJenkinsTerraformNetworkingMonitoringProduction Support

What are you troubleshooting?

Search for a symptom, technology, command or production issue.

78 troubleshooting commands

Linux CPU High

top

Find processes consuming CPU in real time.

ps aux --sort=-%cpu | head

Show the highest CPU-consuming processes.

uptime

Check system load averages.

pidstat 1

Monitor CPU usage by process over time.

Linux Memory High

free -h

Check total, used and available memory.

ps aux --sort=-%mem | head

Find processes consuming the most memory.

vmstat 1

Inspect memory, processes, paging and CPU activity.

dmesg | grep -i oom

Look for out-of-memory killer events.

Disk 100 Percent

df -h

Find filesystems that are running out of space.

df -i

Check inode consumption.

du -xh / | sort -h | tail -20

Find large directories and files.

journalctl --disk-usage

Check how much disk space systemd journals use.

Linux Service Not Starting

systemctl status nginx

Check the current service state and recent errors.

journalctl -u nginx -n 100

Read recent logs for the service.

systemctl cat nginx

Inspect the service unit configuration.

systemctl daemon-reload

Reload systemd after changing unit files.

Port Already In Use

ss -lntp

List listening TCP ports and owning processes.

ss -lntup

List listening TCP and UDP ports.

lsof -i :8080

Find the process using port 8080.

fuser -v 8080/tcp

Identify processes using a TCP port.

SSH Troubleshooting

ssh -vvv user@server

Enable verbose SSH debugging.

nc -vz server 22

Check whether TCP port 22 is reachable.

systemctl status ssh

Check the SSH server service.

ss -lntp | grep :22

Verify that SSH is listening.

DNS Troubleshooting

dig example.com

Inspect DNS resolution details.

dig +short example.com

Return a concise DNS answer.

nslookup example.com

Perform a basic DNS lookup.

cat /etc/resolv.conf

Check configured DNS resolvers on Linux.

Network Connectivity

ping -c 4 8.8.8.8

Check basic IP connectivity.

curl -I https://example.com

Check HTTP connectivity and response headers.

traceroute example.com

Inspect the network path toward a destination.

ip route

Inspect the local routing table.

Docker Container Crashing

docker ps -a

Check running and stopped containers.

docker logs <container>

Read container application logs.

docker inspect <container>

Inspect container configuration and state.

docker stats

Check container CPU and memory usage.

Docker Image Problems

docker images

List locally available images.

docker history <image>

Inspect image layers.

docker image inspect <image>

Inspect image metadata and configuration.

docker system df

Check Docker disk consumption.

Kubernetes CrashLoopBackOff

kubectl get pods

Identify pods that are repeatedly restarting.

kubectl describe pod <pod>

Inspect events, container state and configuration.

kubectl logs <pod> --previous

Read logs from the previous crashed container.

kubectl get events --sort-by=.lastTimestamp

Review recent Kubernetes events.

Kubernetes ImagePullBackOff

kubectl describe pod <pod>

Check image pull errors and events.

kubectl get secret

Check whether image pull secrets exist.

kubectl get pod <pod> -o yaml

Inspect the pod image and imagePullSecrets configuration.

kubectl get events --sort-by=.lastTimestamp

Find registry authentication or image errors.

Kubernetes Pod Pending

kubectl get pods -o wide

Check where pending pods are scheduled or waiting.

kubectl describe pod <pod>

Inspect scheduler events and resource constraints.

kubectl get nodes

Check available cluster nodes.

kubectl describe node <node>

Inspect node capacity, conditions and taints.

Kubernetes Service Not Reachable

kubectl get svc

Check service configuration and ports.

kubectl get endpoints <service>

Verify that the service has backend endpoints.

kubectl get pods --show-labels

Compare pod labels with the service selector.

kubectl describe svc <service>

Inspect service selectors, ports and endpoints.

Kubernetes Node NotReady

kubectl get nodes

Identify nodes reporting NotReady.

kubectl describe node <node>

Inspect node conditions and recent events.

kubectl get pods -A -o wide

Check workloads running on affected nodes.

kubectl get events -A --sort-by=.lastTimestamp

Review cluster-wide events.

Jenkins Build Failure

docker logs <jenkins-container>

Inspect Jenkins container logs when Jenkins runs in Docker.

java -version

Verify the Java runtime used by Jenkins.

df -h

Check whether disk exhaustion is affecting builds.

free -h

Check available memory on the Jenkins host.

Terraform State Problems

terraform state list

List resources tracked in Terraform state.

terraform state show <resource>

Inspect a resource stored in state.

terraform plan

Compare configuration against current state and infrastructure.

terraform refresh

Refresh state from infrastructure when supported by the workflow.

Prometheus Target Down

curl http://localhost:9090/-/healthy

Check Prometheus health.

curl http://localhost:9090/api/v1/targets

Inspect target states through the Prometheus API.

curl http://target:9100/metrics

Check whether an exporter endpoint responds.

promtool check config prometheus.yml

Validate Prometheus configuration.

Production Incident Workflow

date

Record the current time when beginning incident investigation.

uptime

Check host uptime and system load.

df -h

Check for disk exhaustion.

free -h

Check memory pressure.

ss -lntup

Inspect listening network services.

journalctl -p err -n 100

Review recent high-priority system errors.

Production Incident

Where should you start?

When an application or server is failing, start broad and narrow the investigation instead of immediately changing configuration.

Confirm the symptom

What exactly is failing?

Check recent changes

Deployment, config, code or infrastructure?

Check health signals

CPU, memory, disk, logs and network.

Isolate the cause

Use targeted commands before applying a fix.

Explore

Troubleshoot by Technology

Start with the technology closest to the problem and work through the relevant checks.

Kubernetes

Pods, scheduling, services, nodes, images and cluster failures.

5 troubleshooting areas

Linux

CPU, memory, disk, services, ports, SSH and system problems.

6 troubleshooting areas

Docker

Containers, images, logs, resources and runtime problems.

2 troubleshooting areas

Jenkins

Build failures, agents, Java, disk and CI/CD problems.

1 troubleshooting area

Terraform

State, planning, infrastructure drift and deployment issues.

1 troubleshooting area

Networking

DNS, ports, connectivity, routes and SSH investigation.

4 troubleshooting areas

Monitoring

Prometheus health, targets, exporters and monitoring checks.

1 troubleshooting area

Production Support

A structured first-response workflow for production incidents.

Incident workflow

Start with a Common Production Issue

These are common symptoms that DevOps, SRE and application support engineers investigate during incidents.

Kubernetes

Kubernetes CrashLoopBackOff

Check logs, previous container logs, events, probes, configuration and resources.

Find troubleshooting commands →

Kubernetes

Kubernetes Pod Pending

Investigate scheduling, resources, taints, node availability and events.

Find troubleshooting commands →

Linux

Linux Disk Full

Find full filesystems, large directories, inode exhaustion and journal usage.

Find troubleshooting commands →

Linux

Linux High CPU

Identify CPU-heavy processes and investigate application behaviour.

Find troubleshooting commands →

Docker

Docker Container Not Starting

Inspect container state, logs, images, configuration, ports and volumes.

Find troubleshooting commands →

Jenkins

Jenkins Build Failure

Check console output, Java, workspace, disk, memory and build environment.

Find troubleshooting commands →

Terraform

Terraform State Problems

Inspect state, identify drift and investigate state-related failures safely.

Find troubleshooting commands →

Networking

Application Port Not Responding

Check listeners, processes, firewall rules, routes and network connectivity.

Find troubleshooting commands →

Command Reference

Practical Troubleshooting Commands

Searchable commands grouped by the production problems they help investigate.

78 troubleshooting commands

Linux CPU High

top

Find processes consuming CPU in real time.

ps aux --sort=-%cpu | head

Show the highest CPU-consuming processes.

uptime

Check system load averages.

pidstat 1

Monitor CPU usage by process over time.

Linux Memory High

free -h

Check total, used and available memory.

ps aux --sort=-%mem | head

Find processes consuming the most memory.

vmstat 1

Inspect memory, processes, paging and CPU activity.

dmesg | grep -i oom

Look for out-of-memory killer events.

Disk 100 Percent

df -h

Find filesystems that are running out of space.

df -i

Check inode consumption.

du -xh / | sort -h | tail -20

Find large directories and files.

journalctl --disk-usage

Check how much disk space systemd journals use.

Linux Service Not Starting

systemctl status nginx

Check the current service state and recent errors.

journalctl -u nginx -n 100

Read recent logs for the service.

systemctl cat nginx

Inspect the service unit configuration.

systemctl daemon-reload

Reload systemd after changing unit files.

Port Already In Use

ss -lntp

List listening TCP ports and owning processes.

ss -lntup

List listening TCP and UDP ports.

lsof -i :8080

Find the process using port 8080.

fuser -v 8080/tcp

Identify processes using a TCP port.

SSH Troubleshooting

ssh -vvv user@server

Enable verbose SSH debugging.

nc -vz server 22

Check whether TCP port 22 is reachable.

systemctl status ssh

Check the SSH server service.

ss -lntp | grep :22

Verify that SSH is listening.

DNS Troubleshooting

dig example.com

Inspect DNS resolution details.

dig +short example.com

Return a concise DNS answer.

nslookup example.com

Perform a basic DNS lookup.

cat /etc/resolv.conf

Check configured DNS resolvers on Linux.

Network Connectivity

ping -c 4 8.8.8.8

Check basic IP connectivity.

curl -I https://example.com

Check HTTP connectivity and response headers.

traceroute example.com

Inspect the network path toward a destination.

ip route

Inspect the local routing table.

Docker Container Crashing

docker ps -a

Check running and stopped containers.

docker logs <container>

Read container application logs.

docker inspect <container>

Inspect container configuration and state.

docker stats

Check container CPU and memory usage.

Docker Image Problems

docker images

List locally available images.

docker history <image>

Inspect image layers.

docker image inspect <image>

Inspect image metadata and configuration.

docker system df

Check Docker disk consumption.

Kubernetes CrashLoopBackOff

kubectl get pods

Identify pods that are repeatedly restarting.

kubectl describe pod <pod>

Inspect events, container state and configuration.

kubectl logs <pod> --previous

Read logs from the previous crashed container.

kubectl get events --sort-by=.lastTimestamp

Review recent Kubernetes events.

Kubernetes ImagePullBackOff

kubectl describe pod <pod>

Check image pull errors and events.

kubectl get secret

Check whether image pull secrets exist.

kubectl get pod <pod> -o yaml

Inspect the pod image and imagePullSecrets configuration.

kubectl get events --sort-by=.lastTimestamp

Find registry authentication or image errors.

Kubernetes Pod Pending

kubectl get pods -o wide

Check where pending pods are scheduled or waiting.

kubectl describe pod <pod>

Inspect scheduler events and resource constraints.

kubectl get nodes

Check available cluster nodes.

kubectl describe node <node>

Inspect node capacity, conditions and taints.

Kubernetes Service Not Reachable

kubectl get svc

Check service configuration and ports.

kubectl get endpoints <service>

Verify that the service has backend endpoints.

kubectl get pods --show-labels

Compare pod labels with the service selector.

kubectl describe svc <service>

Inspect service selectors, ports and endpoints.

Kubernetes Node NotReady

kubectl get nodes

Identify nodes reporting NotReady.

kubectl describe node <node>

Inspect node conditions and recent events.

kubectl get pods -A -o wide

Check workloads running on affected nodes.

kubectl get events -A --sort-by=.lastTimestamp

Review cluster-wide events.

Jenkins Build Failure

docker logs <jenkins-container>

Inspect Jenkins container logs when Jenkins runs in Docker.

java -version

Verify the Java runtime used by Jenkins.

df -h

Check whether disk exhaustion is affecting builds.

free -h

Check available memory on the Jenkins host.

Terraform State Problems

terraform state list

List resources tracked in Terraform state.

terraform state show <resource>

Inspect a resource stored in state.

terraform plan

Compare configuration against current state and infrastructure.

terraform refresh

Refresh state from infrastructure when supported by the workflow.

Prometheus Target Down

curl http://localhost:9090/-/healthy

Check Prometheus health.

curl http://localhost:9090/api/v1/targets

Inspect target states through the Prometheus API.

curl http://target:9100/metrics

Check whether an exporter endpoint responds.

promtool check config prometheus.yml

Validate Prometheus configuration.

Production Incident Workflow

date

Record the current time when beginning incident investigation.

uptime

Check host uptime and system load.

df -h

Check for disk exhaustion.

free -h

Check memory pressure.

ss -lntup

Inspect listening network services.

journalctl -p err -n 100

Review recent high-priority system errors.

Need the command, not the explanation?

Explore the DevOps command references for Kubernetes, Linux, Docker, AWS, Terraform, Ansible, Jenkins and more.

Explore DevOps Commands →