Kubernetes Pod CrashLoopBackOff

A Pod STATUS of CrashLoopBackOff isn't a single error —
it's an umbrella term for "the container keeps failing to start and kubelet keeps restarting it."
The same STATUS can have completely different root causes.
Today I deliberately created three failure scenarios; the way you investigate each one differs.

note

CrashLoopBackOff is the state where kubelet "restarts the container repeatedly, backing off between restarts," which assumes a restartPolicy that restarts (Always or OnFailure). The examples below all use restartPolicy: Never so the root cause shows up quickly — a failed container won't be restarted, and the STATUS stops directly at Error or OOMKilled. But the way you find the root cause is exactly the same. To see CrashLoopBackOff with your own eyes, drop --restart=Never and use the default Always; RESTARTS will keep climbing.

Pitfall 1: command not found

kubectl run crash-cmd --image=busybox --restart=Never -- /bin/sh -c "notacommand"

• Symptom: under restart=Never the STATUS stops at Error with RESTARTS at 0; switch to the default Always and you'll see CrashLoopBackOff with RESTARTS incrementing.
• Investigation: kubectl logs crash-cmd --previous → shows notacommand: not found.
• Cause: the entrypoint command itself doesn't exist, so the container exits the moment it starts.
• Fix: use a valid command.
• Concept: CrashLoop means "keeps restarting," not "can't be scheduled." The container did start — it just dies immediately — so you can see the application-level error in the logs.

Pitfall 2: missing env

kubectl run crash-env --image=busybox --restart=Never -- /bin/sh -c 'test -n "$REQUIRED" || (echo "REQUIRED missing"; exit 2)'

• Symptom: fails the same way (Error under Never, CrashLoopBackOff under Always).
• Investigation: kubectl logs crash-env --previous → REQUIRED missing, exit code 2.
• Cause: the program self-checks for a required environment variable at startup and exits on its own when it's missing.
• Fix: supply the env var (--env=REQUIRED=x, or inject it via the Deployment's env: / a ConfigMap / a Secret).
• Concept: any non-zero exit code is treated as a failure and triggers a restart. For this kind of "the app decides to exit itself" error, the logs usually explain why — checking the logs first is the easiest path.

Pitfall 3: OOMKilled

# set an extremely small memory limit, then run something that eats memory
kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata: { name: crash-oom }
spec:
  restartPolicy: Never
  containers:
  - name: stress
    image: polinux/stress
    resources: { limits: { memory: "20Mi" } }
    command: ["stress"]
    args: ["--vm","1","--vm-bytes","100M","--vm-hang","1"]
EOF

• Symptom: STATUS shows OOMKilled, but kubectl logs --previous reveals nothing (it may even be empty).
• Investigation: kubectl describe pod crash-oom → look at Last State: Terminated, Reason: OOMKilled, exit code 137.
• Cause: the container's usage exceeded the memory limit and the kernel OOM killer killed it — the app didn't exit on its own, it was killed by an external force.
• Fix: raise resources.limits.memory, or fix the real memory leak / reduce usage.
• Concept: OOMKilled never makes it into the logs, because logs are the "container's stdout/stderr," whereas OOM is the kernel sending SIGKILL directly — the app never gets a chance to print anything. You have to look at Last State via describe. Exit code 137 = 128 + 9 (SIGKILL).

---

Debug Summary

STATUS	Container State	Kubectl	Root Cause
CrashLoopBackOff / Error	Terminated (then restarting)	kubectl logs --previous	command not found, missing env, exit code != 0
OOMKilled	Terminated	kubectl describe -> Last State	Out Of Memory
ImagePullBackOff	Waiting	kubectl describe -> Events	Wrong image name/tag, missing imagePullSecrets