A Kubernetes node suddenly changes from:
Readyto:NotReady
At first, everything looks normal. The node is still running. The server is reachable over SSH. Some containers may even still be serving traffic.
But Kubernetes reports:
kubectl get nodes
NAME STATUS ROLES AGE
worker-01 NotReady <none> 42d
Now workloads may stop being scheduled onto the node, pods can become unhealthy, and the cluster may start moving workloads elsewhere.
So what actually happened?
The answer is usually somewhere between the kubelet, operating system, container runtime, networking, and node resources.
In this article, we'll investigate seven common reasons a Kubernetes node becomes NotReady:
- DiskPressure
- PIDPressure
- MemoryPressure
- NetworkUnavailable
- CNI failure
- Container runtime failure
- Kubelet certificate expiration
First: What Does NotReady Actually Mean?
A node doesn't become NotReady simply because the machine is down. Kubernetes continuously receives node status and heartbeats from the kubelet.
A simplified flow looks like this:
Node
|
+-- kubelet
| |
| +-- Node status
| +-- Heartbeats
| +-- Container health
| +-- Resource conditions
|
+-- Container Runtime
|
+-- CNI
|
+-- Linux / OS
The control plane uses this information to determine whether the node is healthy enough to run workloads.
Check the node: kubectl get nodes
Then: kubectl describe node <node-name>
Pay particular attention to:
Conditions:
Ready
MemoryPressure
DiskPressure
PIDPressure
NetworkUnavailable
These conditions are usually your first major clue.
1. DiskPressure
One of the most common causes of node problems is simply:
The node is running out of disk space or available inodes.
Check: df -h
Then check inode usage: df -i
You might discover:
Filesystem Size Used Avail Use%
/dev/sda1 100G 98G 2G 98%
Kubernetes and the container runtime need disk space for things such as:
- container images
- container writable layers
- logs
emptyDir- kubelet data
- container runtime metadata
A node under serious disk pressure can report: DiskPressure=True
Check the node condition
kubectl describe node <node-name>
Look for: DiskPressure True
Also inspect kubelet logs: journalctl -u kubelet --since "30 min ago"
Search for messages related to:
disk pressure
eviction
image garbage collection
filesystem
no space left
What to investigate
Find what's consuming disk: du -sh /var/log/*
Check container runtime storage and kubelet directories as well. Don't blindly delete files from runtime directories. Instead, identify the source of the growth. A common culprit is excessive container logging.
2. PIDPressure
Linux processes consume process IDs. A node running too many processes can eventually hit its PID limit.
Kubernetes represents this condition as: PIDPressure=True
Check the node: kubectl describe node <node-name>
Then inspect processes on the host: ps aux | wc -l
You can also inspect PID-related limits: cat /proc/sys/kernel/pid_max
What causes PID exhaustion?
Common causes include:
-
applications spawning excessive processes
-
broken workloads creating process storms
-
zombie processes
-
containers with unexpectedly high process counts
-
badly behaved agents or monitoring software
For example:
Application
|
+-- process
+-- process
+-- process
+-- process
+-- thousands more
Eventually the node can run out of available PIDs.
Troubleshooting
Look for processes consuming large numbers of PIDs. Inspect the busiest containers and workloads.
Then check kubelet logs:
journalctl -u kubelet --since "30 min ago"
If PID pressure is caused by a specific workload, fixing the application is more important than simply restarting kubelet.
3. MemoryPressure
Memory pressure occurs when the node doesn't have enough available memory.
Check: free -h
And: kubectl describe node <node-name>
You may see: MemoryPressure=True
Why does this happen?
Possible causes include:
-
containers exceeding memory limits
-
workloads without appropriate limits
-
memory leaks
-
too many pods on the node
-
system processes consuming memory
-
monitoring or security agents using unexpected amounts of RAM
Check memory-consuming processes: ps aux --sort=-%mem | head
For Kubernetes workloads: kubectl top node
and: kubectl top pods -A
if Metrics Server or another metrics provider is available.
Watch for OOM events
Check kernel logs: dmesg | grep -i oom
You may find evidence that the Linux kernel's out-of-memory mechanism has killed processes. Kubernetes may also start evicting pods when node memory crosses configured thresholds.
The important distinction is:
A
NotReadynode caused by memory pressure is usually a resource-capacity problem, not a kubelet bug.
4. NetworkUnavailable
A node can also report a networking-related condition.
Check: kubectl describe node <node-name>
Look for: NetworkUnavailable
This condition is particularly relevant during node startup or when the networking layer hasn't been successfully configured. The exact behavior depends on your Kubernetes networking setup and whether a cloud controller manager or CNI plugin manages the condition.
What can cause it?
Potential causes include:
-
CNI initialization failure
-
broken node networking
-
cloud networking problems
-
incorrect routes
-
missing interfaces
-
firewall/security-group changes
Start by checking the node's network interfaces: ip addr
Then routes: ip route
And basic connectivity: ping <gateway>
For Kubernetes networking, also inspect the CNI pods.
5. CNI Crash or Failure
Sometimes the node itself is healthy, but Kubernetes networking isn't. This can happen when the CNI plugin crashes or becomes unable to configure pod networking.
Depending on your cluster, the CNI might be something such as:
- Cilium
- Calico
- Flannel
- another networking implementation
Check networking pods: kubectl get pods -A
Look for CNI-related pods that are:
CrashLoopBackOff
Error
Pending
NotReady
Then inspect them: kubectl logs -n <namespace> <cni-pod>
Check the node
On the affected node, inspect: ip link
and: ip route
Also inspect CNI configuration: ls -la /etc/cni/net.d/
A broken CNI can lead to symptoms such as:
Node NotReady
Pod networking broken
New pods stuck
DNS failures
Service connectivity failures
Important distinction
Don't assume:
Node NotReady= kubelet failure.
A networking failure can make the node appear unhealthy even when the kubelet process itself is running. Always inspect the CNI layer separately.
6. Container Runtime Failure
Kubelet doesn't directly run containers. It communicates with a container runtime through the CRI.
A simplified architecture is:
Kubelet
|
| CRI
v
Container Runtime
|
v
Containers
If the runtime becomes unavailable, kubelet may be unable to manage pods correctly.
Check the runtime service.
For containerd:
systemctl status containerd
Check its logs:
journalctl -u containerd --since "30 min ago"
You can also check whether the runtime is responding:
crictl info
if crictl is installed and configured.
Common runtime failures
Possible causes include:
-
runtime crash
-
corrupted runtime state
-
disk exhaustion
-
runtime upgrade problems
-
image filesystem issues
-
runtime socket problems
-
resource exhaustion
You may see kubelet errors such as:
failed to get runtime status
container runtime is down
failed to create pod sandbox
The key question becomes:
Is kubelet unhealthy, or is kubelet healthy but unable to communicate with the runtime?
That distinction saves a lot of debugging time.
7. Kubelet Certificate Expiration
This one can be particularly confusing.
The node may look perfectly healthy at the operating-system level:
CPU: OK
Memory: OK
Disk: OK
Network: OK
Runtime: OK
But the kubelet may have a problem authenticating with the Kubernetes API server. Kubelet uses certificates for secure communication with the control plane. If the relevant certificate expires and renewal isn't working correctly, node heartbeats and API communication can fail.
The result may eventually look like: Node NotReady
Check kubelet logs
Start with: journalctl -u kubelet --since "1 hour ago"
Look for errors involving:
certificate
x509
expired
authentication
TLS
For example: x509: certificate has expired is a strong clue.
Why certificate problems are easy to miss
Certificate expiration often doesn't look like a resource problem.
You can have:
DiskPressure = False
MemoryPressure = False
PIDPressure = False
and still have: Ready = False
because the kubelet cannot successfully communicate with the API server. Always check kubelet logs when the usual node conditions look normal.
8. A Simple Production Troubleshooting Workflow
When a node suddenly becomes NotReady, don't start by rebooting it.
Follow the request path.
Step 1 — Check the node
kubectl get node <node-name>
Then:
kubectl describe node <node-name>
Check:
Ready
MemoryPressure
DiskPressure
PIDPressure
NetworkUnavailable
Step 2 — Check kubelet
SSH into the node: systemctl status kubelet
Then: journalctl -u kubelet --since "30 min ago"
Step 3 — Check resources
free -h
df -h
df -i
Look for:
Memory exhaustion
Disk exhaustion
Inode exhaustion
Step 4 — Check the runtime
For containerd: systemctl status containerd
Then: crictl info
Step 5 — Check networking
ip addr
ip route
Then check your CNI pods: kubectl get pods -A
Step 6 — Check certificates
Inspect kubelet logs for:
x509
certificate
expired
TLS
authentication
9. The Fast Diagnosis Matrix
| Symptom | First thing to check |
|---|---|
DiskPressure=True | df -h, df -i |
MemoryPressure=True | free -h, kubectl top node |
PIDPressure=True | Process/PID count |
NetworkUnavailable=True | CNI and node networking |
| CNI pods crashing | CNI logs/configuration |
| Runtime unavailable | containerd / CRI |
| Certificate errors | kubelet logs and certificates |
Ready=False with normal resources | kubelet logs + API connectivity |
10. The Production Incident Pattern
Imagine this happens at 2 AM.
Monitoring reports: Node worker-07: NotReady
You run: kubectl describe node worker-07
and see:
MemoryPressure=False
DiskPressure=False
PIDPressure=False
NetworkUnavailable=False
Ready=False
Don't immediately reboot the server. SSH into it.
Check: systemctl status kubelet
Then: journalctl -u kubelet --since "30 min ago"
You discover: x509: certificate has expired
Now the incident makes sense. The server wasn't overloaded. The container runtime wasn't necessarily broken. The kubelet simply couldn't establish valid authenticated communication with the API server.
That's why understanding the node condition + kubelet logs + underlying system is so important.
Final Takeaway
When Kubernetes reports: Node NotReady
the node isn't necessarily dead. The problem can come from several layers:
Node NotReady
|
+------------------+------------------+
| | |
v v v
Resources Networking Kubelet
| | |
DiskPressure CNI failure Runtime failure
MemoryPressure Network issue Certificate
PIDPressure
Start with: kubectl describe node <node-name>
Then move down the stack:
Node Conditions
↓
Kubelet
↓
Operating System
↓
Container Runtime
↓
CNI / Networking
↓
Certificates / API connectivity
The most important lesson is:
NotReadyis a symptom, not a root cause.
Before restarting or replacing the node, find out why the kubelet stopped reporting the node as healthy.
A few minutes spent reading the node conditions and kubelet logs can turn a mysterious production incident into a very specific problem: disk, memory, PIDs, networking, CNI, runtime, or certificates.

