Understanding Container Network Interfaces (CNI): A Practical Guide for Troubleshooting

As an administrator managing containerized environments, understanding Container Network Interfaces (CNI) is crucial for effective troubleshooting. This guide will help you understand CNI basics and common troubleshooting scenarios for Kubernetes clusters using routes and tunnels as Calico CNI does.

We all know the NIC acronym which stands for Network Interface Card, similarly Container Network Interface (CNI) is like a universal plug adapter for container networking. It’s a standard way to configure network interfaces for Linux containers, regardless of the container runtime (Docker, containerd, etc.) or the network plugin (Calico, Flannel, Weave, etc.) you’re using.

1. CNI Bridge

• Think of it as a virtual switch on your host
• Usually named cni0 or similar
• Connects all pods on a node

2. Pod Network Namespace

• Each pod gets its own isolated network space
• Contains its own network interface (usually eth0)
• Has its own IP address and routing table

3. Virtual Ethernet Pairs (veth)

• Work like a virtual network cable
• One end connects to the pod
• Other end connects to the CNI bridge

Scenario 1: Pod Can’t Reach Other Pods

Check these first:
# Check pod's network interface
kubectl exec -- ip addr

# Check pod’s routing table
kubectl exec — ip route

# Verify CNI bridge exists
ip link show cni0

Common causes:

1. CNI plugin misconfiguration
2. Network policy blocking traffic
3. Corrupted CNI configuration

Scenario 2: Pod Can’t Reach External Services

Troubleshooting steps:

# Check node's DNS resolution
kubectl exec <pod-name> -- nslookup kubernetes.default

# Verify outbound connectivity
kubectl exec <pod-name> — ping 8.8.8.8

# Check pod’s DNS configuration
kubectl exec <pod-name> — cat /etc/resolv.conf

Scenario 3: Pod Stuck in “ContainerCreating” State

Investigation path:

# Check CNI logs
journalctl -u kubelet | grep cni

# Verify CNI configuration
ls /etc/cni/net.d/

# Check kubelet logs
journalctl -u kubelet

1. Network Connectivity Checks

# Check pod networking details
kubectl get pod <pod-name> -o wide

# Test network connectivity between pods
kubectl exec <pod-name> — curl <other-pod-ip>

# View CNI configuration
cat /etc/cni/net.d/10-*.conf

2. Network Plugin Status

# Check CNI pods status (for Kubernetes)
kubectl get pods -n kube-system | grep cni

# Verify CNI binaries
ls /opt/cni/bin/

3. Node Network Status

# Check node interfaces
ip addr show

# View routing table
ip route

# Check iptables rules (if using iptables mode)
iptables-save | grep KUBE

• Regular Health Checks
  • Monitor CNI plugin pods
  • Check network latency between pods
  • Verify DNS resolution regularly
• Documentation
  • Keep network diagrams updated
  • Document IP ranges and network policies
  • Maintain troubleshooting runbooks
• Backup and Recovery
  • Backup CNI configurations
  • Keep known-good configurations ready
  • Document recovery procedures