Kubernetes Network Model #

When we design and operate large-scale distributed systems in Kubernetes, one of the biggest challenges we often face is how application components connect to each other securely, reliably, and quickly. In a dynamic cluster, hundreds to thousands of containers can be created, killed, and moved within seconds.

To tackle this communication complexity, Kubernetes chooses a very specific, structured networking approach that fundamentally differs from the standard Docker network model (docker-scoped networking).

Before we go further discussing higher-level abstractions like Service, Ingress, or NetworkPolicy, we must understand the basic concept that forms the foundation of the entire Kubernetes networking system: the Kubernetes Network Model. This network model isn’t an implementation suggestion — it’s an absolute architectural contract that every network plugin running on top of our Kubernetes cluster must fulfill.

The Four Fundamental Rules (The Four Immutable Rules) #

The Kubernetes network model is built on four main non-negotiable rules. Every network implementation (whether AWS VPC CNI, Calico, Cilium, or Flannel) must comply with these rules to ensure the entire Kubernetes system interacts normally:

flowchart TD
    Contract["KUBERNETES NETWORK CONTRACT"]
    Contract --> Rule1["Rule 1 (Every Pod has its own unique IP across the whole cluster)"]
    Contract --> Rule2["Rule 2 (All Pods connect directly to each other without NAT)"]
    Contract --> Rule3["Rule 3 (Host nodes can communicate with all Pods without NAT)"]

Rule 1: Every Pod Gets Its Own Unique IP Address #

In Kubernetes, the smallest unit of our deployment isn’t a container, but a Pod. Therefore, IP addresses are allocated at the Pod level, not individual containers. All containers inside one Pod share the same IP address, the same port space, and the same network interface. No complicated port mapping (like docker run -p 8080:80) is needed to connect those containers to the outside.

Rule 2: All Pods Can Communicate With Each Other Without NAT #

Any Pod in the cluster can reach any other Pod directly using its destination IP address. This communication must run without Network Address Translation (NAT), regardless of whether the destination Pod is on the same worker node (intra-node) or a different worker node (inter-node). From a Pod’s perspective, the entire cluster looks like one flat virtual LAN network (flat network).

Rule 3: Worker Nodes Can Reach All Pods Without NAT #

Every process running at the worker node (host) OS level must be able to send data packets to any Pod on that worker node or other worker nodes directly without NAT. This rule is very important so system agents like the Kubelet, node-level monitoring systems, and log collector daemons can accurately monitor our containers’ health and status.

Rule 4: The IP Address Seen by a Pod Is the Same IP Address Seen by Outside Parties #

If a Pod asks its own OS about its internal IP address (e.g. through a loopback query), the returned IP address must be exactly the same IP address other Pods see when receiving packets from it. No internal IP masking (IP masquerading) occurs in inter-Pod communication. This greatly simplifies debugging and network log recording.

Flat Network: One Unified IP Address Space #

The fundamental rules above give birth to what’s called a Flat Network. In Kubernetes, we treat all Pods as if they’re connected to the same giant physical network switch.

flowchart TD
    subgraph DockerStandalone["Docker Standalone (Host-Scoped)"]
        direction LR
        subgraph NodeA["Node A (Host 1)"]
            ContA["Container A<br>172.17.0.2"]
            ContB["Container B<br>172.17.0.3"]
        end
        subgraph NodeB["Node B (Host 2)"]
            ContC["Container C<br>172.17.0.2"]
            ContD["Container D<br>172.17.0.3"]
        end
        NodeA -->|"NAT via Port 8080"| Network["External Network"]
        NodeB -->|"NAT via Port 9090"| Network
    end

    subgraph KubernetesFlat["Kubernetes Flat Network (Cluster-Scoped)"]
        direction LR
        subgraph NodeK1["Worker Node 1 (10.0.0.10)"]
            PodA["Pod A<br>10.244.1.5"]
            PodB["Pod B<br>10.244.1.6"]
        end
        subgraph NodeK2["Worker Node 2 (10.0.0.11)"]
            PodC["Pod C<br>10.244.2.3"]
            PodD["Pod D<br>10.244.2.4"]
        end
        NodeK1 <-->|"Direct Routing (No NAT)"| NodeK2
    end

In the traditional Docker standalone model, Container A on Host 1 and Container C on Host 2 have no direct route to each other. They can even have overlapping internal IP addresses (e.g. both getting 172.17.0.2). To connect them, we’re forced to expose ports to the host OS (port forwarding) and rely on NAT processes at the host kernel level.

In contrast, in the Kubernetes Flat Network model:

• Every Pod in the cluster has a universally unique IP within the Cluster CIDR range (e.g. 10.244.0.0/16).
• Pod A (10.244.1.5) can ping or send HTTP data directly to Pod C (10.244.2.3) without any port mapping help.
• Our applications don’t need to know which worker node the destination Pod is on. Our network logic becomes much simpler because we can treat Pods like independent server machines on a local network.

How Pod IPs Are Allocated (Pod CIDR) #

To ensure no IP addresses overlap (IP conflicts) across the cluster, Kubernetes uses a structured IP range division strategy for each worker node.

When the cluster initializes, the administrator determines one main IP range for all Pods in the cluster, called the Cluster CIDR (e.g. 10.244.0.0/16). This range is then divided into small non-overlapping segments for each worker node, called Pod CIDRs.

Let’s look at this IP segment allocation visualization:

flowchart TD
    Cluster["Cluster CIDR: 10.244.0.0/16"] -. Segment Allocation .-> Node1["Worker Node A: 10.244.1.0/24"]
    Cluster -. Segment Allocation .-> Node2["Worker Node B: 10.244.2.0/24"]
    Cluster -. Segment Allocation .-> Node3["Worker Node C: 10.244.3.0/24"]
    
    Node1 --> PodA1["Pod A1: 10.244.1.2"]
    Node1 --> PodA2["Pod A2: 10.244.1.3"]
    
    Node2 --> PodB1["Pod B1: 10.244.2.2"]
    Node2 --> PodB2["Pod B2: 10.244.2.3"]

When we add a new worker node to the cluster, the Kubernetes control plane (via the Controller Manager) detects that node and allocates an exclusive CIDR subnet block (e.g. 10.244.1.0/24 for Node A, capable of hosting up to 254 Pods).

Every time the Kubelet on Node A is ordered to create a new Pod:

1. The Kubelet interacts with the active CNI plugin on that node.
2. The CNI plugin takes one available free IP address from that node’s CIDR block (e.g. 10.244.1.2).
3. That IP address is attached to the new Pod and registered back to the API Server.

[!WARNING] Pod IP addresses are ephemeral. The IP is tied to the Pod’s lifecycle. If our Pod dies, is deleted, or moves to another node due to node maintenance, the replacement Pod gets a completely different IP from the new node’s CIDR block. Therefore, we must never hardcode Pod IP addresses statically in our application code for inter-service communication.

Pod Network Namespaces (Linux Network Namespaces) #

To understand how containers inside one Pod can communicate via localhost, we must look under the hood of the Linux OS, specifically at the Network Namespace (netns) feature.

A network namespace is a Linux kernel isolation feature providing a virtual copy of the OS network stack, consisting of network interfaces, routing tables, and firewall rules (iptables/nftables).

By default, every Docker container running on Linux gets its own unique network namespace. However, Kubernetes does something different:

flowchart TD
    subgraph PodNamespace["POD A NETWORK NAMESPACE"]
        direction TB
        lo["Loopback Interface (lo) -> 127.0.0.1"]
        eth0["Ethernet Interface (eth0) -> 10.244.1.5"]
        
        subgraph Containers["Containers sharing same namespace"]
            direction LR
            App["App Container (Port 80)"]
            Sidecar["Sidecar Container (e.g. Envoy Proxy)"]
            App <-->|"localhost"| Sidecar
        end
    end

1. Container Sandbox (Pause Container): When a Pod is created, Kubernetes first starts a small special container that does nothing, called the Pause Container. This container exists solely to initialize a new network namespace for that Pod.
2. Namespace Sharing: When our application container and sidecar container (e.g. a log forwarder or service mesh proxy) start inside the same Pod, Kubernetes instructs the container runtime (like containerd) to place those containers into the Pause Container’s network namespace.
3. Communication Implications:
   • Intra-Pod Communication (Localhost): Because containers in one Pod share the same namespace, they can communicate with each other directly using the loopback address 127.0.0.1 (localhost). For example, our web app on Port 80 can reach a db-proxy sidecar on Port 5432 via localhost:5432.
   • Port Isolation: Two containers in the same Pod must not use the same port (e.g. both competing for Port 80), because that triggers an Address already in use error.
   • Inter-Pod Communication: Other Pods wanting to reach our application must use the Pod’s external IP address (10.244.1.5) or go through a Service object intermediary.

The Role of CNI (Container Network Interface) as the Contract Executor #

Kubernetes is designed as a modular orchestration platform. The Kubernetes API Server itself has no code for creating virtual network cards, managing routing tables on the Linux host, or building overlay tunnels. Kubernetes fully delegates those tasks to Container Network Interface (CNI) plugins.

CNI is an industry-standard spec defining how container runtimes (like containerd or CRI-O) interact with third-party network plugins through standard JSON configuration manifest files.

The Kubelet interacts with the CNI plugin through two main commands:

• ADD: Called by the Kubelet when a new Pod is about to start. The CNI must create a virtual network interface (usually a veth pair), insert one end into the Pod’s network namespace, allocate an IP from the IPAM pool, and update the host routing table so data packets know how to reach that Pod.
• DEL: Called when a Pod is deleted. The CNI must clean up the created virtual interfaces and return the allocated IP back to the pool for future Pods to use.

Here’s a comparison of the most popular CNI plugins commonly used in the industry:

It’s important to note that no matter which CNI we choose at the infrastructure level, how we write Kubernetes manifests (Deployments, Services, Ingresses) stays exactly the same. CNI is just an implementation detail behind the scenes.

Fundamental Differences from Docker Standalone Networking #

To clarify our understanding, let’s compare the network architecture differences between Docker Standalone (which we often use on laptops during development) and Kubernetes:

Anti-Patterns vs Solutions in Kubernetes Networking #

Let’s study some network configuration mistakes (anti-patterns) most often made by developers in Kubernetes, along with how to fix them.

Anti-Pattern 1: Using Docker-Style hostPort in the Pod Spec #

We deploy a stateless application Pod (e.g. a web server) and include the hostPort parameter on the container spec so the container’s port 80 directly attaches to the physical worker node’s port 80, similar to the docker run -p 80:80 command.

Wrong Manifest Code (Locking the Host Node Port) #

# DON'T DO THIS IN PRODUCTION: Physical host port conflicts
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app-hostport-failure
  namespace: production
spec:
  replicas: 3 # Trying to deploy 3 replicas
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
        - name: web-container
          image: nginx:alpine
          ports:
            - containerPort: 80
              hostPort: 80 # Prevents the second Pod from being scheduled on the same node!

The Bad Consequences #

If our cluster only has 2 physical worker nodes, the third Pod replica stays stuck in Pending status forever. This happens because port 80 on both physical nodes has already been claimed by the first two Pods. Using hostPort drastically limits our cluster’s scalability and confuses the Kubernetes scheduler.

Solution Code (Using the Service Abstraction) #

We must let Pod ports run freely in their own virtual network space without hostPort, then use a Service object to route traffic evenly to those Pods.

# SOLUTION: Separate port publication concerns into a Service object
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app-clean
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
        - name: web-container
          image: nginx:alpine
          ports:
            - containerPort: 80 # Safe virtual container port without hostPort
---
apiVersion: v1
kind: Service
metadata:
  name: web-app-service
  namespace: production
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: 80
  selector:
    app: web-app

Anti-Pattern 2: Using hostNetwork: true on Regular Application Pods #

We enable the hostNetwork: true feature on our microservice application Pod manifests, claiming we want to improve network performance or ease direct access to the worker node VM’s internal resources.

Wrong Manifest Code (Bypassing Host Network Isolation) #

# DON'T DO THIS: Breaks cluster isolation security
apiVersion: v1
kind: Pod
metadata:
  name: backend-hostnetwork-risky
  namespace: production
spec:
  hostNetwork: true # The Pod uses the host VM machine's network namespace directly!
  containers:
    - name: backend-app
      image: my-backend:v1
      ports:
        - containerPort: 8080

The Bad Consequences #

By enabling hostNetwork, the Pod no longer runs inside an isolated network namespace. The Pod can see all the host VM machine’s physical network interfaces, monitor the worker node OS’s data traffic, and dramatically increase security risk. If a hacker successfully exploits the container, they can directly access our internal cloud VPC network as if they were the VM host server’s administrator.

Solution Code (Use the Default Isolated Namespace) #

Use Kubernetes’ built-in virtual network configuration and leverage NetworkPolicies to securely restrict network traffic access.

# SOLUTION: Let the Pod run in an isolated namespace (hostNetwork: false by default)
apiVersion: v1
kind: Pod
metadata:
  name: backend-app-secure
  namespace: production
spec:
  # hostNetwork: false (Disabled by default for security)
  containers:
    - name: backend-app
      image: my-backend:v1
      ports:
        - containerPort: 8080

Summary #

• The Kubernetes Network Model is an absolute contract: Every Pod has a unique IP, all Pods connect directly without NAT, worker nodes can reach all Pods without NAT, and a Pod’s internal IP address is the same IP seen from outside.
• Flat Networks simplify architecture: All Pods live in one flat, cluster-unique IP address space, eliminating traditional port forwarding hassles.
• Pod IPs are ephemeral: Always use the Service object abstraction for inter-service communication instead of relying on Pod IPs that easily change when Pods are recreated.
• The Pause Container initializes the namespace: Containers in one Pod share the same network namespace via the Pause Container, enabling localhost communication between containers in one Pod.
• CNI plugins handle technical details: CNI plugins like Calico, Cilium, or Flannel implement the Flat Network model without affecting how we write Kubernetes manifests.
• Avoid hostPort and hostNetwork: These two features break cluster portability and open serious security holes. Always use Service objects for safe application port exposure.

← Previous: Storage Performance   Next: Pod-to-Pod Communication →