Problems It Solves #

Kubernetes wasn’t born from a theoretical design on an academic’s desk. Instead, it was forged on the “battlefield” of real-world infrastructure operations. As microservices architecture was adopted en masse and containerization became the new standard, operations teams and developers hit a thick wall of complexity in managing distributed systems.

Understanding the concrete problems Kubernetes solves is essential. It helps us honestly assess whether the problems we face today are actually relevant to what Kubernetes offers, or whether we’re dealing with a different kind of problem that this tool simply can’t cure.

Problem 1: Manual Deployment Complexity & Server Scalability #

In the early days of Docker adoption, the typical deployment scenario in the industry looked deceptively simple: developers build a container image, push it to a registry, SSH into the target server, then run shell commands to stop the old container and start the new one.

However, this imperative (step-by-step) model quickly became an operational nightmare as system scale grew.

ANTI-PATTERN: Traditional Manual Deployment Script
// WHAT WE DO:
- Write a bash script like this to deploy to 10 servers:
  #!/bin/bash
  for server in server1 server2 server3 ... server10; do
    ssh user@$server "docker pull myapp:v2 && docker stop app && docker rm app && docker run -d --name app -p 80:80 myapp:v2"
  done
// THE CONSEQUENCES IN PRODUCTION:
- Human Error: If the SSH connection drops at server 5, our cluster is left in an inconsistent state (half the servers run v1, half run v2).
- Downtime: Stopping the old container (`docker stop`) before starting the new one (`docker run`) creates a time gap where the service is unavailable to users.
- Hard Rollback: If the new version has a critical bug, we must repeat the slow manual SSH process across all servers to revert to the old version.

Kubernetes’ Declarative Solution #

Kubernetes eliminates the need for all the imperative scripts above. We simply submit a declarative manifest document (YAML) to the API Server:

# CORRECT: Kubernetes Declarative Deployment Manifest
apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-service
spec:
  replicas: 5
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: payment
    spec:
      containers:
      - name: app
        image: payment-service:v2
        ports:
        - containerPort: 8080

With the manifest above, the Kubernetes Controller automatically manages the transition:

  1. Launches 1 new version-2 container (maxSurge).
  2. Once the new container is healthy, Kubernetes routes traffic to it and shuts down 1 old version-1 container.
  3. The process repeats gradually with no single second of service downtime for users (zero-downtime).
  4. If the new version is detected failing to boot, the process stops immediately and the cluster rolls back to the previous version automatically.

Problem 2: System Failures Without Automatic Recovery (Self-Healing) #

In distributed systems, containers crashing is a fact of life. The causes vary: memory leaks (OOM Kills), deadlocked threads, dropped database connections, or hardware failure on the physical server node hosting the container.

Without an orchestrator, when a container crashes in the middle of the night, the system stays broken until an on-call engineer wakes up, analyzes the logs, and manually restarts the container.

Manual Recovery Scenario (Traditional):
  [02:10] API-Server container crashes (OOM Kills).
  [02:10] Service goes down, users see error 502 Bad Gateway.
  [02:15] Monitoring alert pages the on-call engineer.
  [02:22] Engineer wakes up, opens the laptop, connects to VPN.
  [02:27] Engineer SSHs into the server, diagnoses the crash.
  [02:30] `docker restart` command is run. Service recovers.
  Total Downtime: 20 Minutes.

Kubernetes Automatic Recovery Scenario (Self-Healing):
  [02:10] API-Server container crashes.
  [02:10] Kubelet on the node detects the container process failure.
  [02:10] Kubelet kills the broken container and launches a new one on the same node.
  [02:11] New container is ready to accept traffic. Service recovers.
  Total Downtime: < 10 Seconds (app startup time).

Kubernetes does this continuously through the Liveness Probe and Readiness Probe mechanisms. Kubernetes doesn’t just make sure the container process is running — it actively sends HTTP requests or TCP checks into the container to verify that the application inside is genuinely healthy, not merely “running without errors”.

Problem 3: Low Hardware Resource Efficiency & Utilization #

Before containerization and orchestration, the common server allocation pattern was to isolate applications into separate Virtual Machines to avoid port conflicts and library dependency clashes. Because VM capacity had to be reserved based on estimated peak traffic load, those VMs sat with lots of idle capacity during quiet hours.

This led to bloated cloud server bills while the cluster’s average CPU utilization across the board often sat below 20%.

Traditional VM Allocation (Wasteful):
  - VM-A (8 CPU) -> Dedicated to API Service (Peak Load: 6 CPU, Average: 1.5 CPU) -> 6.5 CPU Idle.
  - VM-B (8 CPU) -> Dedicated to Worker Service (Peak Load: 7 CPU, Average: 2 CPU) -> 6 CPU Idle.
  - VM-C (8 CPU) -> Dedicated to Reporting (Peak Load: 5 CPU, Average: 0.5 CPU) -> 7.5 CPU Idle.
  Total Resources Paid For: 24 CPU. Real Average Utilization: 16%.

Kubernetes’ Smart Scheduling (Bin-Packing) Solution #

Kubernetes treats every server node in the cluster as one large resource pool. When we deploy an application, we define its resource requirements explicitly:

# CORRECT: Declaring Realistic Resource Requirements
resources:
  requests:
    memory: "256Mi"
    cpu: "250m" # 0.25 CPU Core
  limits:
    memory: "512Mi"
    cpu: "500m" # 0.5 CPU Core

The Kubernetes Scheduler compares these declared requests against the remaining CPU and memory capacity of all available nodes. Kubernetes then optimally packs (bin-packing) multiple containers from different applications onto the same node.

If a node starts running out of capacity, Kubernetes spreads new containers to other nodes. As a result, average server machine utilization can be raised to 70-80%, which means we can cut the number of rented VMs and significantly reduce infrastructure budget.

Problem 4: Fragile Networking & Service Discovery #

In a microservices architecture, our applications have to talk to each other. The Frontend app must call the Backend API, and the Backend API must connect to the Database.

Without an orchestrator, managing this network configuration is a huge pain. A Docker container that gets restarted or moves to another server receives a new random IP address.

ANTI-PATTERN: Hardcoding Container IPs
// WHAT WE DO:
- Store the Backend's IP address directly in the Frontend config file:
  BACKEND_API_URL="http://192.168.1.45:8080"
// THE CONSEQUENCES IN PRODUCTION:
- When the Backend server crashes and the container is restarted on another server with IP `192.168.1.99`, the Frontend connection breaks immediately.
- We must manually edit the Frontend config file, commit code, and redeploy the Frontend just to update that IP address.

Kubernetes solves this problem with a built-in Service Discovery system using the Service component and CoreDNS.

sequenceDiagram
    participant Frontend as Frontend Pod
    participant DNS as Cluster CoreDNS
    participant Service as Backend Service (ClusterIP)
    participant Backend1 as Backend Pod 1 (IP v1)
    participant Backend2 as Backend Pod 2 (IP v2)

    Frontend->>DNS: Name resolution: "backend-service"
    DNS-->>Frontend: Returns stable ClusterIP (e.g. 10.96.0.10)
    Frontend->>Service: Sends Request to 10.96.0.10:80
    Note over Service: Service tracks Backend Pod dynamic IPs in realtime
    Service->>Backend2: Load Balances to a healthy Backend Pod

Every time you create a Service object in Kubernetes, it gets:

  1. A Stable DNS Name (e.g. http://backend-service). This name never changes as long as the Service exists.
  2. A Consistent Virtual IP Address (ClusterIP).
  3. An automatic tracking system that follows the dynamic IPs of the backend Pods behind it in realtime. If a Backend Pod dies and is replaced with a new IP, the Service object immediately updates its routing list without any manual intervention or disruption to the Frontend application’s configuration.

Problem 5: Scattered Configuration & Secret Management #

Applications need configuration (such as ports, database names) and sensitive data (API credentials, passwords, SSL certificates) to function. One fatal mistake teams often make is mixing this data in with the code.

ANTI-PATTERN: Storing Credentials in Code or Image Layers
// WHAT WE DO:
- Write database configuration directly inside the application's `config.properties` file:
  db.password="S3cr3tP@ssw0rd!"
- Push that file to a public/private Git repository.
- Bake credentials into the Dockerfile:
  ENV DATABASE_PASSWORD="S3cr3tP@ssw0rd!"
// THE CONSEQUENCES IN PRODUCTION:
- Credential Leak: Anyone with access to the git repository or image registry can see our production database password.
- Slow Changes: To change the database password, we must go through a full image build and redeploy cycle from scratch.

Kubernetes solves this by providing separate abstraction objects:

  • ConfigMap: For storing non-sensitive configuration parameters (e.g. LOG_LEVEL: "info").
  • Secret: For storing sensitive data that Kubernetes encrypts securely (at rest).

Both objects are declared outside our application code. When a container runs, Kubernetes dynamically injects the ConfigMap and Secret contents into the container as environment variables or mounts them as plain text files in memory (tmpfs). This keeps our application code clean of secrets and enables safe, fast credential rotation.

What Kubernetes Doesn’t Solve #

As a highly sophisticated system, Kubernetes is often viewed as a “silver bullet” that can solve all of an organization’s technology problems. That’s a mistaken expectation. We need to understand the limits of what Kubernetes can do:

1. It Doesn’t Fix Bad Application Architecture #

If you have a giant monolithic application that’s slow, memory-hungry, and not designed to be deployed in a distributed (non-stateless) way, moving it to Kubernetes won’t magically make it fast and scalable. Kubernetes manages containers, but it can’t fix the architecture of your code.

2. It Doesn’t Guarantee Application Security Automatically #

Kubernetes provides security features (such as RBAC, Network Policy, Pod Security Standards), but the platform can’t detect security vulnerabilities inside our application code (like SQL Injection or XSS). We remain fully responsible for the security of the applications we write.

3. It Doesn’t Eliminate Complexity #

Kubernetes shifts complexity from manually coordinating applications to managing a platform. Running a self-managed Kubernetes cluster requires a team of engineers who deeply understand networking, storage, TLS certificates, and distributed systems. For small-scale applications with low traffic, Kubernetes’ operational complexity often far outweighs its benefits.

Summary #

  • Manual Deployment Doesn’t Scale — Kubernetes replaces fragile SSH scripts with declarative YAML manifests and automated rolling update strategies.
  • Failures Without Self-Healing — Kubernetes cuts application recovery time from minutes (manual) to seconds by automatically restarting failed Pods.
  • Low Server Utilization — The Kubernetes Scheduler packs containers efficiently (bin-packing) based on declared requests/limits, saving VM costs.
  • Fragile Service Discovery — Through Services and cluster DNS, Kubernetes provides stable domain names for containers, eliminating the problem of ever-changing dynamic IPs.
  • Credential Leaks — ConfigMap and Secret separate configuration data and secrets from application code, keeping them safe and easy to update.
  • Not a Cure-All — Kubernetes cannot fix a broken monolith at the code level and actually adds operational burden when applied by teams that aren’t ready.

← Previous: What is Kubernetes?   Next: Kubernetes Alternatives →

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact