Cost Optimization #

Kubernetes offers extraordinary ease for automatically scaling applications. However, this ease also makes it easy to waste cloud infrastructure budgets if not managed with discipline. It’s very easy for engineering teams to launch test clusters and forget them, set Pod resource request limits far exceeding actual needs, or leave development and staging environments fully running 24/7 even though they’re only used 8 hours a day. Without systematic cost controls, cloud bills can skyrocket exponentially (cloud spend explosion). Cost Optimization is the application of FinOps (Financial Operations) methodology at the Kubernetes cluster level to identify, measure, and cut compute waste without sacrificing production system reliability. This article discusses tactical compute cost-saving strategies, Spot Node utilization with automatic fallback, scale-to-zero techniques, network consolidation, and namespace-based cost allocation governance.

Kubernetes Cloud Cost Component Architecture #

Before optimizing, we must map where Kubernetes cluster costs in the cloud flow. Cluster costs are divided into four main pillars:

flowchart TD
    TotalCost["Total Kubernetes Cluster Cost"] --> Compute["1. Compute (70-80%)\nCPU & RAM Virtual Machines"]
    TotalCost --> Storage["2. Storage (10-20%)\nPersistent Volumes, Registry, Snapshots"]
    TotalCost --> Network["3. Networking (5-15%)\nLoad Balancers, Egress, NAT Gateway Data"]
    TotalCost --> Management["4. Management Fee (Fixed)\nControl Plane SLA Fee"]

1. Compute (The Largest Portion: 70-80%) #

The rental cost of physical/virtual VMs acting as worker nodes. We pay for the total CPU and RAM capacity provided by Nodes, whether that capacity is actively used by containers or idle.

2. Storage (10-20%) #

Costs for Persistent Volumes (SSD/HDD) mounted by databases, backup snapshot storage costs, and container image storage costs in the Container Registry.

3. Networking (5-15%) #

A cost pillar often overlooked but can mysteriously balloon:

• Load Balancer (LB) Fees: Per-instance Load Balancer costs created by LoadBalancer-type Services.
• NAT Gateway Data Processing: Costs for data leaving private clusters to the internet. Cloud providers charge per GB of data passing through the NAT Gateway, which is often more expensive than the NAT VM cost itself.
• Egress Traffic: Data transfer costs across availability regions (cross-AZ traffic) or to the outside internet.

4. Control Plane Management Fee #

The flat fee charged by cloud providers for managing the master plane (e.g. AWS EKS charges $0.10/hour per cluster, or about $72/month).

Identifying Waste Using PromQL Queries #

The first step in FinOps is Inform (Identify). We must not reduce allocations without valid comparison data. Use the following Prometheus PromQL queries to detect containers suffering extreme over-provisioning (requests allocations far larger than real consumption).

1. Detecting Containers with CPU Usage Below 10% of Requests #

This query filters containers booking large CPU amounts but almost never using them.

# Finding containers with an active CPU to CPU Request ratio < 10%
(
  sum by (namespace, pod, container) (
    rate(container_cpu_usage_seconds_total{container!=""}[30m])
  )
  / 
  sum by (namespace, pod, container) (
    kube_pod_container_resource_requests{resource="cpu", container!=""}
  )
) < 0.1

2. Analyzing Memory Efficiency Percentages per Namespace #

Since memory is an expensive physical resource, we must monitor RAM usage efficiency in every namespace.

# Calculating the active memory percentage against total memory requests per namespace
sum by (namespace) (
  container_memory_working_set_bytes{container!=""}
) / 
sum by (namespace) (
  kube_pod_container_resource_requests{resource="memory"}
) * 100

Spot / Preemptible Nodes: Savings Strategies Up to 80% #

Cloud providers sell their empty spare server capacity with extreme discounts ranging 60% to 80% off normal rates through Spot (AWS) or Preemptible VM (GCP) instances. The downside is cloud providers reserve the right to reclaim those VMs at any time if customers are willing to pay normal rates (On-Demand), giving very short early warning notices (preemption notices) (2 minutes on AWS, 30 seconds on GCP).

Workload Type Grouping #

We must discipline the application types allowed on Spot Nodes:

Tolerance and Fallback Consequences in Deployment Manifests #

For our applications to run on Spot Nodes yet stay safe if Spot Nodes are mass-reclaimed by cloud providers at any time, we must configure preferred-type nodeAffinity rules (not required) and include matching tolerations.

# File: k8s/production-spot-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: billing-worker
  namespace: prod-apps
spec:
  replicas: 4
  template:
    spec:
      # 1. Tolerations so Pods are allowed to be scheduled on Spot Nodes
      tolerations:
      - key: "eks.amazonaws.com/capacityType" # For AWS EKS
        operator: "Equal"
        value: "SPOT"
        effect: "NoSchedule"
      - key: "cloud.google.com/gke-spot"     # For GKE
        operator: "Equal"
        value: "true"
        effect: "NoSchedule"
        
      # 2. Preference: Prioritize Spot, but allow fallback to On-Demand if Spot runs out
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100 # Give the highest weight for Spot priority
            preference:
              matchExpressions:
              - key: eks.amazonaws.com/capacityType
                operator: In
                values: ["SPOT"]
          - weight: 10 # Low weight for On-Demand
            preference:
              matchExpressions:
              - key: eks.amazonaws.com/capacityType
                operator: In
                values: ["ON_DEMAND"]

Scale-to-Zero Automation for Non-Production Environments #

Testing environments (development and staging) are usually only used by engineer teams during office working hours (08:00 to 18:00 WIB, Monday to Friday). This means for 16 hours on workdays and 48 full hours on weekends, the cluster runs with no testing activity at all, wasting 60% of rental costs in vain.

We can automate the shutdown (scale-to-zero) of all workloads in non-production environments outside working hours.

Option 1: Schedule Automation via the KEDA Cron Scaler #

Using KEDA to scale Deployment replicas to 0 outside working hours:

# File: k8s/dev-keda-autoscaler.yaml
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: dev-app-scaler
  namespace: dev-apps
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: dev-api-service
  minReplicaCount: 0 # Allow total shutdown (0 pods)
  maxReplicaCount: 5
  triggers:
  - type: cron
    metadata:
      timezone: "Asia/Jakarta"
      start: "0 8 * * 1-5"    # Turns on (scale up to 2) at 08:00 WIB, Monday-Friday
      end: "0 18 * * 1-5"     # Total shutdown (scale down to 0) at 18:00 WIB, Monday-Friday
      desiredReplicas: "2"

Option 2: Simple Kubectl CronJobs #

If we don’t install KEDA, we can use a standard Kubernetes CronJob object running scheduled kubectl scale commands:

# File: k8s/staging-shutdown-cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: staging-nightly-shutdown
  namespace: staging-apps
spec:
  schedule: "0 20 * * 1-5" # At 08:00 PM WIB at night, Monday-Friday
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: kubectl-scaler-sa # SA with RBAC permission to update Deployments
          restartPolicy: OnFailure
          containers:
          - name: kubectl
            image: bitnami/kubectl:latest
            command:
            - /bin/sh
            - -c
            - "kubectl scale deployment --all --replicas=0 -n staging-apps"

Network and Storage Consolidation (Hidden Cost Savings) #

Often teams focus on compute CPU/RAM costs but ignore cost leaks at the network and storage levels. Apply the following consolidation strategies to cut hidden spending:

1. Load Balancer Consolidation via Ingress Controllers #

By default, if we create LoadBalancer-type Services for every microservice application, cloud providers launch one new physical Load Balancer instance (e.g. AWS ALB or GCP LB) costing around $15 to $22 per month per unit.

ANTI-PATTERN (1 Load Balancer per Service):
  Service 1 (LoadBalancer) --> [AWS Network Load Balancer 1] --> $18/month
  Service 2 (LoadBalancer) --> [AWS Network Load Balancer 2] --> $18/month
  Service 3 (LoadBalancer) --> [AWS Network Load Balancer 3] --> $18/month
  Total Cost = $54/month

CORRECT (Consolidation via an Ingress Controller):
  Ingress Path: /api/v1/payment --> Service 1
  Ingress Path: /api/v1/billing --> Service 2  --> [Single Ingress (Shared LB)] --> $18/month
  Ingress Path: /api/v1/auth    --> Service 3
  Total Cost = $18/month (Save 67%)

2. NAT Gateway Data Processing Bypass #

NAT Gateways charge for processing outgoing data traffic (data processing fees). If our application Pods often download gigabyte-sized data from AWS S3 or Google Cloud Storage through public networks, that data passes through the NAT Gateway and triggers bill spikes.

✓ SOLUTION:
Install VPC Gateway Endpoints (AWS) or Private Google Access (GCP) in your VPC subnets. 
Data traffic from pods to Cloud Storage is routed through internal VPC paths 
without passing through the public NAT Gateway, cutting data processing costs up to 100% for internal storage access.

3. Snapshot and Container Registry Retention Policies #

Container images from automatic CI/CD pipeline builds piling up in Container Registries (like ECR/Harbor) can consume terabytes of storage. Apply automatic retention rules (Lifecycle Rules) to delete development-tagged container images (dev-* or temp-*) older than 14 days.

Cost Governance #

To operate FinOps sustainably, we must know which team or project is responsible for cluster cost consumption. Cost observability tools like Kubecost or OpenCost need consistent metadata labels to group spending.

We must install a Kyverno policy at the cluster level to force developers to always include cost allocation labels (team, environment, and project) before their applications are allowed to deploy to production clusters.

# File: k8s-governance/require-cost-labels-policy.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: enforce-cost-allocation-labels
spec:
  validationFailureAction: Enforce # Reject non-compliant manifests!
  background: true
  rules:
  - name: check-mandatory-labels
    match:
      any:
      - resources:
          kinds: ["Deployment", "StatefulSet", "DaemonSet"]
    validate:
      message: "Deploy Failed: You must include the 'team', 'environment', and 'project' labels for FinOps allocation."
      pattern:
        metadata:
          labels:
            team: "?*"        # Any characters, must not be empty
            environment: "?*"
            project: "?*"

Cost Optimization Practice Anti-Patterns #

Avoid the following fatal operational mistakes often wrapped in the name of cost savings:

1. Placing Main Databases on Spot Nodes (Data Loss Risks) #

Deploying main production relational database StatefulSets (like PostgreSQL) on Spot Nodes to save storage costs.

# ANTI-PATTERN: A database StatefulSet on Spot Nodes
spec:
  nodeSelector:
    eks.amazonaws.com/capacityType: SPOT # DON'T!

Operational Risks:
- When cloud providers send 2-minute preemption notices (withdrawal signals), databases must suddenly shut down connections.
- Database replication in etcd can split (*split-brain*) if all Spot nodes die simultaneously due to mass cleanups.
- Data corruption risks are very high due to forced shutdown processes at the node OS level.
✓ SOLUTION: Place all critical production stateful workloads on On-Demand type Node pools. Spot Nodes are only used for stateless workloads.

2. Enabling HPA Without Limiting Maximum Autoscaler Capacity #

Setting the maxReplicas parameter on HPAs or max-nodes on Cluster Autoscalers too large without enforcing cost quota limits on the cloud account.

# ANTI-PATTERN: Too-loose HPA maxReplicas
spec:
  minReplicas: 3
  maxReplicas: 200 # DON'T: Potential extreme cost ballooning!

Operational Risks:
- If our application suffers a memory leak loop bug or is attacked by hacker syndicates through DDoS (Distributed Denial of Service) attacks.
- The HPA detects workload increases and keeps adding Pods until the maximum 200-pod limit.
- The Cluster Autoscaler is triggered to keep renting new VMs on the cloud provider to hold those 200 pods.
- Your cloud bill balloons to tens of thousands of dollars in just one night.
✓ SOLUTION: Limit 'maxReplicas' to realistic numbers (e.g. 20-30 pods) and install budget limit alarm systems (Billing Alerts) at the cloud provider portal level.

Cost Optimization Audit Checklist #

Use this FinOps checklist to audit your Kubernetes cluster’s cost efficiency:

COMPUTE USAGE AUDIT (CPU/RAM):
  □ No non-production clusters (dev/test) run 24/7 without automatic shutdown scheduling.
  □ Scale-to-zero features (KEDA/kube-downscaler) are active to save costs on weekends.
  □ PromQL queries are used to identify containers with < 10% utilization of requests.
  □ Idle/under-utilized worker node pool VMs are periodically removed.
  □ The Spot Node portion is configured at a minimum of 50% in non-production environments.

NETWORK & STORAGE OPTIMIZATION:
  □ All microservice applications are consolidated under one Ingress Controller (sharing one Load Balancer).
  □ VPC Gateway Endpoints are enabled for S3/Object Storage access without NAT Gateway costs.
  □ Volume snapshot retention is limited to a maximum of 7-14 days for non-critical test data.
  □ Automatic retention policies are installed on Container Registries to delete stale dev image tags.

COST GOVERNANCE:
  □ Active Kyverno policies are installed to force FinOps label inclusion (team, project, env).
  □ Kubecost or OpenCost is installed for daily cost allocation visualization.
  □ Cloud budget limit notifications (Cloud Billing Alerts) are actively configured.
  □ Cost allocations are delegated to each developer team based on namespace reports.

Summary #

• Map Spending Components — Understand the cluster cost division (Compute, Storage, Network, Management Fees) so you know which areas give the most impactful savings.
• Use Spot Nodes Wisely — Leverage up to 80% discounts with Spot Nodes for stateless apps, CI runners, and dev environments with On-Demand fallbacks.
• Smooth Scale-to-Zero — Apply total shutdown scheduling (0 replicas) in development environments outside working hours to save up to 60% of testing infrastructure bills.
• Consolidate Load Balancers — Avoid creating one physical Load Balancer per Service; use one shared Ingress Controller to cut LB cost spending.
• Install VPC Endpoints — Bypass expensive NAT Gateway data processing costs by routing internal S3 data traffic through VPC Gateway Endpoints.
• Apply Kyverno Label Enforcement — Ensure accurate Kubecost cost tracking by forcing developers to include FinOps labels using Kyverno ClusterPolicies.

← Previous: High Availability   Next: Disaster Recovery →