Ingress Controller Comparison #

In the Kubernetes ecosystem, the Ingress object acts as a unified entry gateway receiving all external HTTP/HTTPS query traffic into the cluster, then distributing it to the right internal services based on domain or URL path query rules. However, as we know, the Ingress object is just a static configuration file stored in the etcd database. We must install an active Ingress Controller in the cluster so those routing rules are actually executed.

Choosing the right Ingress Controller is one of the most important early architectural decisions with long-term impact on our cluster infrastructure’s stability and cost efficiency. Replacing an Ingress Controller on a running production cluster (live cluster) isn’t a trivial task. We’re forced to rewrite all Ingress manifests, convert dozens of controller-specific annotations, and run re-routing tests that risk downtime.

This article presents an in-depth comparison of the most popular Ingress Controllers in the industry today, dissects each one’s technical characteristics, provides a decision tree guide, and demonstrates how to run several controllers side by side (multi-controller configuration).

Popular Ingress Controller Characteristics Analysis #

Let’s dissect the internal architecture, strengths, and weaknesses of the five most widely used Ingress Controller candidates in production today:

flowchart TD
    Options["INGRESS CONTROLLER OPTIONS"]
    Options --> InCluster["In-Cluster Proxy (NGINX Ingress - Community)"]
    Options --> HotReload["Hot-Reload Proxy (Traefik - Dynamic config)"]
    Options --> OutCluster["Out-of-Cluster LB (AWS Load Balancer Controller)"]

1. NGINX Ingress Controller (kubernetes/ingress-nginx) #

This is the official Ingress Controller developed directly by the Kubernetes community. This controller uses NGINX as the main reverse proxy engine running inside our cluster Pods.

• How It Works: kube-proxy directs external traffic to the NGINX Pod. The NGINX Ingress Controller watches the API Server. Every time an Ingress object changes, the controller rewrites its internal nginx.conf template configuration file, then triggers a graceful NGINX reload process.
• Advantages: Most mature, the world’s largest community, abundant documentation, and manifest examples are very easy to find. NGINX supports various advanced features via annotations (rate limiting, basic auth, CORS, URL rewriting).
• Disadvantages: Although the NGINX reload process is graceful, on large-scale clusters with hundreds of Ingress changes per minute (due to autoscaling or dynamic deployments), chained reloads can trigger high controller CPU usage and minor pauses in new connection establishment.
• Best Use Cases: General on-premise cluster scenarios, testing/staging clusters, or general production clusters where operations teams are already very familiar with traditional NGINX reverse proxy configuration.

2. Traefik Ingress #

Traefik is a modern reverse proxy written in Go, designed from the start to manage very dynamic cloud-native architectures.

• How It Works: Traefik interacts directly with the Kubernetes API. Traefik implements Hot Reload technology (dynamic configuration updates directly in memory) without ever triggering a process restart or reload.
• Advantages: Absolute zero-downtime during routing configuration changes. Traefik provides a very intuitive internal graphical dashboard for monitoring route status. Traefik also natively supports automatic Let’s Encrypt integration and has its own Custom Resource Definition (CRD) object called IngressRoute that’s type-safe.
• Disadvantages: Slightly higher RAM memory footprint than NGINX when idle. Peak throughput performance is a bit below NGINX/HAProxy for very dense pure HTTP queries.
• Best Use Cases: Clusters with dynamic microservices architectures that very frequently update routes, clusters needing visual traffic monitoring dashboards, and teams wanting to avoid typo-prone NGINX annotation writing.

3. HAProxy Ingress #

HAProxy is famous in the industry as one of the fastest and most memory-efficient Layer 4 and Layer 7 load balancer software.

• How It Works: Leverages the HAProxy Runtime API to change backend configuration dynamically in memory without triggering an HAProxy reload process.
• Advantages: Very consistent sub-millisecond latency, the best user session stability handling (session persistence/sticky cookies), and the highest queries-per-second (QPS) throughput among all in-cluster proxies.
• Disadvantages: The user community in the Kubernetes ecosystem is relatively small compared to NGINX and Traefik. Manifest examples and third-party integrations (like cert-manager) require slightly more complicated manual configuration.
• Best Use Cases: Financial or e-commerce workloads with very dense traffic needing ultra-low latency and strict sticky session stability.

4. AWS Load Balancer Controller (AWS ALB) #

Unlike NGINX or Traefik which run as reverse proxies inside the cluster (in-cluster proxy), the AWS Load Balancer Controller acts as a control operator managing the creation of physical Application Load Balancers (ALBs) outside the cluster (in our AWS account).

Let’s look at the data traffic flow difference below:

In-Cluster Proxy (e.g. NGINX Ingress):
  Client ──> [Cloud Load Balancer (L4)] ──> [NGINX Pod (L7 In-Cluster)] ──> [App Pod]
                                            (Extra proxy hop inside the Node)

Out-of-Cluster Controller (AWS ALB):
  Client ──> [AWS Application Load Balancer (External L7)] ───────────────> [App Pod]
                                            (Packets go directly to Pods via the AWS Target Group)

• How It Works: Every time we create an Ingress Resource, this controller calls the AWS API to create one physical ALB unit. The ALB directly registers our application Pod IPs into an AWS Target Group. External traffic enters the external cloud ALB, then flows directly to the Pod’s virtual network card (AWS VPC CNI target group) without passing through an additional proxy layer inside the cluster.
• Advantages: Outstanding native integration with AWS security features (AWS WAF for web application firewalls, AWS Shield for DDoS protection, and ACM for automatic SSL certificates). No extra RAM/CPU consumption inside cluster worker nodes for reverse proxy processing.
• Disadvantages: Vendor-locked (only runs on AWS). Every external physical ALB creation takes minutes to provision and triggers significant AWS bill increases if not configured with the IngressGroup feature.
• Best Use Cases: AWS EKS production clusters with strict security audit requirements (must be protected by AWS WAF) wanting to optimize latency by trimming the in-cluster proxy layer.

Controller Selection Decision Tree #

To make choosing easier, let’s follow this decision flow:

flowchart TD
    Start["Where is the Kubernetes cluster running?"] --> Cloud{"Public Cloud or On-Premise?"}
    
    Cloud -- "AWS EKS" --> AWSRequirement{"Need AWS WAF / Shield for compliance?"}
    AWSRequirement -- "Yes" --> AWSALB["Use the AWS Load Balancer Controller (ALB)"]
    AWSRequirement -- "No" --> Performance{"Frequent rolling updates / route config changes?"}
    
    Cloud -- "GCP GKE" --> GKECDN{"Need Google Cloud CDN / Armor?"}
    GKECDN -- "Yes" --> GKEIngress["Use the built-in GKE Ingress"]
    GKECDN -- "No" --> Performance
    
    Cloud -- "On-Premise / Bare-Metal" --> Performance
    
    Performance -- "Yes (Dynamic)" --> Traefik["Use Traefik Ingress (IngressRoute CRD)"]
    Performance -- "No (Static)" --> CoreChoice{"Need maximum throughput & consistent latency?"}
    
    CoreChoice -- "Yes (Ultra Perf)" --> HAProxy["Use HAProxy Ingress"]
    CoreChoice -- "No (General)" --> Nginx["Use the NGINX Ingress Controller (kubernetes/ingress-nginx)"]

Running Multi-Ingress Controllers Side by Side #

On large-scale production clusters, we often need more than one Ingress Controller at the same time. For example, we deploy the NGINX Ingress Controller for internal traffic between developer teams, and the AWS ALB Controller for public traffic needing AWS WAF protection.

To facilitate this scenario, we use the IngressClass object to register and separate each controller.

1. Registering IngressClasses #

First, we register the nginx-internal and aws-alb-public classes to the Kubernetes API Server:

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: nginx-internal
  annotations:
    ingressclass.kubernetes.io/is-default-class: "true" # Make this class the default if the spec is empty
spec:
  controller: k8s.io/ingress-nginx # Points to the NGINX driver
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: aws-alb-public
spec:
  controller: ingress.k8s.aws/alb # Points to the AWS Load Balancer Controller driver

2. Directing Ingress Resources to the Right Controller #

Developers just write the ingressClassName property inside their Ingress Resource manifests to choose which gateway they want to use:

# Ingress A: Using Internal NGINX
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: internal-api-route
  namespace: production
spec:
  ingressClassName: nginx-internal # Chooses the NGINX gateway
  rules:
    - host: internal.mycompany.local
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: user-service
                port:
                  number: 80
---
# Ingress B: Using Public AWS ALB
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: public-web-route
  namespace: production
spec:
  ingressClassName: aws-alb-public # Chooses the AWS ALB gateway
  rules:
    - host: www.mycompany.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: landing-page-service
                port:
                  number: 80

Ingress Controller Selection Anti-Patterns vs Solutions #

Let’s study some fatal mistakes related to Ingress Controller deployment in production clusters, along with their code comparisons.

Anti-Pattern 1: Ignoring ALB Grouping (IngressGroup) on AWS EKS #

We create 15 different Ingress Resource manifests to route 15 microservices on AWS EKS using the AWS Load Balancer Controller without including grouping parameters.

Wrong Manifest Code (One Ingress = One New Physical ALB) #

# DON'T DO THIS: Wastes cloud provider costs
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: service-billing-ingress
  namespace: production
spec:
  ingressClassName: aws-alb-public
  rules:
    - host: app.mycompany.com
      http:
        paths:
          - path: /billing
            pathType: Prefix
            backend:
              service:
                name: billing-service
                port:
                  number: 80

The Bad Consequences #

The AWS Load Balancer Controller triggers the creation of 15 separate physical Application Load Balancer (ALB) units in our AWS account. This causes a useless monthly AWS bill spike of hundreds of dollars, because we’re forced to pay rental costs for 15 external physical load balancers that each only handle a small amount of traffic.

Solution Code (Using IngressGroup to Share One ALB) #

We must add the alb.ingress.kubernetes.io/group.name annotation to every Ingress manifest. This instructs the AWS Controller to merge the routing rules from all Ingresses into one single physical ALB unit to save cloud spending.

# SOLUTION: Grouping many Ingresses into one physical ALB
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: service-billing-ingress
  namespace: production
  annotations:
    alb.ingress.kubernetes.io/group.name: "production-shared-alb" # The group name must be the same across Ingresses
spec:
  ingressClassName: aws-alb-public
  rules:
    - host: app.mycompany.com
      http:
        paths:
          - path: /billing
            pathType: Prefix
            backend:
              service:
                name: billing-service
                port:
                  number: 80

Anti-Pattern 2: Writing Dozens of Complex Annotations on Standard Ingresses #

We use the standard Kubernetes Ingress, but need very complicated routing configuration (like custom CORS validation, dynamic HTTP header additions, per-IP rate limiting, and URL redirection). We write all this logic in the annotations block.

Wrong Manifest Code (Annotations Hell on a Standard Ingress) #

# DON'T DO THIS: Manifests get messy and prone to typos
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: messy-ingress
  namespace: production
  annotations:
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS"
    nginx.ingress.kubernetes.io/limit-connections: "5"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "15"
    # Dozens of unsafe annotation lines without YAML syntax validation
spec:
  ingressClassName: nginx
  # ... rules ...

The Bad Consequences #

Annotations in Kubernetes are just free untyped raw string data. The Kubernetes API Server can’t do data type validation (type safety) on annotation values. If we mistype one letter in an annotation name (e.g. the typo cors-allow-methdos), that rule is silently ignored by the controller without any error message, triggering hard-to-trace security holes or configuration failures.

Solution Code (Using Traefik’s Type-Safe IngressRoute CRD) #

If we use Traefik Ingress, we’re advised to use their built-in Custom Resource Definition (CRD) object, IngressRoute. This object replaces standard annotations with a type-safe declarative code structure validated directly by the API Server when applied.

# SOLUTION: Clean, validated configuration with the Traefik IngressRoute CRD
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: clean-ingress-route
  namespace: production
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`app.mycompany.com`) && PathPrefix(`/billing`)
      kind: Rule
      services:
        - name: billing-service
          port: 80
      middlewares:
        - name: secure-headers-middleware # Points to a separate structured middleware
        - name: strict-ratelimit-middleware

Summary #

• NGINX Ingress is the common standard: Very mature with the largest community; but its config reload process can have minor impacts on ultra-giant-scale clusters.
• Traefik excels in dynamism: Implements hot-reload without process restarts for zero-downtime, has an internal visual dashboard, and supports the type-safe IngressRoute CRD.
• HAProxy for ultra-low latency: Provides the highest throughput and best sticky session handling; but its Kubernetes user community is relatively smaller.
• AWS ALB Controller trims the in-cluster proxy: Integrates Ingress directly with physical AWS ALB VPC CNI target groups; saves worker node RAM/CPU.
• Must use IngressGroup on AWS: Group our Ingress objects using the group.name parameter to avoid cloud bill bloat from separate physical ALB creation.
• Use IngressClass for multi-controller: Register separate IngressClass objects to regularly split traffic (e.g. separating the internal NGINX gateway from the public ALB gateway).

← Previous: Service Mesh   Next: Network Troubleshooting →