Helm #

Deploying a production-level microservice application into a Kubernetes cluster isn’t a simple task. One application usually doesn’t just consist of one Deployment manifest, but a set of interdependent bound manifests: Deployment (workload), Service (network access), ConfigMap and Secret (configuration), HorizontalPodAutoscaler (scaling), Ingress (external routing), plus ServiceAccount, Role, and RoleBinding (RBAC). If we manually manage all these YAML manifest files in every environment (development, staging, production), we get trapped in tiring code duplication processes prone to triggering human configuration errors. Helm comes as the standard package manager for Kubernetes, abstracting those manifest sets into one modular unit that can be packaged, version-controlled, and deployed with a single command.

Helm Architecture and Workflow #

Helm works as a client-side tool since Helm 3 was released (removing the server-side Tiller component that had RBAC security hole issues in earlier versions). The Helm Client interacts directly with the Kubernetes API Server using local configuration credentials (kubeconfig).

flowchart TD
    ChartSource["Helm Chart Source Directory (templates/, values.yaml)"] -->|"helm package"| ChartTgz["Chart Archive (.tgz File)"]
    ChartTgz -->|"helm push"| OCIRegistry["OCI Container Registry (ECR / Harbor)"]
    
    subgraph DeployCycle["Deployment Cycle (CI/CD / Developer Machine)"]
        direction TB
        HelmClient["Helm Client CLI"] -->|"Download Chart"| OCIRegistry
        HelmClient -->|"Compile: Templates + values.yaml"| YAMLOutput["YAML Manifest Text (Plain Text)"]
        YAMLOutput -->|"Send HTTP POST Payload"| K8sAPI["Kube-API Server"]
    end
    
    subgraph ClusterStorage["Kubernetes Namespace (State Storage)"]
        direction LR
        K8sResources["Active Resources (Pods, Services, Ingress)"]
        ReleaseSecret["Helm Release History (Stored as Secret in the Target Namespace)"]
    end
    
    K8sAPI --> K8sResources
    K8sAPI --> ReleaseSecret

Release Mechanism and State Storage #

Every time we run the helm install or helm upgrade command, Helm compiles manifest templates with configuration values (values.yaml), then sends the compiled YAML payload to the Kubernetes API Server to be applied.

Helm records the cluster’s release history natively inside Kubernetes. Each release version is stored as a Secret in the target namespace (labeled sh.helm.release.v1.<release-name>.<version>). This information contains the entire snapshot of applied manifests, easing instant rollback processes without needing to trace back Git commit histories in CI/CD pipelines.

Helm Chart Structure Anatomy #

A Helm package is called a Chart. A Chart is a structured directory separating manifest templates from dynamic variable values.

Standard Helm Chart Directory Structure:

my-app/
├── Chart.yaml          # Main chart metadata (name, description, chart version, app version)
├── values.yaml         # Default configuration values (can be overridden at deployment)
├── charts/             # Required sub-charts/dependencies (.tgz files)
├── templates/          # The Kubernetes manifest template file directory
│   ├── _helpers.tpl    # Global template helpers (reusable template define blocks)
│   ├── deployment.yaml # The application Deployment template
│   ├── service.yaml    # The Service template
│   ├── ingress.yaml    # The Ingress template
│   ├── NOTES.txt       # Text scripts automatically printed after successful installs
└── .helmignore         # File patterns ignored when the chart is packaged

Writing Chart.yaml (SemVer Versioning) #

The Chart.yaml file contains the chart’s identity and dependencies. We must clearly separate the chart package version from the application code version.

apiVersion: v2             # Must be v2 for Helm 3 usage
name: billing-service
description: E-commerce microservice payment processing service.
type: application          # 'application' for deployable charts, 'library' for shared helper charts
version: 1.4.2             # Chart Version (follows SemVer - bumped if template structures change)
appVersion: "2.1.0"        # Application Version (reflects the app container image tag)
dependencies:
  - name: redis
    version: "17.3.0"
    repository: "https://charts.bitnami.com/bitnami"
    condition: redis.enabled # Only installed if the redis.enabled parameter is true in values.yaml

The Go Template Engine and the Sprig Function Library #

Helm leverages the Go language’s built-in text template engine (text/template), enriched with string manipulation functions from the Sprig library.

The values.yaml File: Configuration Value Abstraction #

All configurations varying between environments (like dev, staging, prod) must not be hardcoded inside manifest template files. Those configurations must be defined as variables in values.yaml.

# File: values.yaml (Default values)
replicaCount: 2

image:
  repository: registry.company.com/apps/billing
  pullPolicy: IfNotPresent
  tag: "" # If empty, defaults to the appVersion in Chart.yaml

service:
  type: ClusterIP
  port: 8080

resources:
  limits:
    cpu: "500m"
    memory: "512Mi"
  requests:
    cpu: "200m"
    memory: "256Mi"

ingress:
  enabled: false
  hosts:
    - host: billing.company.com
      paths:
        - path: /
          pathType: ImplementationSpecific

Manifest Templating Implementation (templates/deployment.yaml) #

We write manifest templates using the double curly braces {{ }} evaluation syntax.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "billing.fullname" . }}
  namespace: {{ .Release.Namespace }}
  labels:
    # Insert the standard labels defined in _helpers.tpl
    {{- include "billing.labels" . | nindent 4 }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      {{- include "billing.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      labels:
        {{- include "billing.selectorLabels" . | nindent 8 }}
    spec:
      containers:
      - name: {{ .Chart.Name }}
        # Default the tag to appVersion if the tag is empty in values.yaml
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
        imagePullPolicy: {{ .Values.image.pullPolicy }}
        ports:
        - containerPort: 8080
          name: http
        resources:
          # Copy the whole resources YAML block with 10-space indentation
          {{- toYaml .Values.resources | nindent 10 }}

Creating Template Helpers (_helpers.tpl) #

Helper template blocks are used to wrap standard naming and label code so manifest code writing isn’t duplicated.

[!IMPORTANT] The Critical template vs include Difference: Inside the Helm template engine, we must use the include function instead of the built-in template command when calling helper templates. The template command can’t be pipelined into other string manipulation functions like nindent (new indent). Using include lets us set the YAML spacing layout accurately.

# File: templates/_helpers.tpl

{{/*
1. Determine the full application name (Release Name + Chart Name)
*/}}
{{- define "billing.fullname" -}}
{{- printf "%s-%s" .Release.Name .Chart.Name | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{/*
2. Standard labels for monitoring and compliance standardization
*/}}
{{- define "billing.labels" -}}
helm.sh/chart: {{ printf "%s-%s" .Chart.Name .Chart.Version }}
app.kubernetes.io/name: {{ .Chart.Name }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}

{{/*
3. Selector labels for service and pod matching
*/}}
{{- define "billing.selectorLabels" -}}
app.kubernetes.io/name: {{ .Chart.Name }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}

Production-Level Helm Operational Commands #

Here’s a list of Helm command-line (CLI) commands that must be integrated into CI/CD pipelines or used by cluster administrators:

1. Compilation Testing and Dry-Run (Early Detection) #

Before applying changes in production, we can view the local YAML text rendering results without touching the cluster to check YAML spacing syntax.

# Test the manifest rendering locally
helm template billing-release ./my-chart --values values-production.yaml

# Test rendering directly to the API Server in dry-run mode (API schema validation)
helm install billing-release ./my-chart --values values-production.yaml --dry-run

2. Configuration Difference Checking (The Helm Diff Plugin) #

Use the helm-diff plugin to see the exact configuration diff output between manifests currently running in production and new manifests to be deployed.

# Install the diff plugin
helm plugin install https://github.com/databus23/helm-diff

# Compare the changes before doing the upgrade
helm diff upgrade billing-release ./my-chart --values values-production.yaml

3. Safe Release Application (Safe Deployment Flags) #

At the production level, we must configure automatic protection mechanisms so if new containers suffer a crash-loop, the system automatically rolls back to the previous healthy revision.

# Upgrade with automatic protection
helm upgrade billing-release ./my-chart \
  --namespace production \
  --values values-production.yaml \
  --set image.tag=v2.1.2 \
  --wait \                  # Wait until all Pods are Ready
  --timeout 10m0s \         # Limit the maximum wait time to 10 minutes
  --atomic \                # Remove new resources and auto-rollback if the process fails/times out
  --cleanup-on-fail         # Clean up orphaned resources if the upgrade fails

Storing Helm Charts in OCI Registries (Helm 3.8+) #

In the past, distributing Helm Charts required a special repository server with an index.yaml file. Starting from Helm 3.8+, we can treat Helm Charts as OCI objects (OCI Artifacts) and store them in the same modern container registry as our application images (like Harbor, AWS ECR, or GCP Artifact Registry).

# 1. Login to the OCI Container Registry using official credentials
helm registry login registry.company.com -u ci-user -p secret-token

# 2. Package the chart directory into a compressed archive file (.tgz)
helm package ./my-chart
# Produces the file: billing-service-1.4.2.tgz

# 3. Push the archive file to the OCI registry using the oci:// url scheme
helm push billing-service-1.4.2.tgz oci://registry.company.com/helm-charts

To install it again from the OCI registry:

helm install billing-release oci://registry.company.com/helm-charts/billing-service --version 1.4.2 -n production

Helm Chart Management Practice Anti-Patterns #

Avoid the following operational mistakes when building and managing Helm Charts:

1. Manual Post-Deployment Modifications (Ad-Hoc Drifts) #

Directly changing manifests using kubectl edit or kubectl patch commands after the application is deployed via Helm.

Manual Modification Risks:
- The state in the Kubernetes API Server becomes out of sync (*drift*) with the Helm release state.
- When the next 'helm upgrade' process runs, Helm overwrites all those manual changes, causing sudden loss of emergency configuration patches in production.
✓ SOLUTION: All configuration changes must go through values.yaml and be committed to Git (GitOps flow).

2. Storing Plain-Text Secrets in values.yaml #

# ANTI-PATTERN: Storing plain-text database passwords directly in values.yaml
db:
  password: "super-secret-password-123" # DON'T: This file is stored in public/internal Git repositories

# ==============================================================================
# CORRECT: Using external references or SOPS encryption.
# We only store the Secret name reference safely created by external operators (ESO).
db:
  secretName: billing-database-credentials

Production Helm Chart Audit Checklist #

Use the following checklist before releasing your Helm Chart to the central repository:

STRUCTURE & CODE LAYOUT:
  □ The Chart.yaml file defines the chart version (version) and application version (appVersion) separately.
  □ Chart versions are consistently bumped following Semantic Versioning (SemVer) rules.
  □ Dynamic variables aren't hardcoded in the templates folder; they're declared in values.yaml.
  □ Helper templates (_helpers.tpl) are used to standardize global names and labels.
  □ Helper template usage is called using the 'include' function, not the 'template' command.

SECURITY & OPERATIONAL PROTECTION:
  □ Secret values aren't stored as plain text in the values.yaml file.
  □ CI/CD pipelines use 'helm upgrade' with the '--atomic' and '--wait' flags.
  □ The 'helm-diff' plugin is used in pipelines to validate changes before deployments.
  □ Helm Charts are packaged and distributed using the official OCI Registry.
  □ Manifest changes are never done using kubectl edit directly in production.

Summary #

  • Helm Manages Manifest Complexity — Abstract Kubernetes manifest sets into one modular package (Chart) to ease replication across environments (Dev/Prod).
  • Use include for Spacing Layout — Always use the include function to call helper templates in _helpers.tpl so nindent YAML indentation formats work perfectly.
  • Enable –atomic Protection — Make sure the --atomic, --wait, and --timeout flags are always active for production releases so Helm triggers auto-rollback if new pods fail to start.
  • Separate Chart Version and App Version — Understand the difference; version represents Helm template file changes, while appVersion represents the application container image tag.
  • Apply OCI Registries for Charts — Eliminate extra chart repository infrastructure; store .tgz Helm archives natively inside OCI Container Registries.
  • Avoid Post-Deploy Ad-Hoc Drift — All cluster manifest modifications must be declared through values.yaml and manual kubectl edit is strictly forbidden in production.

← Previous: Observability Anti-Patterns   Next: Kustomize →

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact