Stop Using `latest` Tags in Your Docker Images

The "Latest" Illusion

I walked into a production incident last year where a service started failing after a routine pod restart. The CI/CD pipeline hadn't run in hours, yet the container that pulled fresh on restart was fundamentally different from the one that had been running perfectly for a week.

The culprit? The deployment manifest was pulling an image tagged myapp:latest.

When the developer pushed a "quick fix" to the dev branch earlier that morning, the build system tagged it as latest. When the production pod restarted due to a node drain, the Kubelet dutifully pulled the "newest" version of the image. The production environment was running code that hadn't passed staging, all because we treated a mutable tag as a stable pointer.

Why Mutable Tags Kill Reliability

The latest tag is not a version. It is a moving target. In a distributed system, consistency is the bedrock of stability. If your nodes have different caches or if your deployment strategy involves rolling updates, latest guarantees you will eventually experience a state of "split-brain" where different pods are running different binary versions.

The Hidden Costs of Mutability

• Impossible Rollbacks: If a deployment fails, kubectl rollout undo is useless. If the image registry has been overwritten, you have no way to point back to the specific binary that worked five minutes ago.
• Non-Deterministic Builds: Your infrastructure should be immutable. If you run docker pull on the same tag twice, you should get the same bits. latest makes this impossible.
• Caching Collisions: Docker’s image layer caching relies on digests. When you constantly push to the same tag, you lose the ability to track exactly what changed between iterations.

The Immutable Alternative: Semantic Versioning and Hashes

Infrastructure should be boring and predictable. If I am looking at a Kubernetes manifest at 3:00 AM during an outage, I want to know exactly what code is running.

The Strategy

1. Semantic Versioning: Use v1.2.4. It tells a story.
2. Git SHA: Use the short commit hash (e.g., myapp:a1b2c3d). It maps directly to the source code.
3. Never Overwrite: Once a tag is pushed, it is immutable. If you need a change, you increment the version or push a new hash.

Here is how I structure my CI/CD output in GitHub Actions:

- name: Build and Push
  run: |
    IMAGE_TAG=${{ github.sha }}
    docker build -t mycompany/api:$IMAGE_TAG .
    docker push mycompany/api:$IMAGE_TAG

In your Kubernetes deployment.yaml, you replace the vague latest with a concrete identifier:

spec:
  containers:
    - name: api
      image: mycompany/api:a1b2c3d  # No ambiguity here

If you really need a "current" pointer for documentation or manual testing, use a separate alias tag like staging-stable or production-current, but keep your production deployment manifests pinned to a specific, immutable string.

What I Actually Do

I treat image tags like I treat database migrations: once they are applied, they are set in stone. In our production environment at Confiance Bizsol, we use a combination of semantic versioning and the CI build number.

I never let an engineer manually tag latest. I enforce this via our CI/CD pipelines. If a workflow tries to push to latest, the pipeline fails. This might seem like extra friction, but it eliminates the "why did the app change?" conversation entirely. When a pod crashes, I check the image tag, run git checkout <tag>, and I am looking at the exact code that was running in production.

If you are currently using latest in production, stop today. Update your deployment pipeline to tag by commit SHA. It is a one-line change in your build script that saves you dozens of hours of "ghost" bug hunting.

Closing / TL;DR

latest is a lie. It is a placeholder that invites chaos into your cluster. Always tag your images with a unique identifier—like a Git SHA or a semantic version number—and make that tag immutable in your registry. If you can’t look at your running pod and know exactly which commit it corresponds to, you don’t have a deployment pipeline; you have a gamble.

Pin your versions. Sleep better.

Tags: docker · kubernetes · cicd · devops · infrastructure · software-engineering · containerization · deployment · git · reliability · production-readiness