Stop Over-Provisioning Your K8s Requests and Limits

The "Safety Margin" Trap

Last year, I audited a client’s EKS cluster that was costing them $12,000 a month. Their infrastructure team had a simple rule of thumb: "Take the peak memory usage, multiply by two, and set that as the limit."

They thought they were being diligent. They were actually just paying AWS for the privilege of keeping their servers 70% idle.

When you define a Kubernetes Request and Limit, you are setting a reservation. The scheduler uses the Request to decide where to place your pod; the kernel uses the Limit to enforce a hard ceiling. If you set these based on "gut feel" or "safety margins," you aren't building a resilient system—you’re creating a massive, expensive mismatch between your provisioned capacity and your actual workload.

The Cost of Garbage Numbers

When you over-provision, the Kubernetes scheduler is tricked into believing your nodes are full. It forces the autoscaler (like Karpenter or the Cluster Autoscaler) to spin up new EC2 instances that are largely empty.

I checked the node-exporter metrics for that $12k/month cluster. The average CPU utilization across the fleet was hovering at 14%. We were running a fleet of m6i.xlarge instances that were mostly doing nothing except idling in a cgroup.

If you set your requests too high, you force fragmentation. If you set your limits too low, you hit OOMKills. The middle ground is found through telemetry, not guesswork.

How to Audit and Right-Size

You cannot optimize what you do not measure. Before you touch a single YAML file, look at the actual consumption over a 7-day period.

Use Vertical Pod Autoscaler (VPA) in Recommendation Mode

Don't let VPA automatically restart your pods in production—that’s a recipe for instability. Instead, deploy it in Recommendation mode. It looks at the historical usage of your pods and tells you exactly what the requests should have been.

# Check the recommendations for a specific deployment
kubectl get vpa my-app-vpa -o yaml

The output gives you lowerBound, target, and upperBound. Ignore the upperBound—that's for spikes that rarely happen. Use the target value as your starting point for your Request.

Focus on the P95

Never optimize for the absolute maximum spike. If your app spikes to 4GB of RAM once a month for a background job, do not set your Limit to 4GB. That is an architectural problem, not an infrastructure one. Handle that with horizontal scaling or a smarter job queue, not by throwing memory at a container that only needs 512MB for 99% of its lifecycle.

What I Actually Do

I follow a strict "No-Guessing" protocol for every service I manage:

1. Baseline with VPA: I run VPA in recommendation mode for 7 days. I collect the target values.
2. Set Requests to Average: I set the Resource.Request to the P90 of the average usage. This ensures the scheduler has enough headroom to place the pod without wasting space.
3. Set Limits with a Buffer: I set the Resource.Limit to 1.5x the P99 usage. This is my "oh-no" room for memory leaks or unexpected traffic bursts.
4. Iterate: If the pod hits the limit, I check the logs. If it's a genuine growth in usage, I bump the limit. If it's a spike, I investigate the application code.

My deployment.yaml snippets always look like this:

resources:
  requests:
    cpu: "200m"
    memory: "256Mi"
  limits:
    cpu: "500m"
    memory: "512Mi"

I never leave limits empty. If you don't set a limit, a single memory leak in your app will consume the entire node’s RAM, eventually triggering the OOM Killer to wipe out every other pod on that node. A capped process is a localized failure; an uncapped process is a cluster-wide outage.

TL;DR

Over-provisioning is a tax on your engineering budget. Use VerticalPodAutoscaler to generate real numbers, set your requests to match your actual P90 usage, and always—always—set a hard limit to prevent rogue processes from taking down your entire node. Stop guessing, start measuring, and get your utilization above 40%.

Kubernetes · Cloud Cost Optimization · EKS · Infrastructure Efficiency · Vertical Pod Autoscaler · DevOps · Resource Management · Cloud Economics · Capacity Planning · SRE