writing / kubernetes

Stop Using `nodeSelector` for Node Placement

Stop hardcoding infrastructure layouts in your manifests. It creates brittle clusters that fail to scale when your instance types change.

Atharva Uday UndeAtharva Uday UndeSeptember 5, 20267 min read
kuberneteseksdevopsschedulingnode-affinityinfrastructure-as-codecloud-nativeawscontainer-orchestrationsystem-reliability

The "Manual Placement" Trap

Last year, a colleague of mine was struggling with a "Pending" pod status during a routine scale-up event. Our EKS cluster had recently migrated from m5.large to m6i.large instances to take advantage of better price-performance. The deployment manifest for our worker service had a nodeSelector baked into it: instance-type: m5.large.

The scheduler looked at the new m6i.large nodes, saw they lacked the instance-type=m5.large label, and refused to place the pods. Our production traffic spiked, the cluster couldn't scale, and we spent twenty minutes manually patching deployments while customers saw 503 errors.

We were trying to treat a dynamic cloud environment like a static server rack. nodeSelector is a blunt instrument that couples your application configuration to your infrastructure hardware.


Why nodeSelector Is a Maintenance Debt

nodeSelector is the simplest way to constrain pods to specific nodes, but simplicity here is a trap. It relies on labels being perfectly consistent across your entire fleet. When you change instance families, upgrade your node groups, or swap your AMI, you have to find and replace those labels everywhere.

The issues with rigid placement:

  • Infrastructure Fragility: If you delete a node group and recreate it without the exact same labels, your deployments break instantly.
  • Manual Overhead: You are effectively performing manual load balancing by dictating exactly where pods land, rather than letting the scheduler calculate optimal packing.
  • Deployment Drift: It is common to see clusters where old services are still pinned to "deprecated" node labels that don't actually exist, preventing those pods from ever scheduling on newer, cheaper hardware.

If you are using nodeSelector to keep your high-memory workloads away from cheap compute, you are using the wrong tool. You are creating a "walled garden" for your pods that prevents Kubernetes from doing its actual job: abstracting away the underlying hardware.


The Better Approach: Node Affinities and Taints

Instead of forcing a pod to look for a specific hardware label, use nodeAffinity for soft requirements and tolerations with taints for hard constraints. These mechanisms are declarative and allow the scheduler to make smarter decisions.

Moving from nodeSelector to nodeAffinity

nodeSelector is an all-or-nothing binary switch. nodeAffinity lets you express preferences. You can tell Kubernetes: "Prefer these high-memory nodes, but if they are full, put me elsewhere."

affinity:
  nodeAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 100
      preference:
        matchExpressions:
        - key: node.kubernetes.io/instance-type
          operator: In
          values:
          - r6g.xlarge

If you absolutely must separate workloads—for example, keeping third-party agents off your core compute—use taints and tolerations. Taints act as a "keep out" sign, and tolerations allow specific pods to ignore that sign.

Strategy Flexibility Maintenance Best For
nodeSelector None High Never
nodeAffinity High Low Scheduling optimization
Taints/Tolerations Medium Medium Hard isolation

What I Actually Do / My Take

I have banned nodeSelector in our CI/CD templates at Confiance Bizsol.

If I need to separate workloads, I use Taints on the nodes and corresponding Tolerations on the pods. This ensures that no pod accidentally lands on a node it shouldn't be on, but it keeps the infrastructure configuration decoupled from the application logic.

If I'm just trying to optimize performance, I use topologySpreadConstraints. This is the "Gold Standard" for production. It forces the scheduler to balance pods across different Availability Zones and node types automatically. I don't care which node a pod runs on; I care that it has enough CPU and that it isn't colocated with all its replicas in a single zone.

Stop micromanaging the scheduler. If you find yourself needing to force a pod to a specific node, you usually have a deeper issue with your resource requests or your node group sizing.


Closing / TL;DR

nodeSelector creates a hard dependency between your code and your hardware that will break the moment you update your infrastructure. Use nodeAffinity for soft placement preferences and taints/tolerations for hard isolation. Let the scheduler do the heavy lifting—it’s better at math than you are.

Tags: kubernetes · eks · devops · scheduling · node-affinity · infrastructure-as-code · cloud-native · aws · container-orchestration · system-reliability