# How to avoid costly instance selection mistakes in Karpenter

By Ido Slonimsky

Tech Lead, Zesty

Your Karpenter configuration can make or break your cluster’s efficiency. Configure it poorly and you risk locking yourself into expensive nodes that never consolidate.

I have seen clusters where Karpenter was installed correctly, yet costs increased, and the root cause was almost always the same: NodePool design.

The key is to configure it widely enough to give it the breadth to make the best decisions for you. This guide provides steps to design NodePools that give Karpenter enough flexibility to optimize while protecting you from costly mistakes.

* * *

## **First, Understand What Karpenter Is Optimizing**

Before tuning anything, clarify how Karpenter works.

### **What Karpenter Actually Does**

Karpenter watches for unschedulable pods and provisions nodes that satisfy:

- Pod resource requests
- Node affinity and anti-affinity
- Taints and tolerations
- Topology constraints
- NodePool requirements

It chooses from available AWS instance types that match your NodePool constraints. It also continuously evaluates whether nodes can be consolidated to reduce cost.

Consolidation works by:

1. Attempting to reschedule pods from one node to others.
2. Terminating the empty or underutilized node if possible.

AWS provides hundreds of instance type combinations across families, sizes, architectures, and pricing models. If you restrict Karpenter too tightly, you reduce its search space. If you leave it completely open, you may get instance types that do not align with your financial or operational goals.

### **Prerequisites**

Before applying the steps in this guide, ensure:

- You are running Karpenter v1 or later.
- IAM roles and instance profiles are configured for Karpenter.
- You have permissions to create or modify NodePools.
- Consolidation is enabled in your Karpenter configuration.
- You have visibility into cost data through AWS Cost Explorer or similar.

Checkpoint:

```shell
kubectl get nodepools
```

You should see your active NodePools.

Now that we understand the decision model, let’s design the constraint matrix properly.

* * *

## **Treat NodePools as a Constraint Matrix**

A NodePool defines the boundaries of what Karpenter is allowed to provision.

It is effectively a matrix that may include:

- Instance families
- Instance sizes
- Architecture
- Capacity type such as spot or on-demand
- Availability zones
- Custom requirements

Here is a simplified example:

```yaml
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general-workloads
spec:
  template:
    spec:
      requirements:
        - key: karpenter.k8s.aws/instance-family
          operator: In
          values: ["m6i", "m6a", "c6i", "c6a"]
        - key: karpenter.k8s.aws/instance-size
          operator: In
          values: ["large", "xlarge", "2xlarge"]
        - key: karpenter.k8s.aws/capacity-type
          operator: In
          values: ["spot"]
```

This definition gives Karpenter several instance families and multiple sizes. That flexibility is critical for cost optimization.

According to AWS documentation, broader instance diversification improves spot availability and price stability. AWS recommends diversified capacity pools to reduce interruption risk. The same principle benefits Karpenter’s optimization logic.

### **Action Step: Broaden Intelligently**

Review your existing NodePools:

- Are you limiting to a single instance family?
- Are you using only one size?
- Are you restricting to a single availability zone?

If yes, expand gradually.

Checkpoint:

After applying changes:

```shell
kubectl apply -f nodepool.yaml
kubectl describe nodepool general-workloads
```

Verify the requirements section reflects multiple families and sizes.

* * *

## **Avoid the Silent Killer: Over-Restricting Instance Types**

Many teams unintentionally restrict Karpenter too much.

Common examples:

- Only allowing one instance family because it was used historically.
- Hardcoding a single size such as m6i.2xlarge.
- Avoiding spot entirely out of caution.

This reduces Karpenter’s ability to:

- Bin-pack workloads efficiently.
- Replace nodes with cheaper alternatives.
- Consolidate across a diverse fleet.

If a pod requests 2 vCPU and 4 GiB, there are dozens of valid instance types. If you only allow one, you force Karpenter into suboptimal choices.

### **Action Step: Expand Instance Diversity**

Instead of:

```yaml
values: ["m6i"]
```

Use:

```yaml
values: ["m6i", "m6a", "m5", "c6i", "c6a"]
```

Ensure the families are appropriate for your workload profile. For general workloads, mixing compute and memory balanced families increases scheduling flexibility.

Expected Outcome:

- New nodes may come from different families.
- Spot interruption rates may decrease.
- Consolidation events should increase over time.

Monitor:

```shell
kubectl logs -n kube-system deploy/karpenter
```

Look for consolidation and provisioning events.

Now let’s address the opposite risk, which is equally dangerous.

Overpaying for CPU and memory you don’t use?  
Rightsize pods to reduce spend without risking performance.

Learn how Zesty automatically aligns resources with real-time demand while preventing CPU throttling and OOM issues.

[Read More](/content/platform/pod-rightsizing/index.html)

* * *

## **Add Guardrails Before Large Instances Burn Your Budget**

Allowing every possible instance type can lead to expensive mistakes.

### **The Large Node Trap**

Consider a 128 vCPU instance. If a single non-evictable pod lands on it, Karpenter may be unable to consolidate that node later.

You end up paying for an oversized node hosting a tiny workload.

This is a common cost trap.

### **The Bare Metal Surprise**

Bare metal instances are significantly more expensive. If not explicitly excluded, Karpenter may consider them valid.

### **Action Step: Cap Maximum Size**

Limit instance sizes to a reasonable upper bound:

```yaml
- key: karpenter.k8s.aws/instance-size
  operator: In
  values: ["large", "xlarge", "2xlarge", "4xlarge"]
```

Avoid including extreme sizes unless required.

To exclude specific categories:

```yaml
- key: karpenter.k8s.aws/instance-category
  operator: NotIn
  values: ["metal"]
```

Checkpoint:

After deployment, confirm no oversized nodes are launched:

```shell
kubectl get nodes -o wide
```

Validate instance types via:

```shell
kubectl get nodes -L node.kubernetes.io/instance-type
```

With safe guardrails in place, we can design for resilience without sacrificing cost control.

* * *

## **Design Tiered NodePools for Resilience and Cost Discipline**

Workloads are not limited to a single NodePool. A pod can match multiple pools if requirements align.

This allows a tiered strategy.

### **Primary NodePool**

- Spot
- Savings Plan aligned
- Broad but controlled instance diversity

### **Secondary Fallback NodePool**

- On-demand
- Slightly broader compatibility
- Still capped at reasonable sizes

Example fallback:

```yaml
- key: karpenter.k8s.aws/capacity-type
  operator: In
  values: ["on-demand"]
```

This ensures that if spot capacity is unavailable or blocked by anti-affinity rules, workloads still schedule.

Expected Behavior:

- Most nodes come from the primary pool.
- Fallback activates only under constraints.
- No runaway provisioning of extreme instance types.

Monitor distribution:

```shell
kubectl get nodes -L karpenter.sh/nodepool
```

You should see most nodes tied to the primary pool.

* * *

## **Rightsize Pods to Unlock Consolidation**

Karpenter can only optimize based on pod requests.

If your pods request 4 vCPU but use 0.5 vCPU, you are blocking consolidation.

Kubernetes scheduling is request-driven, not usage-driven.

According to Kubernetes documentation, resource requests determine bin-packing decisions. Overstated requests directly reduce scheduling density.

### **Action Step: Audit Resource Requests**

Identify overprovisioned workloads:

```shell
kubectl top pods --all-namespaces
```

Compare usage versus requests:

```shell
kubectl get pod <pod-name> -o yaml | grep resources -A 5
```

Adjust deployment specs:

```yaml
resources:
  requests:
    cpu: "500m"
    memory: "512Mi"
  limits:
    cpu: "1"
    memory: "1Gi"
```

Expected Outcome:

- More pods fit per node.
- Consolidation events increase.
- Node count gradually decreases.

If you use automated rightsizing tools, ensure changes are applied continuously. Karpenter benefits from smaller, accurate requests.

* * *

## **Verify That Consolidation Is Actually Working**

Do not assume savings. Verify them.

### **Check Consolidation Activity**

```shell
kubectl logs -n kube-system deploy/karpenter | grep Consolidation
```

You should see nodes being evaluated and occasionally terminated.

### **Check Cost Trends**

Use AWS Cost Explorer:

- Filter by EC2.
- Compare before and after NodePool changes.
- Monitor instance family distribution.

### **Healthy Signals**

- Diverse instance types in use.
- Gradual node count reduction under steady workload.
- No persistent oversized nodes with minimal utilization.

If consolidation rarely occurs, investigate:

- Overprovisioned pods.
- Pod disruption budgets blocking eviction.
- Anti-affinity rules preventing rescheduling.

This final section ties everything together.

## How Zesty Strengthens Your Karpenter Strategy

Everything we covered so far assumes your pod requests reflect real usage. In most clusters, they do not. Teams overprovision CPU and memory to stay on the safe side, and Kubernetes schedules strictly on those requests. When pods ask for more than they use, Karpenter is forced to provision larger nodes and consolidation becomes harder.

Zesty [continuously rightsizes pod requests based on actual usage patterns](/content/platform/pod-rightsizing/index.html). With accurate requests, more pods fit per node, consolidation succeeds more often, and large underutilized instances are easier to remove. If you are investing in well-designed NodePools, automated rightsizing ensures Karpenter can fully capitalize on that design.

* * *

## **Putting It All Into Practice**

To avoid costly instance selection mistakes in Karpenter:

1. Give Karpenter broad instance diversity.
2. Exclude extreme or unnecessary instance categories.
3. Cap maximum instance sizes.
4. Implement tiered NodePools.
5. Continuously rightsize pods.
6. Monitor consolidation and cost trends.

Cost optimization in Kubernetes is iterative. You refine constraints, observe behavior, and adjust. When NodePools are thoughtfully designed and workloads are properly sized, Karpenter becomes a reliable cost optimization engine instead of a hidden cost amplifier.

Next steps:

- Experiment in a staging cluster.
- Introduce spot capacity gradually.
- Review PodDisruptionBudgets for consolidation blockers.
- Track savings over 30 day intervals to capture meaningful trends.

With disciplined NodePool design and rightsized workloads, you can safely give Karpenter the flexibility it needs to do its job well.