Kubernetes Autoscaling skill for Codex

Autoscaling works when requests reflect real usage, the scaling signal leads the bottleneck instead of trailing it, scale-up is fast, scale-down is calm, and nodes appear before pods wait long. Fix sizing before adding autoscalers; an HPA on wrong requests scales on fiction.

Inspect first

1. Read AGENTS.md; find where Deployments, HPAs and node pools are defined (Helm values, Kustomize, Terraform for node groups).
2. What already scales: kubectl get hpa,vpa,scaledobject -A (CRDs may be absent).
3. Is metrics-server running (kubectl top nodes)? Prometheus Adapter or KEDA installed? Cluster Autoscaler or Karpenter? Managed options (GKE Autopilot, EKS Auto Mode, AKS node autoprovisioning) change what you control.
4. Gather a week of data per workload if possible: CPU and memory p50/p95/p99 against requests, request rate, latency, queue depth. Without data, say the first numbers are estimates.

Cluster commands need credentials and network; offline, work from manifests and ask the user for kubectl top and metrics output.

Order of work

- [ ] 1. Right-size requests from observed usage (VPA in Off mode helps)
- [ ] 2. Pick the scaling signal per workload (table below)
- [ ] 3. Add HPA or KEDA with explicit behavior and sane min/max
- [ ] 4. Make sure nodes can follow (Cluster Autoscaler / Karpenter, PDBs, priorities)
- [ ] 5. Load test: watch replicas, latency, pending pods, node count
- [ ] 6. Write down thresholds and the reasoning in the manifest or README

Right-sizing

Resource	Request	Limit
Memory	p99 working set over a representative week plus 20-30 percent	Equal to request
CPU	Around p90-p95 of usage for steady services	Usually none for latency-sensitive services

VPA as a recommender, with no evictions:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: orders
spec:
  targetRef: { apiVersion: apps/v1, kind: Deployment, name: orders }
  updatePolicy:
    updateMode: "Off"
  resourcePolicy:
    containerPolicies:
      - containerName: "*"
        controlledResources: ["cpu", "memory"]

Read kubectl describe vpa orders (Target, Lower Bound, Upper Bound) after several days and copy values into the manifest by hand. Recreate and Auto modes evict pods to apply changes; newer VPA releases add InPlaceOrRecreate on clusters with in-place pod resize (1.33+). Never let VPA and HPA both act on CPU or memory for the same workload: they fight. VPA on memory plus HPA on a custom or external metric is fine.

Choosing the signal

Workload	Signal	Tool
Stateless HTTP, CPU-bound	CPU utilisation of requests, target 60-70 percent	HPA
HTTP, I/O-bound (waits on DB, APIs)	Requests per second per pod or in-flight requests	HPA with custom metric (Prometheus Adapter) or KEDA prometheus trigger
Queue consumers	Queue length or lag per replica	KEDA (SQS, RabbitMQ, Kafka, Redis, Pub/Sub scalers)
Jobs that can idle at zero	Events present / absent	KEDA ScaledObject minReplicaCount: 0 or ScaledJob
Memory-bound	Usually not a good HPA signal (memory rarely drops after scale-out)	Right-size, scale on throughput instead
Scheduled peaks	Time of day	KEDA cron trigger alongside the load trigger

HPA

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: orders
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: orders }
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: { type: Utilization, averageUtilization: 65 }
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - { type: Percent, value: 100, periodSeconds: 30 }
        - { type: Pods, value: 4, periodSeconds: 30 }
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - { type: Percent, value: 20, periodSeconds: 60 }

• Utilisation is measured against requests. No CPU request means the HPA reports <unknown> and never scales.
• Every container in the pod counts toward the average, sidecars included. Use type: ContainerResource to target only the app container.
• Remove replicas from the Deployment (or Helm template) once an HPA owns it, or each apply resets the count. In Argo CD, also ignore /spec/replicas differences.
• minReplicas at least 2 (3 across zones) for anything user-facing.
• maxReplicas must fit the node budget and downstream limits (database connections: replicas x pool size must stay under the DB’s max connections).
• Slow-starting apps need readiness and startup probes tuned, or the HPA counts not-ready pods and overshoots.

KEDA

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: invoice-worker
spec:
  scaleTargetRef: { name: invoice-worker }
  minReplicaCount: 0
  maxReplicaCount: 50
  pollingInterval: 15
  cooldownPeriod: 300
  triggers:
    - type: aws-sqs-queue
      authenticationRef: { name: keda-aws }       # TriggerAuthentication with pod identity
      metadata:
        queueURL: https://sqs.eu-west-1.amazonaws.com/123456789012/invoices
        queueLength: "20"                         # target messages per replica
        awsRegion: eu-west-1

• KEDA creates and manages its own HPA; do not create a second HPA for the same target.
• Scale to zero means cold starts: the first message waits for scheduling, image pull and boot. Keep minReplicaCount: 1 when latency matters.
• Workers must finish or hand back in-flight messages on SIGTERM; set terminationGracePeriodSeconds longer than the longest message’s processing time, or use ScaledJob for long-running units of work.

Nodes

Option	Use when	Key settings
Cluster Autoscaler	Fixed node groups (ASGs, MIGs, VMSS), GKE/AKS standard	--balance-similar-node-groups, --expander=least-waste, scale-down utilisation threshold
Karpenter	EKS (and AKS via its provider), mixed instance types, fast provisioning	NodePool requirements, limits, disruption.consolidationPolicy: WhenEmptyOrUnderutilized, consolidateAfter
Managed autoprovisioning	GKE Autopilot, EKS Auto Mode	Mostly requests and PDBs; node shape is not yours to tune

• Node scale-up takes one to several minutes. Overprovision with a low-priority placeholder Deployment (negative PriorityClass) if bursts cannot wait.
• Consolidation and scale-down evict pods: every production workload needs a PDB that allows at least one disruption, and pods with local state need karpenter.sh/do-not-disrupt: "true" or cluster-autoscaler.kubernetes.io/safe-to-evict: "false", used sparingly.
• Set a hard ceiling (NodePool limits.cpu, node group max size) so a runaway HPA cannot run up an unbounded bill.

Gotchas

• HPA flapping: scale-down window too short, or the metric drops sharply when new pods share load. Lengthen scaleDown.stabilizationWindowSeconds before touching targets.
• Scaling on CPU for a service bottlenecked on a database adds pods that make the database slower. Find the real bottleneck first.
• Requests set far above usage make nodes look full while idle: the cluster autoscaler scales on requests, not usage.
• Changing autoscaling in production changes capacity and cost: show the planned min/max and get the user’s confirmation before applying.

Verification

kubectl get hpa orders -n shop -w                      # TARGETS shows a number, not <unknown>
kubectl describe hpa orders -n shop                    # Conditions: AbleToScale, ScalingActive True
kubectl get scaledobject -n jobs                       # READY True, ACTIVE as expected
kubectl get pods -n shop --field-selector=status.phase=Pending
kubectl top pods -n shop --containers

Run a load test (k6, hey, vegeta) in staging: replicas should rise within about a minute of load, latency SLO should hold, Pending pods should clear as nodes arrive, and replicas should return to the minimum gradually after load stops.

Output

**Workload:** orders (HTTP, CPU-bound)
**Sizing:** requests 250m/384Mi -> 400m/512Mi (VPA target after 7 days), memory limit = request
**HPA:** CPU 65%, min 3 / max 30, fast scale-up, 5 min scale-down window
**Nodes:** Karpenter NodePool limit 200 vCPU; PDB maxUnavailable 1
**Load test (staging, 4x peak):** 3 -> 14 replicas in 90 s, p95 latency 180 ms (SLO 300 ms), 0 errors
**Risks:** DB max_connections 500; at 30 replicas x 15 pool = 450, close to limit