Resize API

Attune uses the Kubernetes 1.32+ in-place pod resize API to adjust container resources without restarting pods. This page explains how the resize API works and how the operator uses it. Version-best behavior across Kubernetes minors (envelope, HPA idle, doctor cgroup) is in Version-best Kubernetes.

The /resize subresource

Kubernetes 1.32 added the /resize subresource on pods as part of the InPlacePodVerticalScaling alpha feature gate. Kubernetes 1.33 graduated the feature to beta (enabled by default). Kubernetes 1.35 graduated it to GA. The subresource accepts a modified PodSpec with new container resource requests/limits.

PATCH /api/v1/namespaces/{ns}/pods/{name}/resize

The kubelet applies the new resources to the running container's cgroup limits without restarting it. CPU changes take effect immediately; memory limit increases take effect immediately.

Memory limit decreases:

Cluster	Behavior
Kubernetes 1.33 and earlier	Kubernetes 1.33 rejects in-place memory limit decreases when resizePolicy for memory is NotRequired (default). Attune clamps the limit (keeps the higher current value) unless policy is RestartContainer.
Kubernetes 1.34+	Live memory limit decreases are allowed. The kubelet does a best-effort check against current usage; a race can still OOM if usage spikes after the check. Attune detects the version at startup and skips the platform clamp so decreases can apply when allowDecrease permits. Attune also applies a client-side usage floor: target limit must stay above recent usage (recommendation raw percentile) times (1 + decreaseUsageMarginPercent/100) (default 10%). Directional maxDecreasePercent still steps large shrinks over multiple cycles.

How Attune uses it

The operator's resize engine (internal/resize/engine.go) performs resizes via the typed Kubernetes client:

clientset.CoreV1().Pods(namespace).UpdateResize(ctx, name, updatedPod, opts)

Pre-checks before resize

The engine writes the request first. maxAllowed caps that request, not the limit. When controlledValues is RequestsAndLimits, scaleControlledLimits then derives the CPU and memory limits from the new request. An omitted limitMultiplier keeps the live request-to-limit ratio. An explicit multiplier replaces that ratio. A value below 1 is rejected, because a limit cannot be smaller than its request. applyLiveResizeTarget runs next: platform memory clamp, usage floor, and, on an already Guaranteed pod, a raise of the memory request up to the memory limit. That applied memory request can exceed maxAllowed. The resize path does not multiply the limit a second time. shouldSkipResize and PreservesQoS see that applied target. A CPU multiplier above 1 on a Guaranteed pod leaves the CPU request and limit unequal, so the resize is skipped and the pod is not evicted. A memory multiplier above 1 on an already Guaranteed pod keeps the class, because the memory request is raised to the new limit, and the resize proceeds.

Before calling UpdateResize, the controller runs several safety checks:

1. Pod already at target: Skips if the running pod's actual resources match the recommendation (compares against the live pod, not the Deployment template).
2. Memory limit platform clamp: On clusters that reject live memory limit decreases (NotRequired policy, Kubernetes 1.33 and earlier), preserves the higher current limit.
3. Memory limit usage floor: When decreasing a memory limit, raises the target if it would fall at or below recent usage plus memory.decreaseUsageMarginPercent headroom.
4. Node capacity / pressure: Verifies that this pod's requests after resize, plus other pods on the node, do not exceed allocatable; skips request increases under MemoryPressure / DiskPressure / PIDPressure.
5. LimitRange/ResourceQuota: Checks that the target doesn't violate namespace constraints.
6. QoS preservation: The check runs on the spec UpdateResize would send, before that call, so InPlaceOrRecreate does not evict a pod whose class would change. Every app container, init container (including a completed one-shot init), and native sidecar counts. A zero quantity is unset. RequestsOnly can still change class when a live limit already equals the new request, and that resize is skipped. Burstable to Guaranteed and BestEffort to Burstable name both classes. The Guaranteed reason keeps the RequestsAndLimits hint. When spec.resources is set, both sides of the check use one rule: the pod-level envelope if in-place pod-level resize is on, and the containers if it is off. Status.QOSClass is not compared with a container class. This does not assume PodLevelResources is on by default. A pod with no spec.resources still uses Status.QOSClass for the current class.
7. Pod-level envelope: If the pod already has spec.resources, Attune never invents an envelope and never shrinks one. When in-place pod-level resize is on, the same UpdateResize writes the container target and a raised envelope (requests cover the container sum; an existing limit covers the max of current limit, needed request, and the max single container limit, and stays at least the request). When the field is present but in-place raise is off, a container increase that would exceed the current envelope is skipped. Under RequestsOnly plus Burstable, Attune skips rather than lift an existing envelope limit (user cap). Skips emit ResizeSkipped and history reason envelope_constraint. CREATE initial sizing and template persist use the same raise-to-cover helper so new pods and workload templates keep a valid envelope.
8. Resize policy warning: If the container has resizePolicy set to RestartContainer, the operator logs a warning but proceeds with the resize (the kubelet will restart the container).

Post-resize tracking

After a successful resize, the operator:

1. Writes tracking annotations and the tracked label with a metadata merge patch (see table below). A kubelet status update does not conflict with that patch.
2. If autoRevert: true, monitors the pod for safety violations (OOMKill, CPU throttle, restart spikes, NotReady).
3. Records the operation in status.resizeHistory.
4. Emits a Kubernetes Event (Normal/Resized).

Pod tracking annotations

After a resize, the operator writes these annotations to the pod for safety observation and revert tracking:

Annotation	Description
attune.io/resized-at	RFC 3339 timestamp when this observation window started. It moves forward when a deferred or still-fresh in-progress resize finishes after the original window would have closed. An in-progress condition that has been true for an hour clears tracking instead of moving this timestamp.
attune.io/resize-apply-pending	true when the observation period elapsed before the kubelet finished applying the resize. Removed when the period restarts, when that in-progress condition has been true for an hour, or when tracking is cleared.
attune.io/resized-containers	Comma-separated list of resized container names
attune.io/resized-workload	Name of the parent workload
attune.io/original-cpu-request.<container>	CPU request before the resize (per container)
attune.io/original-memory-request.<container>	Memory request before the resize (per container)
attune.io/original-restart-count.<container>	Container restart count at resize time (per container)
attune.io/original-cpu-limit.<container>	CPU limit before the resize (when limit is non-zero)
attune.io/original-memory-limit.<container>	Memory limit before the resize (when limit is non-zero)
attune.io/policy	Name of the AttunePolicy managing this pod (used for safety observation provenance checks and finalizer cleanup)
attune.io/startup-boost-at	RFC 3339 timestamp when a startup CPU boost was applied
attune.io/startup-boost-containers	Comma-separated containers that boost applied, or found already at the boost target. Expiry reduces only these. Absent on older stamps, which still expire every container.

These annotations are removed once the safety observation period completes (regardless of whether the resize is kept or reverted). Completion waits while the kubelet still reports the resize in progress or deferred. See Observation period. When a policy is deleted, the attune.io/cleanup finalizer removes all tracking annotations from managed pods before allowing garbage collection. Pods keep their current (resized) resource values; only annotations are cleaned.

When multiple containers in the same pod are resized in the same cycle, each container gets its own set of per-container annotations (e.g., attune.io/original-cpu-request.app, attune.io/original-cpu-request.sidecar). The attune.io/resized-containers annotation lists all resized containers as a comma-separated value.

Those keys are the in-band lifecycle. internal/lifecycle names the states (Idle, Observing, Evaluating, RestorePending, Incomplete) from the same annotations. status.conditions type SafetyObservation reports the dominant state for the policy.

Resize lifecycle

The diagram below shows the complete decision tree for a single pod during a resize cycle. The operator evaluates every pod in the workload through this flow on each reconciliation.

stateDiagram-v2
    [*] --> Discovered: Pod found in workload

    Discovered --> Eligible: IsEligibleForResize = true
    Discovered --> Ineligible: IsEligibleForResize = false

    state Ineligible {
        direction LR
        [*] --> NotRunning: Phase != Running
        [*] --> Deleting: DeletionTimestamp set
        [*] --> InProgress: PodResizeInProgress
        [*] --> Pending: PodResizePending (non-Infeasible)
    }

    Eligible --> Infeasible: IsResizeInfeasible = true
    Eligible --> PreChecks: IsResizeInfeasible = false

    state PreChecks {
        direction TB
        [*] --> AtTarget: Already at recommended values
        [*] --> NodeCap: Would exceed node allocatable
        [*] --> QuotaViolation: Would violate LimitRange/Quota
        [*] --> EnvelopeConstraint: spec.resources would be exceeded
        [*] --> QoSChange: Would change QoS class
        [*] --> PassedChecks: All checks pass
    }

    Infeasible --> EvictOrSkip

    state EvictOrSkip {
        direction LR
        [*] --> InfeasibleSkipped: resizeMethod = InPlaceOnly
        [*] --> EvictionCheck: resizeMethod = InPlaceOrRecreate
    }

    note right of InfeasibleSkipped: Skipped to avoid wasted\nUpdateResize API calls\nthat would fail every cycle

    EvictionCheck --> LastReplicaGuard

    state LastReplicaGuard {
        direction LR
        [*] --> EvictionBlocked: Only 1 running replica
        [*] --> EvictPod: 2+ running replicas
    }

    EvictPod --> Evicted: Eviction API accepts
    EvictPod --> EvictionDenied: PDB blocks eviction

    PassedChecks --> ResizePod: UpdateResize API call

    ResizePod --> ResizeSuccess: API returns success
    ResizePod --> ResizeFailed: API returns error

    ResizeFailed --> EvictOrSkipFailed

    state EvictOrSkipFailed {
        direction LR
        [*] --> Failed: resizeMethod = InPlaceOnly
        [*] --> EvictionFallback: resizeMethod = InPlaceOrRecreate
    }

    EvictionFallback --> LastReplicaGuard

    ResizeSuccess --> SafetyObservation: autoRevert enabled
    ResizeSuccess --> Done: autoRevert disabled

    state SafetyObservation {
        direction TB
        [*] --> Monitoring: Watching for OOMKill, throttle, restarts, NotReady, SLO breach
        Monitoring --> Safe: No violations after observation period
        Monitoring --> Unsafe: Violation detected
    }

    Safe --> Done: Annotations removed
    Unsafe --> RevertPod: Restore original resources
    RevertPod --> Done: Annotations removed

Decision points explained

Decision	Function	Location
IsEligibleForResize	resize.IsEligibleForResize()	internal/resize/engine.go
IsResizeInfeasible	resize.IsResizeInfeasible()	internal/resize/engine.go
Pre-checks	shouldSkipResize()	internal/controller/resize.go
Resize method	policy.Spec.UpdateStrategy.ResizeMethod	CRD field
Last replica guard	tryEvictionFallback()	internal/controller/safety.go
ResizePod	resizer.ResizePod()	internal/resize/engine.go
Safety observation	checkPendingSafetyObservations()	internal/controller/safety.go
Revert	safety.Monitor.RevertPod()	internal/safety/monitor.go

Key behaviors

• Infeasible pods are eligible. IsEligibleForResize returns true for pods the kubelet has marked PodResizePending=Infeasible. These pods cannot be resized in-place on their current node, but they are included in the resize cycle so the eviction fallback can handle them when resizeMethod is InPlaceOrRecreate. With InPlaceOnly, they are skipped with a log message ("Pod resize is Infeasible and resizeMethod is InPlaceOnly, skipping") to avoid wasting an UpdateResize API call that would fail on every reconcile cycle.

• OneShot walks past blocked replicas. OneShot applies at most one needing pod per cycle. Replicas that are already at the applied target, or that are blocked by QoS, node pressure, quota, or Infeasible plus InPlaceOnly, are skipped so another replica can still resize.

• Eviction respects PDBs. The operator uses the Kubernetes Eviction API (EvictV1), which enforces PodDisruptionBudgets. If the PDB would be violated, the eviction is denied and the pod stays as-is.

• Last replica protection. The operator never evicts the last live Running replica of a workload, even if the PDB would allow it. The count comes from a live Clientset list (status.phase=Running and no deletion timestamp). spec.replicas and NotReady pods do not count.

• Eviction fallback restarts the pod from the current template. When a pod is evicted, the workload controller (Deployment, StatefulSet) creates a replacement pod from the current PodTemplate. With templatePersistence.when=AfterSuccessfulResize enabled, Attune patches the template after Evicted history so the replacement starts at the recommended requests. Persist applies the same memory usage floor as live resize: a stale rec.Current that is still the old template cannot write a limit below recent usage. Without persist, the replacement keeps the old template until a later in-place resize. Evicted pods do not enter the safety observation path.

• Fail-open schedule. If the configured timezone is invalid, isWithinResizeWindow returns true (allows resize) rather than silently blocking all resizes. Invalid timezones should be caught by webhook validation at admission time.

Limits and caveats

• Memory limit decreases: On Kubernetes 1.33 and earlier, the API forbids decreasing a container's memory limit in place unless the container's resizePolicy for memory is RestartContainer. If the pod uses NotRequired (the default) or has no resize policy, Attune clamps the memory limit to the current value and only decreases the memory request. Kubernetes 1.34 and newer allow that decrease without a restart. Attune skips the platform clamp on those clusters and still applies the usage floor.
• Memory request decreases: The kernel only reclaims memory when the working set drops below the new limit. If the application holds onto allocated memory, the decrease has no practical effect until the process releases it.
• Init containers: Traditional init containers (no restartPolicy, or any value other than Always) are not resized in-place. Native sidecars (restartPolicy: Always) are CREATE-sized, in-place resized, and startup-boosted. They are still subject to excludeKnownSidecars. See Istio integration.
• Restart policy: Containers with resizePolicy: RestartContainer will be restarted by the kubelet when their resources change.
• Prometheus address limit: The operator caches at most 64 unique Prometheus collector instances. If more than 64 distinct addresses are configured across all policies, additional addresses are rejected with an error status.
• Minimum cooldown: The operator enforces a minimum cooldown of 1 minute regardless of the cooldown field value. This prevents accidental DoS via rapid resize loops.