Recommend Mode¶
Recommend mode is the safest way to start with Attune. The operator
collects Prometheus metrics, computes recommendations, and writes them to the
policy's .status.recommendations field. Recommend does not in-place
resize running pods. CREATE can still size Job and CronJob pods when
updateStrategy.initialSizing is true and the namespace has label
attune.io/initial-sizing=enabled.
Creating a Recommend-mode policy¶
The bounds in this example are explicit. Omitted minimums floor at 1m CPU and 4Mi memory. Omitted maximums are not capped.
apiVersion: attune.io/v1alpha1
kind: AttunePolicy
metadata:
name: api-services
namespace: production
spec:
targetRef:
kind: Deployment
selector:
matchLabels:
tier: api
metricsSource:
prometheus:
address: http://prometheus-server.monitoring:80
historyWindow: 168h
minimumDataPoints: 48
cpu:
percentile: 95
overhead: "20"
minAllowed: "50m"
maxAllowed: "4000m"
controlledValues: RequestsAndLimits
memory:
percentile: 99
overhead: "30"
minAllowed: "64Mi"
maxAllowed: "8Gi"
allowDecrease: false
updateStrategy:
type: Recommend
cooldown: 1h
Reading recommendations from status¶
kubectl get attunepolicy api-services -o jsonpath='{.status.recommendations[*]}' | jq .
kubectl attune recommendations -n production
kubectl attune explain -n production api-services
Each entry in the array contains:
| Field | Description |
|---|---|
workload |
Name of the matched Deployment/StatefulSet/DaemonSet |
containers[].name |
Container name |
containers[].current |
Current CPU/memory requests and limits |
containers[].recommended |
Proposed CPU/memory requests and limits |
containers[].explanation |
Estimator chain (percentile → margin → burst → confidence → bounds → change filter) |
containers[].confidence |
Score between 0 and 1 |
containers[].dataPoints |
Number of Prometheus samples used |
stale |
When true, the newest finite sample is older than 3 * queryStep (default 15m), or the last rec was reused after an empty query. Apply paths skip until fresh data. Stale recs do not count toward Ready or savings |
Why is CPU recommended at Xm?¶
kubectl attune explain (and status.recommendations[].containers[].explanation) walks the same chain the controller used:
| Stage | Field | What it means |
|---|---|---|
| Raw percentile | rawPercentile |
Selected percentile across hourly buckets (max hour) |
| Overhead | afterOverhead |
Safety margin applied to the percentile |
| Burst | burstFactor / afterBurst |
Extra headroom when peak >> percentile |
| Confidence | confidenceFactor / afterConfidence |
Widens when data is sparse |
| Bounds | afterBounds / boundsApplied |
Clamped to minAllowed / maxAllowed |
| Change filter | afterChangeFilter / changeFilterApplied |
Min % change and max step caps |
| Final | final / finalAdjustment |
Value written to recommended (plus any controller post-steps) |
Field names match status JSON so automation can parse without scraping prose.
Why no resize happened?¶
Recommendations can look "ready" while pods stay unchanged. Common reasons:
| Reason | Where to look | What to do |
|---|---|---|
| Change filter / min change | explanation.*.changeFilterApplied |
Expected for tiny deltas; lower min change or wait for drift |
| Cooldown | condition Resizing=False reason CooldownActive |
Wait for cooldown / backoff |
| Budget cap | events BudgetExhausted (will fit on a later cycle) or IncreaseExceedsBudget (one increase is larger than the cap), metric attune_budget_exhausted_total |
Raise the cap or reduce the target. IncreaseExceedsBudget does not clear by waiting |
| Canary not promoted | status.canary.phase |
Wait for observation or set autoPromote |
| Deferred / Infeasible | condition ResizeBlocked, workloads.deferred / infeasible |
See troubleshooting |
| Stale data | recommendations[].stale |
Last-known values stay in status for up to 3 * queryStep. Resize and other apply paths skip. Ready and savings ignore stale recs. After the bound expires the rec drops and Ready can become InsufficientData. Fix Prometheus reachability |
| Schedule window | condition ScheduleBlocked |
Wait for window or adjust schedule |
| At target after clamp | events / logs for filtering or memory clamp | Expected when already near recommendation |
Interpreting confidence scores¶
The confidence score reflects how much data backs the recommendation:
| Score | Meaning | Action |
|---|---|---|
| 0.0 - 0.3 | Very low; sparse recent data | Wait for more data |
| 0.3 - 0.6 | Moderate; partial coverage | Review manually before acting |
| 0.6 - 0.8 | Good; substantial recent coverage | Safe to promote to Canary |
| 0.8 - 1.0 | High; near-complete history coverage | Safe to promote to Auto |
Tip
Increase historyWindow and minimumDataPoints for workloads with
weekly traffic patterns so the estimator captures weekday/weekend
variation.
Automation / JSON¶
For scripts, prefer status JSON over parsing explain prose:
kubectl get attunepolicy api-services -n production -o json | \
jq '.status.recommendations[] | {workload, containers: [.containers[] | {name, recommended, explanation}]}'
Schema fields under explanation are versioned with the CRD; breaking renames require a CRD/API bump and docs update.
Estimating savings¶
The policy status includes aggregated savings:
kubectl get attunepolicy api-services -o jsonpath='{.status.savings}' | jq .
{
"cpuRequestReduction": "1200m",
"memoryRequestReduction": "2Gi"
}
These values represent the total reduction across all matched workloads. Multiply by your per-core and per-GiB cloud pricing to estimate monthly cost savings.
Promoting to an active mode¶
When you are satisfied with the recommendations, change the mode:
- Use Canary to resize a subset first.
- OneShot applies at most one needing pod per cycle. Replicas that are already at the applied target, or that are blocked by QoS, node pressure, quota, or Infeasible plus InPlaceOnly, are skipped so another replica can still resize.
- Use Auto to resize all eligible pods (best for non-critical workloads).
kubectl patch attunepolicy api-services --type merge \
-p '{"spec":{"updateStrategy":{"type":"Canary","canary":{"percentage":10,"observationPeriod":"30m"}}}}'