CloudWatch Setup¶
Attune can use Amazon CloudWatch Container Insights as its metrics source instead of Prometheus. This guide covers prerequisites, IAM configuration, policy setup, and verification for EKS clusters.
Prerequisites¶
- Amazon EKS cluster with
Container Insights
enabled. Container Insights publishes container-level metrics to the
ContainerInsightsCloudWatch namespace. - IAM permissions for the operator's pod to call
cloudwatch:GetMetricData(see IAM setup below). - Attune installed (see Installation).
Required CloudWatch metrics¶
Container Insights publishes these metrics to the ContainerInsights
namespace with dimensions ClusterName, Namespace, PodName, and
ContainerName. With prefer_full_pod_name left false, PodName is
the Deployment name or the CronJob name, not the pod name. Attune
matches those names. A sibling such as api-v2 is a different series.
| Metric | What it measures |
|---|---|
container_cpu_usage_total |
CPU usage per container, converted to cores with metricsSource.cloudwatch.cpuUnit |
container_memory_working_set |
Memory actively used per container in bytes |
container_cpu_usage_total is converted to cores with metricsSource.cloudwatch.cpuUnit. Millicores (the default) divides by 1000. Cores leaves the value unchanged. Nanocores divides by 1e9, which is what Attune did before this field existed. AWS documents the log field pod_cpu_usage_total as millicores and also shows an example that looks like cores. Compare one raw GetMetricData value with the container limit or kubectl top before rollout. Dividing a cores-valued point by 1000 still rounds to 1 millicore. Memory metrics stay in bytes.
The operator uses CloudWatch
SEARCH expressions
to find all matching containers within a namespace and pod prefix, then
groups results by ContainerName.
Note
Container Insights must be enabled for both ContainerName-level and
PodName-level dimensions. The default "enhanced observability" mode
in EKS provides these. If you use the basic Container Insights mode,
only cluster and node-level metrics are available.
Step 1: Configure IAM permissions¶
The operator needs cloudwatch:GetMetricData permission. There are two
approaches:
Create an IAM policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "cloudwatch:GetMetricData",
"Resource": "*"
}
]
}
Create an IAM role with the IRSA trust policy and attach the policy:
# Create the IAM policy
aws iam create-policy \
--policy-name AttuneCWReadOnly \
--policy-document file://attune-cw-policy.json
# Create the IRSA role
eksctl create iamserviceaccount \
--cluster my-eks-cluster \
--namespace attune-system \
--name attune \
--attach-policy-arn arn:aws:iam::<ACCOUNT_ID>:policy/AttuneCWReadOnly \
--approve
Or annotate the existing ServiceAccount via Helm:
helm upgrade attune oci://ghcr.io/attune-io/charts/attune \
--set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::<ACCOUNT_ID>:role/AttuneCWRole
If your cluster uses EKS Pod Identity, create a Pod Identity association:
aws eks create-pod-identity-association \
--cluster-name my-eks-cluster \
--namespace attune-system \
--service-account attune \
--role-arn arn:aws:iam::<ACCOUNT_ID>:role/AttuneCWRole
For centralized monitoring where CloudWatch data is in a different
account, specify roleArn in the policy to assume a cross-account
IAM role:
spec:
metricsSource:
cloudwatch:
region: us-east-1
clusterName: my-eks-cluster
roleArn: arn:aws:iam::123456789012:role/CloudWatchReadOnly
The operator uses STS AssumeRole to obtain temporary credentials.
The target role must trust the operator's IAM role/identity.
On an AttunePolicy or AttuneNamespaceDefaults, roleArn must
match --sigv4-allowed-role-arns (sigv4.allowedRoleArns). The
list is empty until you set it. Cluster AttuneDefaults is not
filtered. A namespaced assume sends ExternalId attune:<namespace>.
Step 2: Create an AttunePolicy¶
apiVersion: attune.io/v1alpha1
kind: AttunePolicy
metadata:
name: my-app
namespace: production
spec:
targetRef:
kind: Deployment
name: my-app
metricsSource:
cloudwatch:
region: us-east-1
clusterName: my-eks-cluster
cpu: {}
memory: {}
Configuration fields¶
| Field | Type | Default | Description |
|---|---|---|---|
region |
string | (required) | AWS region where CloudWatch metrics are stored (e.g., us-east-1) |
clusterName |
string | (required) | EKS cluster name, used as the ClusterName dimension filter (alphanumeric, hyphen, underscore) |
roleArn |
string | (optional) | IAM role ARN to assume for cross-account access (arn:aws:iam::ACCOUNT:role/NAME). On a policy or namespace defaults object it must match --sigv4-allowed-role-arns. |
cpuUnit |
string | Millicores | Scale of container_cpu_usage_total. Millicores divides by 1000. Cores leaves the value unchanged. Nanocores divides by 1e9. Empty means Millicores. |
Step 3: Verify the integration¶
Check policy conditions¶
kubectl get attunepolicy my-app -n production -o wide
| Condition | Meaning |
|---|---|
Ready: True, Reason: Monitoring |
CloudWatch reachable, recommendations computed |
Ready: False, Reason: InsufficientData |
CloudWatch reachable but not enough history yet |
Ready: False, Reason: MetricsUnavailable |
IAM, AssumeRole, or CloudWatch query failure |
Check the operator logs for CloudWatch-specific messages:
kubectl logs -n attune-system deployment/attune \
--tail=50 | grep -i cloudwatch
Verify Container Insights metrics exist¶
Confirm that Container Insights is publishing container-level metrics for your workload:
aws cloudwatch list-metrics \
--namespace ContainerInsights \
--metric-name container_cpu_usage_total \
--dimensions Name=ClusterName,Value=my-eks-cluster \
Name=Namespace,Value=production \
--region us-east-1
If this returns no results, Container Insights is not enabled or not publishing container-level dimensions. See the Container Insights troubleshooting guide.
Common errors¶
| Error | Cause | Fix |
|---|---|---|
loading AWS config |
No AWS credentials available | Configure IRSA, Pod Identity, or an instance profile for the operator pod |
CloudWatch GetMetricData failed: AccessDeniedException |
Missing IAM permission | Add cloudwatch:GetMetricData to the operator's IAM role |
empty result from CloudWatch instant query |
No metrics for the workload | Verify Container Insights is enabled and publishing container-level dimensions |
creating CloudWatch collector: operation error STS: AssumeRole |
Cross-account role assumption failed | Check the trust policy on the target role |
Rate limiting¶
The operator rate-limits CloudWatch API calls to 5 QPS with a burst
of 10, well within CloudWatch's default quota of 50 transactions per
second for GetMetricData.
Collectors are cached by region + clusterName + roleARN, so multiple
policies targeting the same cluster share a single API client.
Using CloudWatch as the cluster default¶
Instead of configuring CloudWatch on every policy, set it in
AttuneDefaults:
apiVersion: attune.io/v1alpha1
kind: AttuneDefaults
metadata:
name: cluster-defaults
spec:
metricsSource:
cloudwatch:
region: us-east-1
clusterName: my-eks-cluster
With defaults configured, policies only need targetRef:
apiVersion: attune.io/v1alpha1
kind: AttunePolicy
metadata:
name: my-app
namespace: production
spec:
targetRef:
kind: Deployment
name: my-app
cpu: {}
memory: {}
Unlike Datadog (which requires a Secret per namespace), CloudWatch
authentication uses IAM roles attached to the pod's ServiceAccount.
A single AttuneDefaults resource works for all namespaces without
additional Secrets.
Limitations¶
- EKS only. Container Insights metrics are only available on Amazon EKS clusters. Self-managed Kubernetes on EC2 can use Container Insights if the CloudWatch Agent is deployed manually, but EKS is the tested configuration.
- No auto-discovery. The
regionandclusterNamefields are always required. The operator does not auto-detect the EKS cluster. - Exclusive provider. At most one of prometheus, datadog, cloudwatch, or vpa. If the policy already sets any provider, cluster AttuneDefaults does not merge another.
- Safety monitor throttle detection uses the same metrics source as
recommendations. CloudWatch Container Insights does not expose CFS
throttle metrics (
container_cpu_cfs_throttled_periods_total), so CPU throttle-based auto-revert is not available when using CloudWatch. OOMKill detection, restart monitoring, and pod readiness checks still work. - Metric resolution. CloudWatch Container Insights publishes metrics at 1-minute intervals (60-second period minimum). The operator clamps the query period to a minimum of 60 seconds.