Changing endpoint to use IPv6 does not work

[nicolabeghin]

Describe the bug
When overriding endpoint in /opt/aws/amazon-cloudwatch-agent/bin/config.json to leverage IPv6-supported one as logs.<REGION>.api.aws in an IPv6-only EC2 instance, the agent keeps calling the IPv4 endpoint.

2025-04-06T10:36:32Z W! {"caller":"ec2tagger/ec2tagger.go:466","msg":"ec2tagger: Unable to describe ec2 tags for initial retrieval","kind":"processor","name":"ec2tagger","pipeline":"metrics/host","error":"RequestError: send request failed\ncaused by: Post "https://ec2.eu-south-1.amazonaws.com/\": dial tcp 52.119.128.109:443: i/o timeout"}

Steps to reproduce
Override endpoint in /opt/aws/amazon-cloudwatch-agent/bin/config.json through key endpoint_override

{
"agent": {
	"metrics_collection_interval": 60,
	"run_as_user": "cwagent"
},
"metrics": {
	"endpoint_override": "https://logs.eu-south-1.api.aws",

Then

/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json

You can check logs with

tail -f /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log

What did you expect to see?
No calls anymore to https://ec2.eu-south-1.amazonaws.com/

What did you see instead?
Calls to https://ec2.eu-south-1.amazonaws.com/

What version did you use?
1.300051.0b992

What config did you use?
config.json

Environment
Rocky Linux 9

[dricross]

The endpoint_override field sets the endpoint to use when pushing metrics to CloudWatch. The referenced error is referring to an EC2 endpoint. There is no configuration setting that will override the endpoints the CloudWatch Agent uses when making requests to EC2. This shouldn't prevent the agent from pushing logs to CloudWatch, but it will not replace InstanceId, InstanceType, or ImageId key/value pairs specified in the agent configuration (see https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html).

[nicolabeghin]

There is no configuration setting that will override the endpoints the CloudWatch Agent uses when making requests to EC2

As far as I understand this means there's no actual way at the moment to instruct Cloudwatch to use IPv6 AWS EC2 endpoints? Even if we can send metric to ipv6 cloudwatch endpoints if EC2 cannot be reached through ipv6 it's pretty much deceptive to say ipv6 support is available for cloudwatch ref. https://aws.amazon.com/about-aws/whats-new/2024/02/amazon-cloudwatch-logs-ipv6/

My 2 cents.

[goblinJoel]

Through this issue, I found out about the endpoint_override setting for metrics and logs and got my log files on my ipv6-only EC2 instance sending to CloudWatch again. However, the memory and disk usage data is not sending, making my alarms for those metrics useless. I still see these "Unable to describe ec2 tags for initial retrieval" errors and suspect they are blocking the agent from sending those metrics.

I tried removing the append_dimensions section, but that changed to an error that "https://monitoring.us-east-2.api.aws" does not exist. And of course, the non "api.aws" version of the monitoring endpoint is ipv4 only. Turns out monitoring is not on the list of endpoints that support ipv6. It appears I can't send metrics at all!

I second the request this issue raises, which is for the CloudWatch agent to fully work on ipv6-only EC2 instances. Although if I'm wrong about why my memory and disk usage metrics are broken, I'd be happy to hear how to fix the real issue!

[github-actions]

This issue was marked stale due to lack of activity.

[nicolabeghin]

I think this shouldn't be closed