Today’s distributed applications and systems are complex and constantly changing, making system observability challenging. For example, customers use multiple AWS SDKs and agents from different monitoring services to collect and analyze different performance data for their applications. Yesterday we announced the AWS Distro for OpenTelemetry, a 100% open source distribution of the OpenTelemetry project, which makes it easier for developers to collect and send application metrics and traces to multiple AWS monitoring services. The OpenTelemetry project provides a single set of open source APIs, SDK libraries, and agents for capturing metrics and distributed traces from applications. In our collaboration with the OpenTelemetry community, we contribute all changes back to the upstream project.
Open Telemetry offers a community-driven way to standardize the collection and transfer of telemetry data. AWS Distro for OpenTelemetry aims to make this even easier for developers to use OpenTelemetry components to instrument their application once. They can then send correlated performance data to AWS monitoring services, including Amazon CloudWatch and AWS X-Ray for monitoring and analysis. AWS Distro for OpenTelemetry provides auto-instrumentation agents that automatically collect trace data from the applications, while lessening the need to do this through manual coding. Customers can collect metadata about the underlying infrastructure, such as an AWS Lambda function identifier or container identifier, and correlate this data with application performance data, providing insights for troubleshooting and resolving issues. The distro supports applications running on:
- Amazon Elastic Container Service (Amazon ECS)
- Amazon Elastic Kubernetes Service (Amazon EKS)
- Amazon Elastic Compute Cloud (Amazon EC2)
- AWS Fargate
- on premises
AWS shares the core values of the upstream OpenTelemetry project, including reducing the costs of software telemetry instrumentation and ensuring the availability of open source, high-quality technologies that everyone can use. In this we are grateful to be collaborating with all other OpenTelemetry stakeholders, including Microsoft, Google, LightStep, New Relic, Red Hat, and Splunk.
Components of AWS Distro for OpenTelemetry
The AWS Distro for OpenTelemetry consists of SDKs, auto-instrumentation agents, collectors, and exporters to send data to backend services. AWS applies a comprehensive testing framework before each release to ensure that any overhead of OpenTelemetry components is quantifiable and predictable.
- OpenTelemetry SDK: AWS enhanced upstream OpenTelemetry SDKs in three areas. First, a collection of AWS resource-specific metadata elements, including container identifier, EC2 instance identifier, Task, and Pod identifier are now available. Second, OpenTelemetry SDKs now support the X-Ray trace format and context. Third, OpenTelemetry SDKs now correlate ingested trace and metrics data from X-Ray and CloudWatch.
- Auto-instrumentation agent: The Java auto-instrumentation agent in OpenTelemetry now supports AWS SDK and X-Ray trace data.
- OpenTelemetry Collector: AWS-specific exporters have been added to the Collector to send traces to X-Ray and metrics to CloudWatch.
AWS Distro for OpenTelemetry works alongside the CloudWatch agent to collect data needed for application monitoring. This means you can use the CloudWatch agent if it is already instrumented for your needs. You can use the CloudWatch agent to collect logs and infrastructure metrics. You can use OpenTelemetry to collect traces and application metrics. If you are using X-Ray currently, you can continue using the X-Ray SDK and daemon/agent for applications already instrumented. We recommend using AWS Distro if your objective is to standardize with the open source APIs from upstream OpenTelemetry.
AppDynamics – What is OpenTelemetry and Why Should You Care?