We recently announced Amazon Managed Service for Prometheus (AMP) that allows you to create a fully managed, secure, Prometheus-compatible environment to ingest, query, and store Prometheus metrics. In a previous blog post from the AWS Management & Governance Blog, we explained how you could set up the service to monitor containerized environments. For some critical use-cases, containerization is too far ahead or sometimes not even possible.
In this article, we’ll demonstrate how you can use AMP for those systems running on Amazon Elastic Compute Cloud (Amazon EC2) or on-premises environments.
In this example, we walk through the following steps:
- Set up an Amazon EC2 instance running Amazon Linux.
- Run a demo application written in Go that exposes a Prometheus endpoint under
/metricsusing the Prometheus client library.
- Create an Amazon Prometheus Service (AMP) workspace.
- Set up aws-sigv4-proxy on the EC2 instance, which will secure our calls to AMP.
- Run a Prometheus server to export the application metrics to AMP through the proxy.
- Configure a Grafana server on a remote desktop to query our AMP workspace. You can also use our recently announced Amazon Managed Service for Grafana to query AMP.
The corresponding architecture can be visualized as follows:
In this example, we’ve select the Ireland (eu-west-1) region. Please visit the AWS Regional Service List to see AWS Regions supported by the service.
Amazon EC2 setup
The first step in this walkthrough is to set up an EC2 instance, which will host our application and forward its metrics to the AMP workspace that we’ll create later on. We recommend using IAM roles attached to the instance, to which we can attach the policy
AmazonPrometheusRemoteWriteAccess to provide the instance with the bare minimum permissions.
Once the instance is configured, we can log in to our instance and run a sample application. Create a file named
main.go and add the content shown below. Use the Prometheus http handler to auto-expose few system metrics via HTTP. You can implement your own metrics using the Prometheus client library.
Before running our sample application, let’s make sure we have all the dependencies installed.
The application should be up and running on port
8000. At this stage, we should be able to see all Prometheus metrics exposed by the application:
Create an AMP workspace
To create a workspace, simply open AMP on the AWS Console and enter the name of the workspace.
Once created, the service should provide us with a remote write URL and a query URL.
remote-write configuration, prometheus-server will issue API calls to our AMP workspace. For the service to validate our requests, they must be signed using valid IAM credentials. To do so, we will install aws-sigv4-proxy.
Running Prometheus server
To install Prometheus, we can run the following command:
Create a new file named
prometheus.yaml, and edit the
remote_write configuration with your workspace ID from the AMP workspace on the AWS console.
We are finally ready to run Prometheus and send our application metrics to AMP.
Our metrics are now being sent to AMP. You should see outputs in the aws-sigv4-proxy console.
Visualize metrics with Grafana
Grafana is a commonly used platform for visualizing Prometheus metrics. On a local machine, let’s install Grafana and configure our AMP workspace as a data source.
Make sure the local environment has the following permissions, or more, to query the workspace.
We will enable AWS SIGv4 authentication before running Grafana in order to sign queries to AMP with IAM permissions.
Log in to Grafana and go on the data sources configuration page /datasources to add your AMP workspace as a data source. The URL should be without the
/api/v1/query at the end.
SigV4 auth, then select the appropriate region and save.
Our application metrics should now be available and ready to be used.
In this post, we’ve detailed the setup metrics collection architecture using the recently announced Amazon Managed Service for Prometheus in a non-containerized environment, using Amazon EC2 instances. To automate metrics collections on your EC2 instances, you can preconfigure AMIs with all involved dependencies or use AWS Systems Manager. Learn more about Amazon Managed Service for Prometheus, Amazon Managed Service for Grafana, and OpenTelemetry from the links provided.