This blog post is the second in a two-part series. Part one of this blog post showed how to deploy and configure the CloudWatchAutoAlarms Lambda function to create a default alarm set and custom alarms for your Amazon Elastic Compute Cloud (Amazon EC2) instances using EC2 instance tags. In this post, I show how you can use AWS Config rules to enforce and remediate the required activation tag for your EC2 instances. In an account with many different workloads and workload owners, it can be difficult to consistently monitor and add alarms to your EC2 instances. If users provision instances without the required activation tag mentioned in the first blog post of this series, then your AWS environment might include EC2 instances without alarms. You can use an AWS Config rule to identify instances that are missing the required activation tag and an AWS Config rule remediation action to add the activation tag to these instances for standard alarm set creation.
- Configure and deploy the CloudWatchAutoAlarms Lambda function from Part 1 of this blog post.
- An AWS account with permissions to AWS Config, AWS Identity and Access Management (IAM), AWS CloudFormation, Amazon CloudWatch, AWS Lambda, and AWS Systems Manager. If you don’t have permissions, contact your security team.
- The AWS Command Line Interface (AWS CLI) installed with the permissions mentioned in the first prerequisite.
Identify EC2 instances missing the required activation tag for CloudWatchAutoAlarms
AWS Config includes a large collection of predefined rules that you can enable to track configuration compliance and perform remediation for AWS resources. In this walkthrough, I show how to use the required-tags AWS managed rule to track the Create_Auto_Alarms tag used by the CloudWatchAutoAlarms Lambda function.
- Open the AWS Config console. If this is the first time you have used AWS Config then choose Setup up AWS Config, otherwise choose Settings. Turn on recording of resource changes for the EC2: Instance resource type. If you are already recording All resources or the AWS EC2 Instance resource type, then you can skip this step.
- From the left navigation pane, choose Rules, and then choose Add rule.
- Choose Add AWS managed rule and enter
required-tagsin the Find rules search box. Select the required-tags rule and choose Next.
required-tags-cloudwatch-auto-alarmsfor the Name. In Description, enter
Identifies EC2 instances missing the Create_Auto_Alarms tag and adds the tag to the EC2 instance.
- For Trigger type, use the When configuration changes default option. This performs a check when a new EC2 instance is launched and whenever a setting on any existing EC2 instance changes, such as adding and removing tags.
- For Scope of changes, use the default option (Resources).
- For Resources, remove all resource types except for AWS EC2 Instance.
- For rule parameters, for tag1Key, enter
Create_Auto_Alarms. Choose the Remove button next to all other rule parameters so that only the tag1Key parameter remains and then choose Next.
- Choose Add rule on the review page to create the customized required-tags-cloudwatch-auto-alarms AWS config rule.
The required-tags-cloudwatch-auto-alarms AWS config rule is now active and AWS config will display any non-compliant Amazon EC2 instances that do not have the Create_Auto_Alarms tag in the AWS Config Rules menu.
Setup AWS Config rule remediation to tag Amazon EC2 instances missing the required activation tag for CloudWatchAutoAlarms
You can create a remediation action to automatically tag Amazon EC2 instances that are missing the Create_Auto_Alarms tag.
- From the left navigation pane, choose Rules and then select the required-tags-cloudwatch-auto-alarms rule.
- Choose Actions, and then Manage Remediation.
- For Select remediation method, choose Automatic remediation. This runs the remediation action you selected without any manual intervention.
- In the Retries field, enter 1 retry in 60 seconds. The rule checks compliance for every 60 seconds and retries the remediation if the resource is still noncompliant.
- For Choose remediation action, select AWS-PublishSNSNotification. AWS Config supports any AWS Systems Manager Automation runbook as a remediation action. The AWS-PublishSNSNotification runbook enables you to integrate your remediation action with many different endpoints using Amazon Simple Notification Service (SNS), like AWS Lambda functions.
- Skip the Rate Limits section to accept the default values.
- For Resource ID parameter, choose Message. The EC2 instance ID is used for the Message parameter in the AWS-PublishSNSNotification Systems Manager Automation runbook.
- Deploy this sample Amazon SNS CloudFormation template to create an Amazon SNS topic for this AWS Config rule. When the stack is deployed, enter the created SNS topic Amazon Resource Name (ARN) in the TopicArn parameter value. You can retrieve the ARN in the CloudFormation Outputs tab for the CloudFormation stack. This SNS topic will be notified when an EC2 instance is found that is missing the Create_Auto_Alarms tag.
- Deploy this sample Systems Manager automation role CloudFormation template to create an AWS Systems Manager automation role with Amazon SNS publish permissions. Enter the created SNS topic ARN from step 8 for the SNSPublishMessageTopicARN CloudFormation parameter. When the stack is deployed, enter the created AWS Identity and Access Management (IAM) role ARN in the AutomationAssumeRole* parameter value. You can retrieve the ARN in the CloudFormation Outputs tab for the CloudFormation stack.
- Choose Save Changes to activate the remediation.
Currently, there are no subscriptions to the SNS topic specified for the rule, so messages sent to the topic do not have any effect. You can deploy this sample AWS Lambda CloudFormation template to deploy a simple AWS Lambda function that adds tags to an EC2 instance using tags that you specify. The CloudFormation template deploys a Lambda function named EC2Tagger, subscribes it to an SNS topic you specify in the SNSTopicArn CloudFormation parameter, and gives the topic permissions to invoke the Lambda function. You can use the ARN for the SNS topic you created in step 8 for the SNSTopicArn CloudFormation parameter.
When the AWS Config rule is triggered because of a missing tag on an EC2 instance, a new message is sent to the Amazon SNS topic you specified with the instance ID in the Message field. The EC2Tagger Lambda function is called by the SNS topic and the EC2 instance is tagged with the tag key, Create_Auto_Alarms. You can then stop and start the instances at a convenient time (for example, a maintenance window) to create standard Amazon CloudWatch alarms as described in the first blog post of this series.
You can test the AWS Config rule remediation by launching an Amazon EC2 instance into your account without the Create_Auto_Alarms tag. Follow the directions in Part 1 of the blog post for an example of how to launch an EC2 instance with the CloudWatch agent from the AWS EC2 console. Once your AWS Config Rule remediation is executed, the missing Create_Auto_Alarms tag will be added to the instance. You will still need to stop and start the instance in order for the CloudWatchAutoAlarms Lambda function to execute and create the standard alarm set for the EC2 instance.
To avoid incurring additional charges in your account, clean up the resources you created.
- Delete the required-tags-cloudwatch-auto-alarms AWS Config rule and turn off AWS Config recording.
- Delete the CloudFormation stack you deployed using the sample Systems Manager automation role CloudFormation template,
- Delete the CloudFormation stack you deployed using the sample Amazon SNS CloudFormation template
- Delete the CloudFormation stack you deployed using the sample AWS Lambda CloudFormation template for the EC2Tagger AWS Lambda function.
In large AWS environments where many users use different methods for provisioning instances, it becomes difficult to consistently monitor and add CloudWatch alarms to your instances. In this blog post, I showed how to use AWS Config rules to identify EC2 instances that are missing the required tags for standard alarm set creation with the open source CloudWatchAutoAlarms Lambda function. AWS Config rules allow you to remediate violated rules. The EC2Tagger AWS Lambda function included in this post can be used to retag an instance so that standard alarms are created. You can modify the EC2Tagger function to perform other actions or create multiple remediations using any of the supported Amazon SNS endpoints.
About the Author
Khurram Nizami is an Operations Management Consultant at Amazon Web Services (AWS). Khurram is passionate about helping people build innovative solutions using technology. In his free time, Khurram enjoys hiking, nature, DIY projects, and travel.