By Biswajeet Rakshit, AWS Solution Architect at TCS
By Sigit Priyanggoro, Sr. Partner Solutions Architect at AWS
By Will Horn, Manager – Partner Solutions Architect at AWS
Now that multi-tenancy in the cloud is driving IT operational costs down, large enterprises are seeking seamless service delivery methods that address day-to-day activities in their cloud environment.
In this post, we will explain how AWS Identity and Access Management (IAM) user access keys and secrets may be stored in AWS Secrets Manager and rotated automatically using AWS Lambda at deterministic intervals.
Leveraging the solution outlined in this post allows customers to improve overall security posture by providing auditable and managed access key lifecycle for IAM users with Amazon Web Services (AWS) API keys and secrets.
AWS Identity and Access Management plays a crucial role for the authentication and authorization of users, applications, and services running on AWS.
The importance of IAM is heightened in environments that require and leverage native AWS authentication mechanisms. For these use cases, IAM user access keys and secrets provide the ability to authenticate with the component application services and data programmatically.
For example, IAM user access and secret keys are commonly utilized in environments where developers want to access an AWS resource like Amazon Simple Storage Service (Amazon S3) from their laptop or desktop, or with applications or services running on-premises or via third-party providers.
IAM users may also be utilized when applications running on Amazon Elastic Compute Cloud (Amazon EC2) instances are not capable of leveraging EC2 instance roles.
While IAM access keys and secrets serve their purpose to provide ease of service integration and controls within customer applications, IAM doesn’t have native capabilities to deterministically rotate user access keys and secrets.
For some customers, IAM key lifecycle and rotation may be a critical security requirement to maintain adherence to security or compliance standards for their environment.
Components and Workflow
Let’s look at the components required to incorporate automatic IAM access key rotation into your IAM operations workflow:
- AWS Lambda generates API calls to IAM services to rotate and update keys.
- Amazon CloudWatch initiates events on a scheduled basis to rotate keys.
- AWS IAM provides user access and secret keys for accessing AWS resources and services from non-AWS systems.
- AWS Secrets Manager stores secrets that are consumed by application or content developers.
- Amazon Simple Notification Service (SNS) sends notifications whenever keys are changed.
Figure 1 – Workflow of key IAM rotation.
Once the components—IAM user with API key or secret, AWS Lambda functions, Amazon CloudWatch Events, AWS Secrets Manager secrets—are created, the solution follows these steps:
- Every 90 days, a Lambda function creates a new key and uploads it as a current key in AWS Secrets Manager.
- SNS updates the application owner that a key has been rotated. Applications can use the AWS Secrets Manager API to retrieve the current key instead of hard coding in the application.
- After 100 days, the same Lambda function disables the old keys and a notification is sent to the user.
- After 110 days, the old keys are deleted and notifications are sent to the user again.
You can opt to rotate the keys every day and schedule the Lambda function every day, or you can schedule it at your convenience.
Setting Up the Key Rotation
In this post, we will build automated key rotation based on the following policies:
- All IAM users have to use new access key and secret key every 90 days.
- Deactivate previous access key and secret key every 100 days.
- Delete the previous access key and secret key every 110 days.
The keys lifecycle length can be customized based on your organization’s need by modifying the CloudWatch Event rules. In the next steps, we’ll create our Lambda function and integrate it with CloudWatch Event Rules.
Creating AWS Lambda Functions
Create Python 3.6 Lambda functions with a lambda_basic_execution role (or any other Lambda service role) that you decide should have permission to modify secrets and IAM user access and keys. Let’s name the Lambda function AutomatedKeyRotation for this example.
This Lambda function has three sub-functions:
Create_Key: This takes input from the CloudWatch Event for user id, creates the key in IAM, adds the key in AWS Secrets Manager, and sends a notification to the subscribed user. Here are some code snippets from the function:
Deactive_Key: This takes input from the CloudWatch Event for user id, compares IAM access keys and deactivates whichever key is older, and sends a notification to the subscribed user. Here are some code snippets from the function:
Delete_Key: This takes input form the CloudWatch Event for user id, checks whichever key is deactivated and deletes the key, and sends a notification to the subscribed user. Here are some code snippets from the function.
In the above sub-functions, you can also use send emails via your SMTP gateway server instead of using SNS topics. You can access the full Lambda code on GitHub.
Setting Up Amazon CloudWatch Event Rules
Create three CloudWatch Event rules with a schedule to call the Lambda function every 90, 100, and 110 days as outlined by the rotation policies. These CloudWatch rules should have the Lambda function as the target. The Lambda function name and version should be same across all the CloudWatch rules.
To create the CloudWatch Event rules, go to the AWS Management Console’s CloudWatch Events tab. On the Rules section, click Create rule as follows:
Figure 2 – Amazon CloudWatch Event rule configuration.
Once the Lambda function AutomatedKeyRotation has been set as target, we can click Configure details to name the rule and save it. Repeat the same process to create the three CloudWatch Event rules.
In Figure 2 above, you can see there are two parameters being passed to Lambda via CloudWatch Event rule input configuration:
- Parameter 1:
- key: action
- value: create
- Parameter 2:
- key: username
- value: <name of the user>
The Lambda function handler will use the Parameter 1 value as input to choose which function to be executed. In this case, the action value is create and Lambda will execute create_key function.
Figure 3 – AWS Lambda function handler.
The Lambda function will use the username value passed from the CloudWatch Event rule as uname input for create_key function.
Figure 4 – The create_key function is called based on event value ‘create.’
Next, create an SNS topic for Lambda to send notifications, and provide the SNS topic’s Amazon Resource Name (ARN) on the Lambda code below.
Figure 5 – SNS topic ARN.
Follow the same steps to set up the deactivate and delete functions based on CloudWatch Event rule input configuration.
IAM User and AWS Secrets Manager
To test our workflow, follow the steps below to create the IAM user and secret inside AWS Secrets Manager as a target of our Lambda function:
- Create the IAM user with access key and secret key for programmatic access.
- Create the AWS Secrets Manager secret for the IAM user you created, and set the secret name to the same value as the IAM user name.
- The secret contains AccessKey and SecretKey for the IAM user you created; see example below:
Figure 6 – IAM user AccessKey and SecretKey.
The AccessKey and SecretKey will be modified on the time intervals provided in the CloudWatch Event rule. Figure 7 below provides an example of a modified AccessKey and SecretKey based on our Lambda function and CloudWatch Event rule.
Figure 7 – Modified AccessKey example.
The Lambda code also modifies the access key variable with the new rotated values. Figure 8 below shows how the new access keys look like inside AWS Secrets Manager.
Figure 8 – Modified AccessKey example inside AWS Secrets Manager.
Here are some of the benefits this solution provides:
- IAM access keys and secrets lifecycle management based on configurable time periods.
- Enhanced visibility and auditability improves security posture and compliance while maintaining scalability.
- AWS Secrets Manager provides secure storage and masking of IAM keys and secrets.
- AWS native solution is lightweight and cost effective.
This post provides a method for using AWS-native services to seamlessly rotate, store, and consume IAM access keys and secrets using AWS services like Amazon CloudWatch, AWS Lambda, and AWS Secrets Manager to improve operational security and auditability.
The entire workflow and project detailed herein is available as part of the TCS Cloud Exponence platform >>
TCS Cloud Exponence was developed by Tata Consultancy Services (TCS) and helps enterprises implement a default service mechanism to remove duplication of IT services and achieve agility for their cloud operations. TCS is an AWS Partner Network (APN) Premier Consulting Partner and Managed Service Provider (MSP) with multiple AWS Competencies.
The TCS Cloud Exponence operations service model delivers managed services for all prevalent cloud architecture patterns, including public and private clouds. The platform provides enterprise customers with frictionless and rapid onboarding enabled by the TCS Machine First Delivery Model, while reducing transition time significantly and offering a standardized delivery experience.
TCS Cloud Exponence addresses key operational functionalities such as monitoring, incident, change, and release, along with cost management and security operations as well as integrated and automated patch management across multiple operating systems and platforms.
Tata Consultancy Services (TCS) – APN Partner Spotlight
TCS is an APN Premier Consulting Partner and Managed Service Provider (MSP). An IT services, consulting, and business solutions organization, TCS has been partnering with many of the world’s largest businesses in their transformation journeys for the last 50 years.
*Already worked with TCS? Rate this Partner
*To review an APN Partner, you must be an AWS customer that has worked with them directly on a project.