This article is a guest post from Olaf Conijn, the creator of org-formation.

In the first two parts of this series, we learned how to manage your AWS Organizations using Infrastructure as Code (IaC) (Part 1) and how to create a continuous deployment pipeline for changes to your Organizations (Part 2). In the final installment of this series, we will look at org-formation-specific extensions to the AWS CloudFormation IAC language that make AWS CloudFormation aware of the organizational context within the AWS account. We’ll also create references to other resources across different AWS accounts and regions.

Org-formation Annotated CloudFormation templates

Another feature of org-formation is the ability to add organization-aware annotations to regular CloudFormation templates. Regular CloudFormation has no knowledge of organization resources and only supports specifying resources within a template that all need deployment to the same target account and region.

For each individual resource within a template, org-formation allows us to specify which account and region to deploy the resource. The mechanism by which we specify where to deploy a resource is the same Organization Binding as used within a tasks file. This means that resources within a template can be bound to multiple account/region combinations (e.g., by specifying the binding Account: '*').

Note that this is different from CloudFormation StackSets. With the StackSet feature of CloudFormation, you can execute a template in different target accounts and regions. The template, however, will always be the same for all targets. In practice, this means that for any unique set of resources, you must create a new CloudFormation template, resulting in a lot of work spent managing the relationships between these templates.

When executing the org-formation update-stacks command or adding an update-stacks task to a task file, org-formation will generate a CloudFormation template for each target you specified within your bindings. It will also create the resources bound to that target using CloudFormation.

\> org-formation update-stacks template.yml --stack-name my-stack

The following is an example of an Annotated CloudFormation template:

AWSTemplateFormatVersion: '2010-09-09-OC' # Include the file that contains the Organization Section.
# The Organization Section describes Accounts, Organizational Units, etc.
Organization: !Include ../organization.yml # Any Binding that does not explicitly specify a region will default to this.
# Value can be either string or list
DefaultOrganizationBindingRegion: eu-central-1 # Bindings determine what resources are deployed where
# These bindings can be !Ref'd from the Resources in the resource section
# Any Resource that does not specify a binding will use this binding.
# This specific binding selects all accounts from your organization that have a budget-alarm-threshold tag.
DefaultOrganizationBinding: AccountsWithTag: budget-alarm-threshold Resources: Budget: Type: AWS::Budgets::Budget Properties: Budget: BudgetName: !Sub 'budget-${AWSAccount.Alias}' # AWSAccount.Alias resolves to IAM Alias of current account BudgetLimit: Amount: !GetAtt AWSAccount.Tags.BudgetAlarmThreshold # Resolves to value of tag of current account Unit: USD TimeUnit: MONTHLY BudgetType: COST NotificationsWithSubscribers: - Notification: NotificationType: FORECASTED ComparisonOperator: GREATER_THAN Threshold: 1 Subscribers: - SubscriptionType: EMAIL Address: !GetAtt AWSAccount.Tags.AccountOwnerEmail

The previous template creates a Budget resource for every account in the organization with a tag BudgetAlarmThreshold. In the properties of this resource, various references to the organization.yml file are used:

  • The BudgetName of the Budget resource is a composite of budget and the value of the IAM alias in the created account. This is useful for identifying to which AWS account a Budget notification applies.
  • The Amount of the BudgetLimit specifies the value of the tag BudgetAlarmThreshold of the Budget resource in the created account.
  • The Address of the Email Subscriber specifies the value of the tag AccountOwnerEmail of the Budget resource in the created account.

Note that when resolving these references, the values are read from the organization.yml file that is included either by Organization attribute, or by the tasks file. If we manually change the value of the tag in the AWS console, org-formation will not know. If we change the value of a tag in the organization.yml, then org-formation knows that it needs to run both update-organization and update-stacks for templates that reference tags. Also, the Organization attribute does not require specification when including a template from within a tasks file. Overwrite attributes like DefaultOrganizationBindingRegion and the bindings from within a tasks file.

A reference to AWSAccount will resolve to the account the CloudFormation template executes in, much like AWS::AccountId. However, we can refer to any account in the organization.yml file by its logical name (e.g., !GetAtt MyDevAccount.Tags.AccountOwnerEmail or !Ref MyDevAccount) are also valid expressions, assuming we declared an account named MyDevAccount.

Cross account references in CloudFormation templates

As org-formation templates contain resources that will be deployed to multiple accounts, they can also contain the relationships (!Ref or otherwise) between these resources.

For example:

AWSTemplateFormatVersion: '2010-09-09-OC' # Include the file that contains the Organization Section.
# The Organization Section describes Accounts, Organizational Units, etc.
Organization: !Include ../organization.yml # Any Binding that does not explicitly specify a region will default to this.
# Value can be either string or list
DefaultOrganizationBindingRegion: eu-central-1 # Section that contains a named list of Bindings.
# Bindings determine what resources are deployed where
# These bindings can be !Ref'd from the Resources in the resource section
OrganizationBindings: # Binding for: S3Bucket, S3BucketPolicy CloudTrailBucketBinding: Account: !Ref ComplianceAccount # Binding for: CloudTrail CloudTrailBinding: Account: '*' IncludeMasterAccount: true Resources: S3Bucket: OrganizationBinding: !Ref CloudTrailBucketBinding DeletionPolicy: Retain Type: AWS::S3::Bucket Properties: BucketName: !Sub 'cloudtrail-${ComplianceAccount}' S3BucketPolicy: OrganizationBinding: !Ref CloudTrailBucketBinding Type: AWS::S3::BucketPolicy DependsOn: S3Bucket Properties: Bucket: !Ref S3Bucket PolicyDocument: Version: '2012-10-17' Statement: - Sid: 'AWSCloudTrailAclCheck' Effect: 'Allow' Principal: { Service: 'cloudtrail.amazonaws.com' } Action: 's3:GetBucketAcl' Resource: !Sub 'arn:aws:s3:::${CloudTrailS3Bucket}' - Sid: 'AWSCloudTrailWrite' Effect: 'Allow' Principal: { Service: 'cloudtrail.amazonaws.com' } Action: 's3:PutObject' Resource: !Sub 'arn:aws:s3:::${CloudTrailS3Bucket}/AWSLogs/*/*' Condition: StringEquals: s3:x-amz-acl: 'bucket-owner-full-control' CloudTrail: OrganizationBinding: !Ref CloudTrailBinding Type: AWS::CloudTrail::Trail DependsOn: - CloudTrailS3BucketPolicy Properties: S3BucketName: !Ref S3Bucket IsLogging: false IncludeGlobalServiceEvents: true IsMultiRegionTrail: true

The previous example demonstrates a CloudFormation template with three resources: CloudTrail, S3Bucket, and S3BucketPolicy. The CloudTrail resource deploys to all accounts, and the S3Bucket and S3BucketPolicy will only generate in the ComplianceAccount.

Executing org-formation creates a template for every account in the organization (the CloudTrail resource is bound to all accounts). All these templates will contain a CloudTrail resource. The template created for the ComplianceAccount will additionally contain the S3Bucket and S3BucketPolicy resources.

The CloudTrail resource has a reference to the S3Bucket resource, which is bound only to the ComplianceAccount account. What org-formation will do for all accounts that do not have both resources is create a CloudFormation export in the template deployed to the ComplianceAccount and declare a parameter in the templates deployed to all other accounts. When deploying, org-formation will create a dependency between the templates, to ensure the right order of execution, and copy the value from the export into the parameter of the other templates when deploying these.

This example illustrates the fragments from deploying the template to the ComplianceAccount:

Resources: S3Bucket: DeletionPolicy: Retain Type: AWS::S3::Bucket Properties: BucketName: cloudtrail-111111111111 # ... S3BucketPolicy omitted .... # Output section generated by org-formation for template deployed to the ComplianceAccount
Outputs: printDashCloudTrailS3Bucket: Value: !Ref S3Bucket Description: Cross Account dependency Export: Name: mystackname-CloudTrailS3Bucket

The cross account expression (!Ref S3Bucket) will be copied to the Value of the output. This can be any expression, also !GetAtt or !Sub.

This example illustrates the fragments from the template that will be deployed all accounts, except for the ComplianceAccount:

Parameters: CloudTrailS3Bucket: Description: Cross Account dependency Type: String ExportAccountId: '340381375986' ExportRegion: eu-central-1 ExportName: mystackname-CloudTrailS3Bucket # ... further down in the Resources section ... CloudTrail: Type: AWS::CloudTrail::Trail Properties: S3BucketName: Ref: CloudTrailS3Bucket

Note the removal of the DependsOn attribute from the original template. Although org-formation understands the relationship between the templates, CloudFormation does not, and there is no use for the DependsOn within the template deployed to CloudFormation. Being able to use references to organization resources and resources bound to different accounts allows us to create templates. These templates describe how to apply entire best practices and patterns to a multi-account setup. References also allow us to re-use these templates, as they do not contain account IDs or require you to deploy multiple CloudFormation templates.

Additional CloudFormation Annotations

Once we start modelling different parts of our resource baseline in CloudFormation, we will notice that we might need more than just the ability to refer to organization resources or resources across accounts/regions. Other features that can be useful are:

  • ForeachAccount attribute: Specifying a binding as the value of this attribute will create a copy of the resource for each account in the binding. This can be useful when setting up host names and certificates in our MasterAccount for each account that needs one, or when implementing Amazon GuardDuty and applying this to all accounts in our organization.
  • Fn::EnumTargetAccounts function: This function allows us to create an array of values for each account in a binding. Use this when setting up cross account IAM permissions that adhere to the principle of least privilege.

This is an example of the use of ForeachAccount:

Member: Type: AWS::GuardDuty::Member OrganizationBinding: IncludeMasterAccount: true ForeachAccount: Accounts: '*' Properties: DetectorId: !Ref Detector Email: !GetAtt CurrentAccount.RootEmail MemberId: !Ref CurrentAccount Status: Invited DisableEmailNotification: true

In the example, a Member resource generates for each account in the specified binding (Accounts: '*'). When creating a resource for each account in the binding, useCurrentAccount to resolve information about the account being iterated over. AWSAccount will still refer to the AWS account part of the target (in this case the MasterAccount). A full example on how to implement GuardDuty using org-formation can be found in GitHub.

This is an example of Fn::EnumTargetAccounts, and how to create a resource policy and provide access to other accounts:

OrganizationBindings: # Binding for: Bucket, BucketPolicy BucketAccountBinding: Account: !Ref MyAccount # Binding for: S3BucketReadAccessPolicy ReadAccessAccountBinding: # default = empty binding Conditions: CreateReadBucketPolicy: !Not [ !Equals [ Fn::TargetCount ReadAccessAccountBinding, 0 ] ] Resources: Bucket: Type: AWS::S3::Bucket OrganizationBinding: !Ref BucketAccountBinding DeletionPolicy: Retain Properties: BucketName: !Sub '${bucketName}' BucketReadPolicy: Type: AWS::S3::BucketPolicy OrganizationBinding: !Ref BucketAccountBinding Condition: CreateReadBucketPolicy Properties: Bucket: !Ref Bucket PolicyDocument: Statement: - Sid: 'Read operations on bucket' Action: - s3:Get* - s3:List* Effect: "Allow" Resource: - !Sub '${Bucket.Arn}' - !Sub '${Bucket.Arn}/*' Principal: AWS: Fn::EnumTargetAccounts ReadAccessAccountBinding arn:aws:iam::${account}:root

In this example, we created a Bucket resource in MyAccount. A BucketPolicy provides Get/List access to this bucket for all the accounts that are part of the ReadAccessAccountBinding. In the same example, the task file supplies the ReadAccountAccountBinding. The default specified in the template is an empty binding (no account will get access to the Bucket resource).

Note that as the default is an empty binding, the EnumTargetAccounts will generate an empty array and it is only possible to create a valid BucketPolicy if there are more than zero accounts part of the ReadAccountAccessBinding. The function Fn::TargetAccount will return the number of accounts part of a binding, which can be used in a CloudFormation condition. We can find a more complete example on how to set up cross account access to Amazon S3 buckets in GitHub.

Summary

In this series, we learned about three features that the org-formation tool provides. Use each of these to set up and manage resources across  AWS Organizations:

We wrote this article on version 0.9.6 of org-formation. For the most recent version, examples, and documentation, refer to the Github project page.

Feel free to engage, create issues, ask questions over Slack, provide feedback, and share your experiences.

Olaf Conijn

Olaf Conijn

Olaf Conijn is a software developer and architect with almost 20 years experience. Throughout his career he has had experience in a variety of different companies, which includes big tech, startups, and large financials. In recent years Olaf has grown an interest in building serverless architectures and managing the infrastructure used to run serverless. His current mission is to help organizations to implement scalable, secure, compliant yet cost-effective infrastructure using the AWS cloud.

The content and opinions in this post are those of the third-party author and AWS is not responsible for the content or accuracy of this post.