Managing infrastructure efficiently and securely is a top priority for any organization. At Tray.io, we have adopted Terraform and Atlantis as our preferred methods for infrastructure management. To bolster our security measures, we use AWS CloudTrail to monitor and audit changes made to our AWS environment. However, tracking the source of these changes has been a challenge. In this blog post, I’ll walk you through how we solved this problem by combining features from Atlantis and the Terraform AWS provider.
The Challenge: Tracking Changes in Terraform
When you apply Terraform changes using the same base AWS role or credentials, it becomes difficult to pinpoint the source of each change request. We wanted to know the repository, pull request number, and GitHub user associated with every change. This information is crucial for maintaining transparency and accountability in our infrastructure management process.
Solution: Leveraging Atlantis and Terraform
We found a solution by leveraging the power of Atlantis and the Terraform AWS provider. Here’s how we did it:
Atlantis Environment Variables
First, let’s explore Atlantis. When running a plan or apply with Atlantis, it injects some valuable environment variables into the process. In our case, we are particularly interested in the following variables:
BASE_REPO_NAME: The name of the base repository.BASE_REPO_OWNER: The owner of the base repository.PULL_NUMBER: The pull request number.PULL_AUTHOR: The GitHub user who initiated the pull request.
Combining Environment Variables
Our goal was to combine these environment variables into a single new variable and pass it to Terraform’s plan and apply commands. To achieve this, we needed to override the default plan and apply workflows in the server side atlantis.yml file.
Here’s what you need to include in your atlantis.yml file:
workflows:
default:
plan:
steps:
- env:
name: TF_APPEND_USER_AGENT
command: 'echo "PLAN|$BASE_REPO_OWNER/$BASE_REPO_NAME|$PULL_NUMBER|$PULL_AUTHOR"'
- init
- plan
apply:
steps:
- env:
name: TF_APPEND_USER_AGENT
command: 'echo "APPLY|$BASE_REPO_OWNER/$BASE_REPO_NAME|$PULL_NUMBER|$PULL_AUTHOR"'
- apply
The Role of TF_APPEND_USER_AGENT
You might be wondering why we specifically used TF_APPEND_USER_AGENT for the new combined environment variable. The reason is that TF_APPEND_USER_AGENT is an expected and recognized environment variable for the AWS Terraform provider. If set, the provider will append its content to the User-Agent header when making requests AWS.
CloudTrail Logs
With these changes in place, when you check the User-Agent header value in your CloudWatch logs, you will find the new values appended to the end. The string is structured in a way that makes it easy to parse using your logging platform of choice, allowing you to expose them in dedicated searchable fields if needed.
Conclusion
Tracking infrastructure changes can be challenging, but with the right combination of tools and strategies, we can enhance transparency and accountability. By integrating Atlantis and the Terraform AWS provider, we’ve successfully identified the source of changes in our infrastructure, making our management process more secure and efficient.
It’s worth noting that there is another way to do this using Assume Role Session Names. This did not work for us because of the way we pass credentials to Atlantis.
Big shout out my colleague James Eastwood who was pivotal in brainstorming a implementing this.