Test twice, deploy once: Testing infrastructure code on AWS

In a previous 2i blog we introduced The Role of Software Testing in ‘Infrastructure as Code’.

As the term suggests it means applying practices from software development to the activities of managing the underlying IT infrastructure.

In a re:Invent 2020 presentation “Test twice, deploy once: Testing infrastructure code on AWS” Darko Meszaros, an AWS Developer Advocate, explores this scenario in detail, explaining the use of AWS services like CloudFormation and the CDK - The Cloud Development Kit.

From 3m:30s Darko describes the main reasons for the infrastructure as code approach, notably ensuring that infrastructure changes are repeatable and predictable, releasing the infrastructure changes using the same tools as the local changes and being able to replicate the production in a staging environment to enable continuous testing.

He explains that having infrastructure as code would allow the developers or team to test and validate the code before the actual deployment, and from 5m:20s, Darko begins to talk about Cloudformation, which is an AWS service for infrastructure as code, to test it before deploying to production.

Cloudformation provides a common language to describe and provision all the infrastructure resources in the cloud environment, via text files written in YAML or JSON. From this, there is no need for any manual action or custom scripts to build infrastructure configurations.

Different text editors can be used to write cloud formation code, such as VS Code, one of the most commonly used text editors for this purpose. There are plugins available on VS code like Cloudformation mag, which will link and check the CloudFormation code within the editor.

Testing Infrastructure Code

From 7m:00s Darko focuses on the testing aspect, highlighting tools like CFN-NAG. This is a Ruby gem that you can set up locally on your workstation, and that you can use to detect potential patterns for insecure infrastructure. In addition, you can also set up your own rules like testing against multiple standard templates.

At 9m:18s he moves on to talk about the features of TaskCat, a testing framework to test the CloudFormation templates by actually deploying them. This framework supports report generation and log collections, and additionally, you can deploy to multiple regions with different parameter sets. The framework also provides you with the ability to build and package multiple Lambda functions.

From 11m:50s Darko talks about the next important topic, ‘Testing with the AWS CDK’. CDK code can be written in Java, Javascript, Typescript, Python, C-Sharp, etc., and the Jest framework can be used to test the CDK code. Three types of unit testing can be done with CDK testing. They are Snapshot tests, fine-grained assertions, and validation tests.

At 13m:44s he mentions that snapshot testing is an easy way to test CDK code. If the snapshot is not present when you run the snapshot test, a new snapshot will be created out of the current setup. Current CDK code will be synthesized into cloud formation to create a snapshot.

From 15m:37s Darko moves on to talk about the unit testing method, which is fine-grained assertion testing, an approach that will help in testing the specific aspects of your cloud formation code, and that are best suited for when you develop new features. He adds that validation tests are a generic way of testing the code through conditional statements.

Development Pipelines

To conclude, Darko zooms out from 17m:15s to look at the broader context of development pipelines, emphasizing that treating infrastructure code as like any other code is very important.

He walks through the pipeline process. Once you commit your code to a repository, the changes get picked by the pipeline and it goes through to the build phase, where the initial unit tests and packaging happens.

You can deploy these changes to a set of pre-production environments and then finally to production. At 18m:50s he adds that CDK pipelines help in creating a multi-account, multi-region, multi-stage testing pipeline for the application and infrastructure as code.

He demonstrates how the CDK application actually consists of three different stacks: The application stack, the stage and the pipeline stacks, with the stage stack acting as a wrapper for deploying the application, creating a service that makes reuse much easier and deploys across multiple accounts and regions.

At 23m:00s he talks about the deployment phase, explaining the combination of commands needed to successfully instantiate the configurations you have been building.

Conclusion

2i can assist organisations to adopt these techniques to also develop the qualities, practices and approaches that your people need to form high-performing Agile Delivery Teams. As testing experts, we can help your organisation embed best practices throughout your DevOps life-cycle and infrastructure management.

This requires a holistic understanding of a large, complex enterprise environment, including multiple technologies, departments and workflow interactions. 2i specialises in mapping this complexity and from that defining a DevOps blueprint that synthesises them together to achieve faster throughput of successful code deployment.