Travis CI to automate the process of building containers and deploying to Amazon Elastic Container Registry

With Apache Airflow we were able to solve our need for a tool that provides visibility into our task management workflow, scheduling, and maintenance. Using ECS operator we were able to reduce the number of dependencies but scaling this required automation around our container infrastructure with ability to quickly iterate through, build and push containers for production use.

In this post we walk through how we used Travis CI to build docker images and push them to AWS ECR (Elastic Container Registry) to be run by ECS operator in Airflow.

The .travis.yml file is the starting point for travis build and following sample illustrates the .yml file that contains encrypted variables to secure account details along with slack hook for build notifications

Note we have specified the path to build.sh and docker_push.sh files in .travis.yml which are used to provide commands to build and push docker images to ECR. In addition, EB_REGION is an example of an unencrypted environment variable that is available during the build. We can add encrypted variables such as secrets to .travis.yml using following command:

$ travis encrypt AWS_ACCOUNT_ID=variable_secret --add

In addition to the .travis.yml file, following files are required to build and push the container to ECR. Everytime the code is pushed to git, travis will pick up build and push the latest container image to ECR for further use.

• containers/sample_container/requirements.txt - Contains the requirements used to build the container
• containers/sample_container/sample.py - Contains the code to be run within the container
• containers/sample_container/.dockerfile - Dockerfile for building the container
• .travis/build.sh - Code to build the container
• .travis/docker_push.sh - Code to push the built image to Amazon ECR