End To End Tests On CircleCI 2.0 With Docker

It changes rapidly, not waiting for anyone. We have to adjust. I will show you today how to use CircleCI 2.0 as a workhorse for your end to end tests.

This blog post is a continuation of End to End Tests on CircleCI with Docker - Rails, Capybara, Selenium. The previous post regards CircleCI v1.0. It is strongly recommended to start with that at first to grasp the concept and then follow here to set everything up with CircleCi v2.0. If you want to read something about CircleCI 2.0 alone please take a look on my recent article.

Quick setup summary:

• Ruby on Rails Web Application

• Selenium with Chromedriver as automation tool

• Docker Composer setup for frontend and backend containers

End to End tests configuration

Setup similarities

At some points of setup you can refer to previous post. In the following steps there's no difference - everything should remain the same:

1. Local tests - backend

2. Local tests - frontend

3. CircleCI - backend

Of course you will have to migrate backend's configuration file to CircleCI 2.0. It should be pretty straight forward. All assumptions regarding end to end tests should extend in this case as well. Final file for backend might look similar to what follows:

Setup differences

First of all - you will have to follow the rules I described above to migrate your whole configuration file to new release syntax.

Aside from that there's also one catch hidden within end to end tests setup. You must create custom job with machine executor to have access to docker-compose commands. Builds executed in machine are slower than those with docker executor. They are more resource and time consuming. Unfortunately there's no other way to incorporate docker-compose to circle's built in docker environment, because containers and their relations are too much complicated and too custom to simply use predefined sets. Lucky us, CircleCI is kind enough to allow us to use different executors for different jobs. Because of this we will be able to run end to end tests independently, not disturbing the main build flow.

To make it work we may construct job similar to this (please refer to previous post for broader explanations):

run_e2e_tests:
  working_directory: ~/project
  machine:
    image: circleci/classic:latest
    docker_layer_caching: true # This speeds up build time by caching docker containers' layers. BEWARE it's paid feature!
  steps:
    - checkout
    - attach_workspace:
        at: ~/project
    - run:
        name: pull backend repo
        command: >-
                 GIT_SSH_COMMAND='ssh -i ~/.ssh/custom_ssh_key'
                 git clone git@github.com:netguru/backend_project.git ~/backend
    - run:
        name: prepare temporary directory for rspec results
        command: sudo mkdir /rspec_output
    - run:
        name: build containers with docker-compose # If you look at previous configuration we had to split container build to two steps. Now we can use multistage build and use one Dockerfile specified in docker-compose
        command: docker-compose -f docker-compose.ci.yml build
    - run:
        name: start containers and run e2e tests
        command: docker-compose -f docker-compose.ci.yml up --exit-code-from backend
    - store_test_results:
        path: /rspec_output

The job presented above can be optimised - that's for sure. Anyway I will leave this part for you dear reader. Psst... caching dependencies (like node_modules and gems) for containers might be the solution!

If you want to know more - few words about machine executor (here) and docker layer caching (here).

Final circle configuration may look more or less:

Afterword

This is just a mere example of how your end to end tests configuration may look like. With a huge variety of technologies and languages it's impossible to give you all-purpose solution. I think that the proposed way of doing it may be really handy to start your own setup. Please reach me in the comment section if you have any questions!