Parallelizing Jest and Tests on CircleCI

By Christopher Pappas

At Artsy, exploring ways to improve the developer experience is part of our makeup. Whether it’s implementing hot-swapping for Express.js or integrating the Rust-based SWC compiler into our front-end build pipeline, we’re always trying to reduce the amount of time it takes for a code cycle to take place. CI is no exception. When a developer opens a PR, we want to ensure they get timely feedback. Do their unit tests pass? Does the app build correctly? And how about smoke tests? Each of these jobs are complex processes that take time, and the more one can parallelize said tasks the less devs will need to wait. Scaled out to a whole engineering org, minor improvements to CI can be radical.

In this regard, two things came across our radar recently that we’d like to share: sharding via Jest, and a (free) way to parallelize integration tests via CircleCI’s split command.

Sharding in Jest

“What is sharding?” Good question! In short, it means “a small part of a whole”. The database community has employed sharding techniques for decades, where a large database is split up into smaller, more manageable chunks, usually to improve performance at scale. The same idea can be applied to any process or task involving a lot of data, including tests.

Think about it like this. Imagine an app that has thousands of tests. One can open up their terminal and run yarn test and execute all of the tests at once in a single process, or one can open two terminal tabs and run yarn test src/utils and yarn test src/routes, and have both processes allocate a pool of memory to complete each (smaller) subset of tasks. Because each process has its own memory pool the performance characteristics are generally better, and thus the overall time required to run our tests is reduced / decreased. Running each of these commands scoped to a particular folder is easy enough, but in a CI environment this is somewhat cumbersome; we’d need to define two new jobs and then the conditions in which they run, increasing the scope and complexity of our configuration file.

This is where Jest’s new sharding feature comes into play, which taps nicely into most modern CI runners. Using a hypothetical app containing 100 tests, here’s a quick example of how it works:

$ yarn jest --shard 1/5

What this says is: take the total number of tests (100), divide them into five buckets (containing 20 tests each), and execute the test runner against the first bucket (the first 20 tests). Continuing:

$ yarn jest --shard 2/5

Now take the second bucket and execute the next 20 tests – and so on. Simple enough.

Taking this further, we could turn this into a bash loop, including an & symbol to run things in parallel and automating some of the redundancy away:


for i in {1..${BUCKETS}}
   yarn jest --shard $i/$BUCKETS &

For many the above snippet should be sufficient to speed up your test suite, but who wants to write bash loops? Thankfully, most modern CI task runners contain the ability to split jobs into separate processes programatically and so this kind of logic is unnecessary.

Here’s how to do this in Circle CI:

  parallelism: 5
    - run: yarn test --shard=$(expr $CIRCLE_NODE_INDEX + 1)/$CIRCLE_NODE_TOTAL

Set a parallelism value, and drop the jest command into a cool one-liner. The variable CIRCLE_NODE_INDEX refers to which container index the job is running on, and CIRCLE_NODE_TOTAL points to the value of parallelism.

On, we’ve been able to reduce the average time it takes to run our unit tests from around ~10 minutes per PR to just above 2m. A 4-5x performance improvement.

Parallelizing Integration Tests (For Free)

For those who want robust integration test coverage, has been a game-changer due to its reliability and ease of use. Here at Artsy we use it in a number of apps, most notably Integrity. One complaint, however, is just how slow it is. This is reasonable; Cypress is simulating a user browsing your website and sometimes a user needs to do x and y (such as logging in) before they can do z. At scale this can really slow things down and lead to bottlenecks, especially if deploys are dependent on all of your integration tests passing.

The team has recognized this bottleneck and released the Cypress Dashboard, a paid product which includes the ability to unlock parallelized tests on your CI. For those willing to pay for another SAAS product this will get the job done well, but for those with leaner budgets there’s another way to accomplish this for free, and on CircleCI it’s very easy to setup via the CircleCI CLI command split.

You can check out the full example here, but in short:

  parallelism: 5
  run: |
    TESTS=$(circleci tests glob "cypress/integration" | circleci tests split | paste -sd ',')
    cypress run --spec $TESTS

We use the circleci tests glob command to gather all of our tests, and then pipe that into the circleci tests split which will divide our tests into buckets, similar to how Jest’s --shard command works up above. We then assign that to a $TESTS variable and pass it into cypress run --spec $TESTS. CircleCI sees the parallelism prop in the config and automatically divides our tests into 5 separate containers, each running a small subset of our integration tests in parallel.

On, our smoke tests times have gone from around ~7m on average down to ~3m. A huge reduction for only a few lines of config!