Rust: When Cargo Code Coverage Works Locally But Fails in CI

A short post to help if you run into the same issue.

TLDR: If your code coverage reports show vastly different results between local development and CI, explicitly set --engine llvm in your cargo tarpaulin command.

I recently ran into a frustration you may be familiar with: your code works perfectly on your machine, but fails on CI. The most annoying part are the waiting times for CI to finish. You context-switch to other work, if you do not forget, return to debug. It breaks flow and wastes time. Stuff you do not want to be spending time on.

The issue

My Rust code coverage reports were showing different results between local development (94%) and CI (22%). Same codebase, same test suite.

The original cargo command:

cargo tarpaulin --out xml

Relevant in this case: I test most of my functionality in the tests directory (integration/acceptance tests) and this is the coverage that seemed to be missing.

The Solution

Add the --engine llvm flag to your cargo tarpaulin command:

cargo tarpaulin --engine llvm --out xml

This forces tarpaulin to use the coverage engine that works in all environments.

Explanation

Cargo tarpaulin uses different default engines on different platforms. On Linux (CI), it defaults to the ptrace engine. On macOS (my local machine), it uses the llvm engine.

The ptrace engine traces system calls to measure coverage. The ptrace engine struggles with tests that spawn subprocesses (like acceptance tests running your compiled binary), whilst the llvm engine handles this correctly.