Reproducing failed CI tests locally with seeds

When tests fail in CI, you can use the test seed to reproduce the exact same test run locally. This is especially useful for debugging flaky tests or race conditions.

The problem

Sometimes tests pass locally but fail in CI. The test order might be different, or timing issues might surface under different conditions. ExUnit randomizes test order using a seed, so reproducing the exact same test run is crucial for debugging.

Using the seed

When tests fail in CI, copy the seed from the test output and run it locally:

# From CI output, look for something like:
# Running ExUnit with seed: 909692, max_cases: 4

# Run locally with the same seed
mix test --seed 909692

Finding the seed

The seed is printed in the test output. Look for lines like:

Running ExUnit with seed: 909692, max_cases: 4

Debugging strategy

Once you can reproduce the failure locally:

# Run with more verbose output
mix test --seed 909692 --trace

# Run only the failing test file
mix test test/my_failing_test.exs --seed 909692

# Run with coverage to see what code paths are hit
mix test --seed 909692 --cover

Pro tips

• Save the seed: When you encounter a failing test, always note the seed before it gets lost in CI logs
• Use --repeat: Combine with seeds to run the same test multiple times: mix test --seed 909692 --repeat 10
• Document flaky tests: If a test only fails with certain seeds, document this behavior while you fix the root cause

The seed ensures you can reproduce the exact same test conditions, making debugging much more predictable than hoping to randomly encounter the same failure again.