---
name: flaky-test-triage
description: Triage one intermittently failing (flaky) test. Reruns it to measure the failure rate, checks for order dependence, classifies the cause such as timing, shared state, clock, randomness, network or ordering, and proposes and verifies a targeted fix. Use when the user reports a flaky, intermittent or sometimes-failing test.
---

# Flaky test triage

Test: `$ARGUMENTS`. If empty, ask for the test identifier and any CI failure output.

## 1. Reproduce

1. Find the command that runs only this test (for example `pytest path::name`, `npx vitest run path -t "name"`, `npx jest path -t "name"`, `go test ./pkg -run '^TestName$' -count=1`, `./vendor/bin/phpunit --filter name`).
2. Run it in a loop, default 20 times, and count failures. Example for a POSIX shell:
   ```sh
   pass=0; fail=0
   for i in $(seq 1 20); do
     if <command> > /tmp/flaky-run-$i.log 2>&1; then pass=$((pass+1)); else fail=$((fail+1)); fi
   done
   echo "pass=$pass fail=$fail"
   ```
   For Go, `-count=20` does the loop for you.
3. Run the whole file or package containing the test several times as well. A test that only fails alongside others is order-dependent.
4. If your runner supports random ordering or parallel workers and the project already uses them, try those modes too.
5. Keep one failing log and read it closely.

If you get zero failures, say so, and list the CI-only differences to check: parallel workers, CPU speed, timezone and locale, environment variables, service containers, test ordering.

## 2. Classify

Check each cause against the test and the code it exercises. Cite `file:line` evidence.

| Cause | Signs |
|---|---|
| Async not awaited | Missing `await`, callbacks or events asserted before they fire, fire-and-forget calls |
| Fixed sleeps | `sleep`, `setTimeout`, `time.sleep` used to wait for work to finish |
| Shared state | Module-level variables, singletons, caches, env vars or globals mutated by tests without reset |
| Leftover data | Database rows, files or queue messages from earlier tests; missing transaction rollback or cleanup |
| Clock and timezone | `now()`, `Date.now()`, date math near midnight or month end, DST, local timezone assumptions |
| Randomness | Unseeded random values, generated IDs used in ordering, fake data generators |
| Ordering | Comparing sets, dict/map iteration, or unordered query results as ordered lists |
| Network | Real HTTP or DNS calls, external services, rate limits |
| Resource collisions | Fixed ports, shared temp paths, the same DB name across parallel workers |
| Precision | Floating point equality |

Pick the most likely cause. If two are plausible, say which evidence would separate them.

## 3. Fix

Propose the smallest change that removes the cause:
- Replace sleeps with waiting on an explicit condition or awaiting the promise/future.
- Fake or freeze the clock; pass time in as a parameter.
- Reset shared state in setup/teardown, or construct fresh instances per test.
- Wrap DB tests in a rolled-back transaction or use unique data per test.
- Seed randomness or inject the generator.
- Sort before comparing, or compare as sets.
- Stub network calls at the HTTP boundary.
- Use port 0 / unique temp dirs per worker.

Do not add retries or increase timeouts as the fix unless you can show the operation is legitimately slow, and say so explicitly if you do.

Ask before editing non-test source code.

## 4. Verify and report

Rerun the same loop after the fix.

```
Test: <id>
Before: <fail>/<runs> failures (isolated), <fail>/<runs> (with file/package)
Cause: <category> - <explanation> (`file:line`)
Fix: <what changed>
After: <fail>/<runs> failures
Quarantine while in review: yes/no and why
```