Test Impact Analysis skill for Codex

Test selection is done well when pull requests run a small, correct subset quickly, every change that could affect everything (dependencies, config, shared test helpers) still runs everything, the full suite runs on main so misses are caught within hours, and the team knows the miss rate instead of assuming it is zero.

Inspect first

1. Read AGENTS.md, CI workflows and the repo layout: single package, multi-module build, or monorepo with a tool (Nx, Turborepo, Bazel, Pants, Gradle, Maven reactor, Go modules).
2. Measure the current state: suite duration p50/p90, how much is tests versus setup, and whether sharding or caching is already in place (ci-speedup covers those).
3. Check how CI gets the base commit. Selection needs history: actions/checkout with fetch-depth: 0 (or enough depth to reach the merge base), and git fetch origin main.
4. Identify hidden coupling that static graphs miss: dynamic imports, reflection, DI containers, config files read at runtime, code generation, shared fixtures and snapshots, database migrations.

Choose the mechanism

Repo type	Mechanism	Command
Nx monorepo	Project graph	npx nx affected -t test --base=origin/main --head=HEAD
Turborepo	Package graph + git	npx turbo run test --affected (Turbo 2.1+), or --filter='...[origin/main]'
Bazel	Target graph, remote cache	bazel test //... already skips cached targets; for explicit selection bazel query 'kind(".*_test", rdeps(//..., set(<changed file labels>)))' or bazel-diff
Pants	Dependency inference	pants --changed-since=origin/main --changed-dependents=transitive test
Maven multi-module	Reactor	mvn -pl <changed modules> -amd test (also-make-dependents)
Gradle multi-project	Task graph + build cache	./gradlew test with build cache on; per-project :module:test plus dependents
Go module	Package graph + test cache	go test caches unchanged packages; compute reverse deps with go list
Single JS/TS package, Jest	Module graph	jest --changedSince=origin/main; jest --findRelatedTests <files>
Single JS/TS package, Vitest	Module graph	vitest run --changed origin/main; vitest related <files> --run
Python	Coverage-based mapping	pytest --testmon (stores .testmondata; persist it between CI runs)
Very large suites, any stack	Predictive or coverage-based selection services	Evaluate with a shadow period before trusting

Prefer the build tool’s own graph. Hand-written path maps drift as code moves.

Always-run triggers

Run the full suite (or the whole affected workspace) when any of these change:

• Lockfiles and manifests: package-lock.json, pnpm-lock.yaml, yarn.lock, poetry.lock, uv.lock, go.sum, Cargo.lock, pom.xml parents, Gradle version catalogs
• Tooling and CI config: .github/workflows/**, tsconfig*.json, babel/vite/jest/ vitest config, pytest.ini/pyproject.toml test sections, Dockerfile, .nvmrc, .tool-versions
• Shared test infrastructure: fixtures, factories, test utilities, global setup, snapshots shared across packages
• Database migrations and schema files
• Environment and runtime config read at startup

# GitHub Actions: decide between full and affected runs
- uses: actions/checkout@v4
  with: { fetch-depth: 0 }
- id: scope
  run: |
    base=$(git merge-base origin/${{ github.base_ref || 'main' }} HEAD)
    if git diff --name-only "$base" HEAD | grep -Eq '(^|/)(package-lock\.json|pnpm-lock\.yaml|uv\.lock|go\.sum)$|^\.github/|(^|/)(tsconfig.*\.json|vitest\.config\.[jt]s)$|^db/migrations/|^test/(fixtures|support)/'; then
      echo "mode=full" >> "$GITHUB_OUTPUT"
    else
      echo "mode=affected" >> "$GITHUB_OUTPUT"
      echo "base=$base" >> "$GITHUB_OUTPUT"
    fi
- if: steps.scope.outputs.mode == 'full'
  run: npx vitest run
- if: steps.scope.outputs.mode == 'affected'
  run: npx vitest run --changed ${{ steps.scope.outputs.base }}

Go reverse dependencies

# packages whose import graph includes a changed package
changed=$(git diff --name-only "$base" HEAD -- '*.go' | xargs -r -n1 dirname | sort -u | sed 's|^|./|')
pkgs=$(for d in $changed; do go list "$d" 2>/dev/null; done)
go list -f '{{.ImportPath}} {{join .Deps " "}}' ./... \
  | awk -v want="$(echo $pkgs)" 'BEGIN{n=split(want,w," ")} {for(i=1;i<=n;i++) if (index(" "$0" ", " "w[i]" ")) {print $1; break}}' \
  | xargs -r go test

go test ./... with a warm test cache already skips packages whose inputs did not change; persist $(go env GOCACHE) in CI before writing custom selection.

CI path filters

Workflow-level paths: filters skip whole workflows. If that workflow provides a required status check, a skipped run leaves the check pending and blocks merges. Prefer one workflow that always runs and decides inside (as above), or job-level filters with a final aggregator job that always reports.

Safety nets

• Full suite on every merge to main (or merge queue), with failures alerting the author
• Nightly full run including slow and quarantined tests
• Fallback to full run if the selection step errors, or if the diff touches more than a threshold (for example 25% of packages)
• Manual override: a PR label or commit trailer ([full-ci]) forces the full suite
• Merge queue runs the full suite if main must never break

Measure the miss rate

A miss is a test that the full run on main shows failing for a commit whose PR run did not select it.

1. Log the selected test set per PR run (as an artifact).
2. When main’s full run fails, check whether the failing tests were in the PR’s selection.
3. Track misses per month and investigate each: usually a hidden dependency to add to always-run triggers or the build graph.

Roll out in shadow mode first: run selection and the full suite side by side for one or two weeks, compare, then switch PRs to selection.

Verification

• A change to a leaf module runs only its tests and dependents; confirm in CI logs.
• A lockfile or CI config change triggers the full suite.
• Introduce a breaking change in a shared utility on a test branch; the selection includes the tests that fail.
• Report time saved: p50 PR duration before and after.

Output template

## Test selection setup: <repo>
Mechanism: <tool + command>. Base: merge-base with origin/main (fetch-depth 0).
Always-run triggers: <list>. Fallbacks: <errors, large diffs, label override>.
Safety nets: full suite on <main/merge queue>, nightly at <time>.
Shadow results: <n> PRs, selected <x>% of tests on average, misses <m> (<causes>).
PR duration p50: <before> -> <after>.

Gotchas

• Shallow clones make --changedSince and affected commands compare against nothing and either run everything or nothing; always fetch the base.
• Jest --onlyChanged compares against uncommitted changes only; in CI it selects nothing.
• Coverage-based maps (testmon) go stale after refactors and environment changes; refresh them on main regularly.
• Static graphs miss runtime coupling: string-based imports, plugin registries, SQL in migration files, feature flag config. Add those paths to the always-run list.