Git Bisect skill for Codex

A bisect is finished when it names a single first-bad commit, the check script’s verdict on that commit and its parent has been reproduced by hand, and you can point at the change inside that commit that causes the symptom. A bisect that ends on a range of skipped commits, or on a commit whose diff cannot explain the failure, is not finished.

Preconditions

• A known bad ref (usually HEAD) and a known good ref (tag, release, commit). Verify both with the check script before starting; never trust memory.
• A clean working tree. Bisect checks out other commits. Ask before stashing the user’s work: git stash push -u -m "before bisect".
• A check that runs in seconds to a few minutes and answers only this question. Bisect needs about log2(commits) steps: 1,000 commits is ~10 runs.

git rev-list --count <good>..<bad>      # how many candidates
git log --oneline --first-parent <good>..<bad> | wc -l   # how many merges on the main line

Write the check script

Rules, each learned the hard way:

1. Store it outside the repo (or untracked in $TMPDIR). A tracked script changes or vanishes as bisect checks out old commits.
2. Exit codes carry meaning: 0 good, 1-127 except 125 bad, 125 skip (cannot test this commit), 128+ aborts the whole bisect. Map every failure mode deliberately.
3. Test only the symptom. Running the whole suite turns every unrelated historical failure into a false “bad”. Run one test, or grep for the specific error.
4. Make “cannot build” a skip, not a bad.
5. Make it import the checked-out code. A script outside the repo runs with its own directory on the import path; set PYTHONPATH=., use repo-relative paths, or cd to the repo root explicitly.

#!/bin/sh
# $TMPDIR/bisect-check.sh  -- run from the repo root by `git bisect run`
# 0 = good, 1 = bad, 125 = skip

npm ci --prefer-offline --no-audit --no-fund >/dev/null 2>&1 || exit 125   # only if lockfile changes in range
npm run build >/dev/null 2>&1 || exit 125

out=$(npx vitest run src/cart.test.ts -t "bulk discount" 2>&1); status=$?
[ "$status" -eq 0 ] && exit 0
echo "$out" | grep -q "expected 836 to be 880" && exit 1    # the symptom we are hunting
exit 125                                                     # failed for some other reason

Python variant with a test that does not exist in old commits (inject it from outside):

#!/bin/sh
cp "$TMPDIR/test_regression.py" ./test_bisect_regression.py
trap 'rm -f ./test_bisect_regression.py' EXIT
python3 -c "import billing" 2>/dev/null || exit 125
PYTHONPATH=. python3 -m pytest -q -p no:cacheprovider ./test_bisect_regression.py >/dev/null 2>&1 && exit 0
exit 1

Delete injected files in a trap, or the next checkout may fail on an untracked file that later commits track.

Before bisecting, prove the script on both ends:

chmod +x "$TMPDIR/bisect-check.sh"
git switch --detach <good> && "$TMPDIR/bisect-check.sh"; echo "good ref -> $?"   # expect 0
git switch --detach <bad>  && "$TMPDIR/bisect-check.sh"; echo "bad ref -> $?"    # expect 1
git switch -                                                                  # back to the branch

Run it

git bisect start <bad> <good>
git bisect run "$TMPDIR/bisect-check.sh"
git bisect log > "$TMPDIR/bisect.log"      # keep the evidence
git bisect reset                           # always return to the original branch

Useful variations:

Need	Command
History is full of merges; find the merged PR first	git bisect start --first-parent <bad> <good>
Hunting a change that is not a bug (got faster, output changed)	git bisect start --term-old=before --term-new=after, then git bisect after <bad>, git bisect before <good>; script exits 0 for “before”
Several known-good refs (release branches)	git bisect start <bad> <good1> <good2>
Only commits touching some paths matter	git bisect start <bad> <good> -- src/billing/
Manually mark an untestable commit	git bisect skip <sha> or a range git bisect skip a..b
Resume after a mistake	Edit the saved log, then git bisect reset && git bisect replay "$TMPDIR/bisect.log"

Flaky or performance symptoms

Intermittent failures fool bisect: one lucky pass marks a bad commit good, and the answer is wrong with no warning. Make “good” require repeated passes:

for i in 1 2 3 4 5 6 7 8 9 10; do
  npx vitest run src/sync.test.ts -t "merges offline edits" >/dev/null 2>&1 || exit 1
done
exit 0

If the failure rate is p per run, ten clean runs leave (1-p)^10 chance of a false good: for p = 0.2 that is about 11%. Raise the count until that risk is acceptable.

For performance, compare a median against a threshold with margin, and use the custom terms above so the output reads “first slow commit”, not “first bad”:

ms=$(for i in 1 2 3 4 5; do ./bench.sh; done | sort -n | sed -n 3p)   # median of 5, prints ms
[ "$ms" -lt 450 ] && exit 0   # baseline ~300 ms, regression ~600 ms
exit 1

Interpreting the result

1. Read the culprit: git show --stat <sha> then the full diff of the relevant files.
2. Confirm by hand: run the check on <sha>^ (expect good) and <sha> (expect bad).
3. If the culprit is a merge commit (common with --first-parent), bisect again inside the merged branch: git bisect start <merge> <merge>^1 searches exactly the commits that merge brought in. Or inspect the PR’s commits directly.
4. If bisect ends with “only skipped commits left”, it lists the candidates. Make those commits testable (fix the build in the script, for example by applying a known patch) or inspect them manually.
5. If the culprit’s diff cannot plausibly explain the symptom, suspect the script: flaky check, unrelated failure counted as bad, stale build artefacts, or dependency versions that did not change with the checkout.

Then hand over to systematic-debugging to explain the mechanism and fix it, or to root-cause-analysis if this was an incident.

Hard rules

• Always finish with git bisect reset, even after errors, and restore any stash you made (git stash pop) after confirming with the user.
• Never let the check script modify tracked files without restoring them.
• Do not bisect across commits that need secrets or network the sandbox lacks; mark them skip or ask the user to run that part.

Report

First bad commit: 8c3bbdc "Apply tier discounts cumulatively" (2026-09-14, PR #1412)
Range: v4.2.0 (good) .. HEAD (bad), 312 commits, 9 steps, 1 skipped (broken build)
Check: vitest "bulk discount", symptom-specific grep; verified good on 8c3bbdc^, bad on 8c3bbdc
Cause in diff: src/cart/tiers.ts:41 applies each qualifying tier in sequence
Log saved: $TMPDIR/bisect.log