Test Plan Writing skill for Codex

A test plan is useful when a tester who has never seen the feature can pick it up and know what to test first, why, with which data and devices, and when to stop. It is short, ranked by risk, and mixes a few scripted checks for paths that must never break with time-boxed exploratory charters for everything else.

Inspect first

1. Read the source material: PRD or spec, acceptance criteria, design files, the PR diff or changed files (git diff --stat origin/main...HEAD), feature flag config, release notes.
2. Read AGENTS.md and existing QA docs for templates, environments and bug tracker conventions; match them.
3. Find what is already automated (unit, integration, E2E) for the area so the plan does not repeat it manually.
4. Collect past defects in the same area (bug tracker search, git log --grep fix -- <path>).
5. Ask the user for missing facts that change the plan: release date, supported platforms and browsers, locales, who tests, and how much time is available.

Build the risk table first

Rate each area by likelihood of failure (complexity, change size, past bugs, new dependencies) and impact (money, data, security, legal, number of users).

Area	What could go wrong	Likelihood	Impact	Priority	Covered by automation?
Promo code at checkout	Discount applied twice; currency rounding	High	High	P1	Unit only
Saved cards	Wrong card charged after deletion	Medium	High	P1	No
Order history filter	Wrong date range in other time zones	Medium	Low	P3	No

Effort follows priority: P1 gets scripted cases plus exploration; P3 gets a charter at most.

Plan structure

# Test plan: <feature / release> (<version>, <date>)
## Scope
In: <features, platforms, integrations>
Out: <explicitly not tested, and why>
## Risks
<risk table>
## Approach
- Automated coverage relied on: <suites>
- Scripted checks: <count, for P1 paths>
- Exploratory sessions: <count x duration, charters below>
- Non-functional: <accessibility pass, performance smoke, security checks>
## Environments and data
<matrix and test accounts>
## Entry criteria
## Exit criteria
## Schedule and owners
## Charters
## Test cases
## Reporting

Environments and data matrix

Pick combinations by usage share and risk, not the full cartesian product. Use pairwise selection when several dimensions interact.

Dimension	Values to cover
Platform / browser	Top 3 by traffic + the oldest supported + Safari on iOS
Account state	New, existing with history, suspended, admin
Locale and time zone	Default, one RTL or long-text locale, UTC+ and UTC- offsets
Data edge cases	Empty, one, many (pagination), max length, unicode, special characters
Network	Fast, slow 3G, offline mid-action
Feature flags	On, off, partial rollout cohort

List test accounts by role and state, never real customer accounts. Keep credentials in the team’s secret store; the plan names the account, not the password.

Exploratory charters

Use the charter form: Explore target with resources to discover information. Time-box each session (45-90 minutes) and record notes, bugs, questions and coverage.

### Charter C1 (P1, 60 min)
Explore promo codes at checkout
with stacked codes, expired codes, currency switches and back-button navigation
to discover ways the discount is applied wrongly or twice.
Notes: <tester fills in> | Bugs: <ids> | Questions: <...> | Coverage: <what was not reached>

Heuristics for generating charters: interrupt the flow (back, refresh, close tab, lose network), repeat the action (double submit, two tabs), boundaries (0, 1, max, max+1), permissions (role changes mid-session), time (expiry, time zones, daylight saving), concurrency (two users edit the same object), and data that looks like code (quotes, emoji, <script>, very long strings).

Scripted test cases

Write them only for P1 paths that must be checked identically every release and are not yet automated. Keep each case atomic and reproducible.

| ID | Title | Preconditions | Steps | Expected |
|---|---|---|---|---|
| TC-07 | Removing a saved card prevents its use | Account `qa-buyer-2` with cards ending 4242 and 1881 | 1. Settings > Payment > remove 1881. 2. Checkout any item. | Only 4242 is offered; API call uses 4242's payment method id |

Flag every scripted case that would be cheap to automate; list them as automation candidates in the plan.

Entry and exit criteria

Entry (testing starts when): build deployed to the test environment, feature flag configured, test data and accounts ready, automated suites green, known issues listed.

Exit (sign-off when):

• All P1 scripted cases pass on the agreed matrix
• All charters run, notes recorded
• No open critical or high bugs; medium bugs triaged with owner decision
• Accessibility pass done for new UI (see accessibility-testing)
• Residual risks written down and accepted by the product owner

Bug reports

**Title:** Promo code applied twice when Pay is double-clicked (web, Chrome 141)
**Severity:** high - customer charged less than price; revenue loss
**Environment:** staging build 2025.10.3, account qa-buyer-1, flag new_checkout=on
**Steps:**
1. Add SKU A1 to cart, apply code SAVE10
2. On the payment step, double-click Pay
**Expected:** one order, 10% discount
**Actual:** two orders created; second has 20% discount
**Evidence:** screen recording, order ids, HAR file, console log
**Frequency:** 4/5 attempts

One bug per report; steps from a clean state; expected versus actual stated plainly.

Verification of the plan

• Every P1 risk maps to at least one scripted case or charter.
• Every acceptance criterion in the spec maps to a check (automated or manual).
• Someone unfamiliar with the feature can run TC-01 from the plan alone; walk through one case to confirm.
• The total estimated effort fits the available time; if not, cut from P3 upwards and list what was cut under Out of scope.

Gotchas

• Hundreds of scripted cases nobody runs are worse than twenty that always run.
• Plans that copy acceptance criteria as test cases miss everything the spec did not anticipate; exploration finds those.
• “Test on all browsers” without a matrix means testing on one.
• Sign-off without written residual risks transfers the risk to whoever asked for sign-off.