Contract Testing skill for Codex

Contracts are working when a provider cannot merge a change that breaks a known consumer, a consumer cannot deploy expecting something the provider does not offer, and both sides find out in their own CI within minutes, without a shared staging environment.

Inspect first

1. Read AGENTS.md and identify the integration: HTTP JSON, gRPC, GraphQL, or messages.
2. List who consumes the API and who owns each side. Consumer-driven contracts need the consumer team to write tests; a public API with unknown consumers needs schema diffing instead.
3. Check for an existing Pact setup (pacts/ folder, @pact-foundation/pact, pact-python, au.com.dius.pact dependencies) and a broker URL (self-hosted Pact Broker or PactFlow). Broker calls need network and a token.
4. Find the API description (OpenAPI, .proto, GraphQL SDL) and where it lives in git.

Choose the approach

Situation	Approach
Internal services, known consumers, teams can coordinate	Pact consumer-driven contracts
Public or partner API, many unknown consumers	Schema diffing in CI (oasdiff, buf, GraphQL diff) plus a deprecation policy
Provider already has a reliable OpenAPI spec and tests it	Bi-directional contracts (PactFlow) or schema diff plus consumer Pact
gRPC	buf breaking on every PR; Pact plugin only if semantic expectations matter
Async messages (Kafka, SNS)	Pact message pacts, or schema registry compatibility rules

Contract tests check the shape and meaning of interactions. They do not replace provider functional tests, and they are not end-to-end tests.

Pact workflow

- [ ] 1. Consumer writes tests for each interaction it relies on
- [ ] 2. Consumer CI publishes the pact with version = git SHA and branch
- [ ] 3. Provider CI verifies pacts from main and deployed consumer versions
- [ ] 4. Provider publishes verification results
- [ ] 5. Both sides run can-i-deploy before deploying
- [ ] 6. Both sides record-deployment after deploying

1. Consumer test (JavaScript)

import path from "node:path";
import { Pact, Matchers, SpecificationVersion } from "@pact-foundation/pact";
import { OrdersClient } from "../src/orders-client";

const { like, integer, regex, eachLike } = Matchers;

const provider = new Pact({
  consumer: "web-checkout",
  provider: "orders-api",
  dir: path.resolve(process.cwd(), "pacts"),
  spec: SpecificationVersion.SPECIFICATION_VERSION_V4,
});

it("fetches an existing order", async () => {
  await provider
    .addInteraction()
    .given("order 42 exists")
    .uponReceiving("a request for order 42")
    .withRequest("GET", "/orders/42", (b) => {
      b.headers({ Accept: "application/json" });
    })
    .willRespondWith(200, (b) => {
      b.headers({ "Content-Type": "application/json" });
      b.jsonBody({
        id: integer(42),
        status: regex("pending|paid|shipped", "paid"),
        items: eachLike({ sku: like("LAMP-01"), quantity: integer(1) }),
      });
    })
    .executeTest(async (mockServer) => {
      const order = await new OrdersClient(mockServer.url).get(42);
      expect(order.status).toBe("paid");
    });
});

Older pact-js releases expose the same builder as PactV4 with MatchersV3; follow the version installed.

Rules for consumer tests:

• Exercise the real client code (OrdersClient), not raw fetch calls in the test. Otherwise the pact describes what the test does, not what the app does.
• Include only fields the consumer actually reads. Every extra field becomes a constraint the provider cannot change.
• Use type and regex matchers, not literal values, except where the exact value matters.
• One interaction per distinct behavior: found, not found, unauthorized, validation error.
• Provider state names (given) are a shared vocabulary. Agree them with the provider team and keep them declarative (“order 42 exists”), not procedural.

3. Provider verification

import { Verifier } from "@pact-foundation/pact";

await new Verifier({
  provider: "orders-api",
  providerBaseUrl: "http://localhost:8080",
  pactBrokerUrl: process.env.PACT_BROKER_BASE_URL,
  pactBrokerToken: process.env.PACT_BROKER_TOKEN,
  consumerVersionSelectors: [{ mainBranch: true }, { deployedOrReleased: true }],
  enablePending: true,
  publishVerificationResult: process.env.CI === "true",
  providerVersion: process.env.GIT_SHA,
  providerVersionBranch: process.env.GIT_BRANCH,
  stateHandlers: {
    "order 42 exists": async () => {
      await db.orders.insert({ id: 42, status: "paid", items: [{ sku: "LAMP-01", quantity: 1 }] });
    },
  },
}).verifyProvider();

• Run verification against the provider started locally with a test database; stub the provider’s own downstream dependencies, not its handlers.
• enablePending: true stops a brand-new consumer expectation from breaking the provider build before the provider has implemented it.
• Publish results only from CI, never from a developer machine.

5 and 6. Gate deployments

pact-broker can-i-deploy --pacticipant web-checkout --version "$GIT_SHA" --to-environment production
# deploy only if the command exits 0, then:
pact-broker record-deployment --pacticipant web-checkout --version "$GIT_SHA" --environment production

can-i-deploy and record-deployment change what other teams’ pipelines see. Run them from CI, and ask before running record-deployment by hand.

For the full broker setup, webhooks that trigger provider verification when a pact changes, and a GitHub Actions layout for both sides, read references/pact-broker-ci.md.

Schema diffing

Run on every PR that touches an API description, comparing against the main branch.

# OpenAPI: fail on breaking changes
oasdiff breaking origin-main-openapi.yaml openapi.yaml --fail-on ERR

# Protobuf with buf
buf breaking --against '.git#branch=main'

# GraphQL (graphql-inspector)
npx @graphql-inspector/cli diff old-schema.graphql schema.graphql

Extract the main-branch file with git show origin/main:openapi.yaml > /tmp/base.yaml. Tools need installing once (network). Check oasdiff breaking --help for the flag spelling in the installed version.

Breaking for clients, in short:

Change	Breaking?
Remove or rename a response field, endpoint, enum value	yes
Add a required request field or parameter	yes
Narrow a type or tighten validation (max length, pattern)	yes
Add an optional request field, new endpoint, new response field	no
Add an enum value to a response	breaking for clients with exhaustive switches; flag it
Change error status codes or error body format	yes

Gotchas

• Pacts generated from tests that mock the consumer’s own client are worthless; the mock server must be hit by production client code.
• A provider that verifies against latest pacts instead of selectors breaks on consumers’ unmerged branches. Use branch and deployment selectors.
• Provider states that depend on test execution order fail when run alone. Each handler sets up everything it needs, and verification resets data between interactions.
• Contracts that pin exact timestamps, ids or ordering cause false failures; use matchers.
• Without record-deployment, can-i-deploy --to-environment has nothing to compare against and either blocks everything or passes everything.

Verification

1. Consumer test passes and writes a pact file; inspect it and confirm it contains only fields the consumer reads.
2. Provider verification passes locally against that file (pactUrls: ["./pacts/web-checkout-orders-api.json"]).
3. Remove a field the consumer uses from the provider response: verification fails.
4. For schema diffing, introduce a deliberate breaking change on a branch: CI fails.

Output

Report as:

**Contracts:** web-checkout -> orders-api (4 interactions: found, not found, 401, 422)
**Provider states:** "order 42 exists", "no orders exist"
**CI:** consumer publishes on every push; provider verifies main + deployed; can-i-deploy gates both
**Schema checks:** oasdiff breaking on PRs touching openapi.yaml
**Open items:** broker webhook to trigger provider verification on pact change (needs admin)