API Testing skill for Codex

A useful API suite exercises the service over real HTTP the way clients do, covers every response a client must handle (success, 4xx families, auth), checks the body shape and not only the status, creates its own data, and runs headless in CI against any environment by changing one variable.

Inspect first

1. Read AGENTS.md, then find the API description: an OpenAPI file, route definitions, or existing Postman/Bruno/Hurl collections. Reuse what exists.
2. Learn how to start the service locally (compose file, make run) and its health endpoint. Tests against a local server need no network; tests against a remote environment do, and may need sandbox approval in Codex.
3. Identify auth: API key, bearer token from a login endpoint, OAuth client credentials. Credentials come from environment variables or a secrets store, never committed files.
4. Ask before running any suite that writes data against a shared or production environment. Default to local or a dedicated test environment.

Choose the tool

Situation	Tool
Language-agnostic smoke and regression checks, reviewed in PRs as text	Hurl
Team already explores APIs in a GUI and wants the same collection in CI	Bruno + bru run
Python service, or tests need fixtures, loops, DB checks and rich logic	pytest + httpx
Find crashes and spec violations across all endpoints from an OpenAPI file	Schemathesis, in addition to the above

Do not introduce a second API test tool when one is already established.

What every endpoint needs

• Happy path: status, key fields, Content-Type
• Response shape validated against a schema (OpenAPI or JSON Schema), not field by field
• Missing or invalid auth -> 401; valid auth without permission -> 403 (or 404 if the API hides existence)
• Each validation rule -> 400/422 with the documented error format (e.g. RFC 9457 problem details)
• Not found -> 404; conflict or duplicate -> 409
• Pagination: first page, last page, empty page, invalid cursor
• Idempotency for PUT/DELETE and for POST with an idempotency key
• Tenant isolation: user A cannot read or modify user B’s resource
• No stack traces or internal hostnames in error bodies

Hurl

# tests/api/orders.hurl
POST {{base_url}}/api/login
Content-Type: application/json
{
  "email": "{{user_email}}",
  "password": "{{user_password}}"
}
HTTP 200
[Captures]
token: jsonpath "$.access_token"

POST {{base_url}}/api/orders
Authorization: Bearer {{token}}
Content-Type: application/json
{
  "sku": "LAMP-01",
  "quantity": 2
}
HTTP 201
[Captures]
order_id: jsonpath "$.id"
[Asserts]
header "Location" endsWith "/api/orders/{{order_id}}"
jsonpath "$.status" == "pending"
jsonpath "$.total_cents" isInteger
duration < 800

GET {{base_url}}/api/orders/{{order_id}}
HTTP 401

POST {{base_url}}/api/orders
Authorization: Bearer {{token}}
Content-Type: application/json
{
  "sku": "LAMP-01",
  "quantity": 0
}
HTTP 422
[Asserts]
jsonpath "$.errors[0].field" == "quantity"

hurl --test --variables-file env/local.env --report-junit build/hurl.xml tests/api/*.hurl
hurl --test --variable base_url=http://localhost:8080 tests/api/orders.hurl --very-verbose

• Each file runs its entries in order; captures flow forward. Keep one scenario per file so files can run in parallel.
• Put secrets in the variables file outside git or pass --variable from CI secrets.

Bruno

Bruno stores requests as plain-text files in the repo. A request with tests:

meta {
  name: Create order
  type: http
  seq: 2
}

post {
  url: {{baseUrl}}/api/orders
  body: json
  auth: bearer
}

auth:bearer {
  token: {{token}}
}

body:json {
  { "sku": "LAMP-01", "quantity": 2 }
}

assert {
  res.status: eq 201
  res.body.status: eq pending
}

script:post-response {
  bru.setVar("orderId", res.body.id);
}

tests {
  test("total is an integer number of cents", function () {
    expect(Number.isInteger(res.body.total_cents)).to.be.true;
  });
}

npx @usebruno/cli run --env ci --reporter-junit build/bruno.xml       # or: bru run ...
bru run orders --env local --env-var token=$API_TOKEN

• Match the file format already in the collection; newer Bruno versions can also store collections in a YAML format. Check bru run -h for the installed version’s flags.
• Since Bruno CLI v3, scripts run in a safe sandbox by default; requiring npm packages or the filesystem needs --sandbox=developer.

pytest + httpx

# tests/api/conftest.py
import os, httpx, pytest

@pytest.fixture(scope="session")
def base_url() -> str:
    return os.environ.get("API_BASE_URL", "http://localhost:8080")

@pytest.fixture(scope="session")
def client(base_url):
    with httpx.Client(base_url=base_url, timeout=10) as c:
        yield c

@pytest.fixture
def auth_client(client):
    r = client.post("/api/login", json={"email": os.environ["E2E_USER"],
                                        "password": os.environ["E2E_PASSWORD"]})
    r.raise_for_status()
    client.headers["Authorization"] = f"Bearer {r.json()['access_token']}"
    yield client
    client.headers.pop("Authorization", None)

# tests/api/test_orders.py
import pytest
from jsonschema import validate
from .schemas import ORDER_SCHEMA

def test_create_order_returns_location_and_valid_body(auth_client):
    r = auth_client.post("/api/orders", json={"sku": "LAMP-01", "quantity": 2})
    assert r.status_code == 201, r.text
    assert r.headers["Location"].endswith(f"/api/orders/{r.json()['id']}")
    validate(r.json(), ORDER_SCHEMA)

@pytest.mark.parametrize("payload, field", [
    ({"sku": "LAMP-01", "quantity": 0}, "quantity"),
    ({"quantity": 1}, "sku"),
])
def test_rejects_invalid_orders(auth_client, payload, field):
    r = auth_client.post("/api/orders", json=payload)
    assert r.status_code == 422
    assert r.json()["errors"][0]["field"] == field

def test_other_users_order_is_not_visible(auth_client, other_users_order_id):
    assert auth_client.get(f"/api/orders/{other_users_order_id}").status_code == 404

For an in-process ASGI app (FastAPI, Starlette) use httpx.AsyncClient with transport=httpx.ASGITransport(app=app) to test without starting a server; keep at least a smoke run against the real server for middleware and proxies.

Always include r.text in the assertion message for status checks; a bare assert 500 == 201 hides the reason.

Schema-driven fuzzing

With an OpenAPI document, Schemathesis generates requests from it and checks for server errors and responses that violate the schema. Install it (needs network), point it at a local instance, and treat each 5xx it finds as a bug with a reproducible request. Check schemathesis run --help for the installed version’s options rather than copying flags from older posts.

Data and environments

• Tests create the data they need and clean up, or run against a database reset per run.
• Use unique values per run (suffix with a run id) so parallel runs do not collide.
• One environment file per target (local, ci, staging); only the base URL and credentials differ. Production runs, if any, are read-only smoke checks approved by the user.

Gotchas

• Asserting only status codes misses broken bodies; validate the schema.
• Order-dependent suites where test 7 uses an id created by test 3 fail in isolation; chain within a scenario, not across scenarios.
• Timestamps and ids differ per run; assert type or format, not exact values.
• Retries in HTTP clients hide flaky endpoints; disable automatic retries in test clients.
• For contract compatibility between separate services, use $contract-testing; for throughput and latency under load, $load-testing.

Verification

1. Run the suite against a fresh local instance; all pass.
2. Break one rule in the service (return 200 instead of 403) and confirm a test fails.
3. Run the suite twice in a row to prove it does not depend on leftover data.
4. Confirm CI publishes the JUnit report and that no secret appears in logs.