Latency Budgets skill for Codex
Split p99 across every hop, then set timeouts. Codex sets an end-to-end target from the user experience, maps the critical path from a real trace, and allocates a p99 budget to each sequential and parallel hop with headroom. It accounts for fan-out tail amplification and queueing, derives timeouts, deadlines and retry limits from the allocations, measures with open-model load, and checks percentiles from raw samples with a bundled script.
curl -fsSL codexskills.dev/install.sh \ | sh -s -- latency-budgetsInstalls to ~/.agents/skills/latency-budgets/ for every project on this machine.
curl -fsSL codexskills.dev/cli.mjs -o codex-skills.mjsnode codex-skills.mjs install latency-budgetsThe CLI also searches, updates and removes skills. Run it with no arguments to pick from a list. Needs Node 20.12+.
curl -fsSL codexskills.dev/install.sh \ | sh -s -- --project latency-budgetsInstalls to .agents/skills/latency-budgets/ in the current repository. Commit it and everyone who runs Codex there gets it.
curl -fsSLO codexskills.dev/downloads/latency-budgets.zipunzip -o latency-budgets.zip -d ~/.agents/skills/The archive holds the skill folder with its generated agents/openai.yaml. Unzip into .agents/skills/ to scope it to a repo.
& ([scriptblock]::Create((irm codexskills.dev/install.ps1))) latency-budgetsPowerShell 5.1 or 7. Installs to %USERPROFILE%\.agents\skills\. Add -Project for the current repo.
104-line POSIX script. No sudo, no telemetry. Read install.sh
Use it in Codex
Start a prompt with $latency-budgets, or pick it from /skills.
You can also just describe the task: Codex reads the description below and loads the playbook when your request matches.
What Codex reads before loading it
Builds end-to-end latency budgets for a user-facing request and splits them across hops - edge, gateway, services, databases, third-party APIs - at p50, p99 and p99.9, accounting for fan-out tail amplification, retries, queueing and coordinated omission, with timeouts and deadlines that fit inside the budget. Includes a stdlib script that computes percentiles from raw samples, checks them against a budget and shows fan-out risk. Use when defining latency targets for a new API or page, when p99 is far worse than p50, when a call graph fans out to many services, or when timeouts and retries are set without a plan.
Use it when
- Defining latency targets for a new API, service or page
- p99 is far worse than p50, especially with fan-out to many services
- Timeouts and retries were set by default rather than from a budget
Use something else when
- You need SLO definitions and burn-rate alerting: use slo-error-budgets
- You need to write the load test scenarios themselves: use load-testing
How to check the result
An open-model load test at target throughput keeps end-to-end p99 within target with each hop inside its allocation, backed by at least 1,000 samples per endpoint.
The mistake it prevents
Averaging p99 values across instances or adding per-hop p99s as if percentiles were additive, which produces budgets that real tails never meet.
Done well means: the end-to-end target comes from the user experience, every hop on the critical path has an allocation at the percentile that matters (usually p99), the allocations add up with headroom, timeouts and retries are derived from the budget rather than defaults, and measurements use correct percentiles from raw data, not averages.
Principles
- Budget at tail percentiles. p50 describes the typical request; users and SLOs feel p99.
- Averages of percentiles are meaningless; percentiles do not add. Budget per hop, then validate end to end with measured distributions.
- Fan-out amplifies tails: if a request waits on N parallel calls, each with a 1% chance
of being slow, the chance the request is slow is
1 - 0.99^N(N = 10 -> ~10%, N = 100 -> ~63%). A per-call p99 becomes the request’s p90 or worse. - Queueing explodes near saturation: latency rises sharply above ~70–80% utilisation.
Budgets assume a utilisation target; capacity planning keeps you there
(
$capacity-planning). - Load generators that wait for responses before sending the next request hide stalls
(coordinated omission). Use open-model load (k6
constant-arrival-rate, wrk2, Gatling open injection) when measuring tails.
Workflow
- [ ] 1. Set the end-to-end target from UX/SLO (e.g. API p99 ≤ 400 ms; page LCP ≤ 2.5 s)
- [ ] 2. Draw the critical path: sequential hops vs parallel fan-out, from a real trace
- [ ] 3. Measure each hop's current p50/p99 (traces, histograms, raw samples)
- [ ] 4. Allocate the budget per hop with 15-25% headroom; find hops that cannot fit
- [ ] 5. Derive timeouts, retry policy and deadlines from the allocations
- [ ] 6. Fix or redesign the hops that do not fit (parallelise, cache, hedge, drop from critical path)
- [ ] 7. Verify end to end under open-model load at target throughput; wire alerts to SLOsAllocation example
Target: GET /checkout/summary p99 ≤ 400 ms at the edge.
| Hop | Type | Measured p99 | Budget p99 | Timeout |
|---|---|---|---|---|
| CDN/edge + TLS to origin | sequential | 25 ms | 30 ms | - |
| Gateway + auth | sequential | 18 ms | 25 ms | 50 ms |
| Checkout service own work | sequential | 40 ms | 50 ms | - |
| Cart, pricing, inventory | parallel fan-out (3) | 60 / 140 / 90 ms | 150 ms (slowest of three) | 180 ms each |
| Postgres (2 queries, sequential) | sequential | 2 x 30 ms | 70 ms | 100 ms statement timeout |
| Serialisation + response | sequential | 10 ms | 15 ms | - |
| Total | 340 ms + 60 ms headroom | client deadline 400 ms |
Rules applied: sequential hops add; parallel hops cost the slowest member (at the request percentile, which is worse than any single member’s p99); headroom absorbs variance and growth.
Measuring
Raw samples beat pre-aggregated averages. Export durations from access logs, traces or load-test output and run:
python3 scripts/latency_report.py samples.txt --budget p95=250 --budget p99=400
python3 scripts/latency_report.py access.csv --csv-column duration_ms --budget p99=400 --fanout 3
k6 run --summary-export=summary.json script.js # k6 thresholds can also enforce p(99)The script prints count, mean, p50/p90/p95/p99/p99.9/max (nearest-rank), exits 1 when
a budget is exceeded, warns when there are too few samples for tail percentiles (under
1,000), and with --fanout N shows what share of requests would hit at least one call
slower than each per-call percentile.
- Prometheus histograms:
histogram_quantile(0.99, sum by (le) (rate(http_server_request_duration_seconds_bucket[5m]))). Accuracy depends on bucket boundaries; put boundaries near your budget values. - Never average p99s across instances or time windows; aggregate the histograms or raw data, then compute the percentile.
Turning budgets into timeouts and retries
- Each hop’s timeout ≥ its p99 budget (or p99.9) so normal slow calls succeed, but well below the caller’s remaining budget so a retry still fits.
- Propagate deadlines, not fixed timeouts: gRPC deadlines, or a request-scoped deadline passed downstream (remaining = deadline - now - local overhead). Stop work when the deadline passes.
- Retries: at most one, only for idempotent calls, only if the remaining budget allows it, with jitter and a retry budget (e.g. ≤ 10% extra load). Retries at every layer multiply (3 layers x 3 attempts = 27 calls).
- Hedged requests (send a second request after the p95 delay, take the first answer) cut tails for idempotent reads at a few percent extra load.
- Timeout and retry mechanics (circuit breakers, bulkheads) belong to
$resilience-patterns.
Fixing hops that do not fit
| Problem | Options |
|---|---|
| Sequential calls that are independent | Parallelise |
| Wide fan-out with bad tail | Reduce N (batch endpoints), hedge, tolerate partial results, cache |
| One slow dependency on the critical path | Async it (queue, precompute), cache with TTL, serve stale |
| DB hop over budget | $database-performance-triage |
| GC or runtime pauses in tail | Runtime tuning ($jvm-tuning, $go-pprof, $node-profiling) |
| Cross-region hop | Co-locate, or move the call off the request path |
| High utilisation queueing | Scale out, shed load, rate-limit low-priority traffic |
Hard rules
- Measure at the same point the budget is defined (edge vs service) and say which.
- Report sample counts with tail percentiles; p99.9 of 500 samples is a single request.
- Budget changes that alter production timeouts or retry counts are behavioural changes; roll out gradually and get the owning team’s agreement.
- Load tests against shared or production environments need explicit approval.
Verification
- Open-model load test at target throughput: end-to-end p99 within target, each hop within allocation (from traces), error rate unchanged.
- The script’s budget checks pass on samples from the test, with ≥ 1,000 samples per endpoint for p99 claims.
- SLO alerts (
$slo-error-budgets) reference the same percentile and threshold.
Report template
## Latency budget: <request> (target p99 <= 400 ms at edge, 600 rps)
Critical path (from trace): edge -> gateway -> checkout -> [cart | pricing | inventory] -> postgres
| Hop | Budget p99 | Measured p99 | Timeout | Status |
|---|---|---|---|---|
| pricing | 150 ms | 210 ms | 180 ms | over: cache price lists (TTL 60 s) |
End to end (k6 constant-arrival-rate 600 rps, 10 min, 360k samples): p50 120, p99 470 ms -> after fixes p99 360 ms
Fan-out risk: 3 parallel calls, per-call p99 => ~3% of requests hit a slow call
Retries: one retry for cart only (idempotent), only if > 150 ms of budget remainsWhat the zip contains
Codex reads bundled files only when a step points to them. Every one is shown here, so you can review it before installing.
agents/openai.yaml
Gives the Codex app a display name, a one-line description and a starter prompt for this skill.
interface:
display_name: "Latency Budgets"
short_description: "Split p99 across every hop, then set timeouts"
default_prompt: "$latency-budgets Our checkout API p50 is 80 ms but p99 is 1.4 s, build a latency budget and find the hop that blows it"scripts/latency_report.py
#!/usr/bin/env python3
"""Percentiles from raw latency samples, checked against a latency budget.
Standard library only. Reads one number per line (milliseconds) from a file or stdin,
or one column of a CSV with --csv-column. Prints count, mean, p50/p90/p95/p99/p99.9/max,
and optionally:
--budget p95=300 --budget p99=800 fail (exit 1) if a percentile exceeds its budget
--fanout 20 show how per-call tail latency compounds when one
request waits for N parallel calls
Usage:
python3 latency_report.py samples.txt --budget p99=500
python3 latency_report.py access.csv --csv-column duration_ms --budget p95=250 --fanout 10
some_command | python3 latency_report.py -
Exit: 0 within budget, 1 budget exceeded, 2 bad input.
"""
from __future__ import annotations
import argparse
import csv
import math
import statistics
import sys
PCTS = [50, 90, 95, 99, 99.9]
def percentile(sorted_vals: list[float], p: float) -> float:
"""Nearest-rank percentile: the smallest value with at least p% of samples at or below it."""
if not sorted_vals:
raise ValueError("no samples")
rank = max(1, math.ceil(p / 100 * len(sorted_vals)))
return sorted_vals[rank - 1]
def read_samples(path: str, column: str | None) -> list[float]:
fh = sys.stdin if path == "-" else open(path, newline="", encoding="utf-8")
vals: list[float] = []
with fh:
if column:
for row in csv.DictReader(fh):
try:
vals.append(float(row[column]))
except (KeyError, TypeError, ValueError):
continue
else:
for line in fh:
line = line.strip()
if not line or line.startswith("#"):
continue
try:
vals.append(float(line))
except ValueError:
continue
return vals
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("path", help="file with samples in ms, or - for stdin")
ap.add_argument("--csv-column", help="read this column from a CSV with a header row")
ap.add_argument("--budget", action="append", default=[], help="e.g. p95=300 (ms); repeatable")
ap.add_argument("--fanout", type=int, help="number of parallel calls one request waits for")
a = ap.parse_args()
vals = sorted(read_samples(a.path, a.csv_column))
if len(vals) < 2:
print("need at least 2 numeric samples", file=sys.stderr)
return 2
res = {f"p{p:g}": percentile(vals, p) for p in PCTS}
res["max"] = vals[-1]
print(f"samples={len(vals)} mean={statistics.fmean(vals):.1f} ms")
for k, v in res.items():
print(f" {k:>6}: {v:10.1f} ms")
if len(vals) < 1000:
print(" note: fewer than 1,000 samples; p99 and above are unreliable")
failed = 0
for b in a.budget:
key, _, limit = b.partition("=")
key = key.strip().lower()
if key not in res or not limit:
print(f"bad budget '{b}'; use one of {', '.join(res)}=<ms>", file=sys.stderr)
return 2
ok = res[key] <= float(limit)
failed += not ok
print(f"budget {key} <= {float(limit):g} ms: {'ok' if ok else 'EXCEEDED'} ({res[key]:.1f} ms)")
if a.fanout:
n = a.fanout
print(f"\nfan-out {n}: chance a request waits on at least one call slower than ...")
for p in (95, 99, 99.9):
q = 1 - (p / 100) ** n
print(f" per-call p{p:g} ({res[f'p{p:g}']:.1f} ms): {q * 100:5.1f}% of requests")
return 1 if failed else 0
if __name__ == "__main__":
sys.exit(main())
About this skill
Written and maintained by the Codex Skills editors to the same editorial standard as every skill here: a selective description, a real workflow, known pitfalls and a way to verify the work. First published , last updated . MIT licensed. Something wrong or out of date? Tell us.