codexskills.dev

CI Speedup skill for Codex

Measure the critical path, then cut it. Codex starts by timing the slowest required checks and reading JUnit reports, then applies the fix for the real bottleneck: lockfile-keyed package caches, BuildKit layer caching, timing-balanced shards for Playwright, Vitest, Jest or pytest, and Nx or Turborepo affected builds. A bundled script finds slow tests and proposes shards, and every change is proven with before and after numbers and equal test counts.

Install

curl -fsSL codexskills.dev/install.sh \  | sh -s -- ci-speedup

Installs to ~/.agents/skills/ci-speedup/ for every project on this machine.

Download ci-speedup.zip

104-line POSIX script. No sudo, no telemetry. Read install.sh

Use it in Codex

Start a prompt with $ci-speedup, or pick it from /skills. You can also just describe the task: Codex reads the description below and loads the playbook when your request matches.

What Codex reads before loading it

Cuts CI wall-clock time by measuring the critical path first, then applying the fix that matches the bottleneck - dependency and Docker layer caching, job parallelism, timing-based test sharding for pytest, Jest, Vitest and Playwright, affected-only builds with Nx, Turborepo or path filters, cancelling superseded runs, and bigger runners. Use when CI takes too long, PR feedback is slow, a test suite needs splitting across machines, caches miss, or a monorepo rebuilds everything on every change.

Use it when

  • PR feedback takes longer than about 10 minutes
  • One test job dominates the pipeline duration
  • Dependency installs or Docker builds start cold every run
  • A monorepo rebuilds and tests everything on each change

Use something else when

How to check the result

Required-check p50 drops on the same commit across repeated runs, the second run reports cache hits, and JUnit totals show the same number of tests executed.

The mistake it prevents

Sharding or filtering tests and silently running fewer of them, so the pipeline looks faster because it checks less.

~/.agents/skills/ci-speedup/SKILL.md

--- name: ci-speedup description: Cuts CI wall-clock time by measuring the critical path first, then applying the fix that matches the bottleneck - dependency and Docker layer… license: MIT ---

Fast CI is measured, not guessed: you know the critical path of the slowest required run, each change targets the biggest segment on it, and you prove the gain with before and after numbers on the same commit. Target for PR feedback: required checks green in under 10 minutes at p50; under 5 is excellent.

Measure before changing anything

  1. Read AGENTS.md and the CI config to find the required checks (the ones that block merge).
  2. Pull recent run durations (network, gh authenticated):
gh run list --workflow ci.yml --branch main --limit 20 --json databaseId,conclusion,createdAt,updatedAt
gh run view <run-id> --json jobs --jq '.jobs[] | [.name, .startedAt, .completedAt] | @tsv'

On GitLab, use the pipeline’s job durations in the UI or glab ci view. 3. Draw the critical path: queue wait, setup (checkout, toolchain, install), build, test, upload. Write the numbers down; this is the baseline table for the final report. 4. Find the slowest tests from JUnit XML: python3 scripts/junit_timings.py reports/ --top 15 --shards 4 (offline, standard library). It prints the slowest tests and files and a timing-balanced split into N shards.

Pick the fix for the bottleneck

Bottleneck on the critical pathFirst fix
Queue wait before jobs startMore or bigger runners, fewer jobs per run, cancel superseded runs
Dependency install (npm ci, pip install)Cache the package manager’s download cache keyed on the lockfile
Docker buildBuildKit layer cache (cache-from/cache-to), reorder Dockerfile so deps install before source copy
Single long test jobShard by timing across N parallel jobs
Every job rebuilds the same thingBuild once, pass artifacts, or use a remote build cache
Monorepo runs everythingAffected-only: Nx, Turborepo, Bazel, or path filters
Many short jobs each paying setup costMerge them; setup can exceed the work
Slow single testsFix the test (sleeps, real network, per-test DB setup)

Change one thing at a time and re-measure; caches and sharding interact.

Caching

  • Cache the package manager store, not node_modules: restoring node_modules breaks on native modules and different OS images.
- uses: actions/setup-node@v4
  with:
    node-version: 22
    cache: npm                  # keyed on package-lock.json automatically
- uses: actions/setup-python@v5
  with:
    python-version: "3.13"
    cache: pip
- uses: astral-sh/setup-uv@v6
  with:
    enable-cache: true
  • Docker with Buildx on GitHub Actions:
- uses: docker/build-push-action@v6
  with:
    context: .
    cache-from: type=gha
    cache-to: type=gha,mode=max
  • Key caches on lockfiles plus OS and tool version; include a restore fallback prefix so a lockfile change still gets a warm partial cache.
  • GitHub Actions cache is 10 GB per repo by default with LRU eviction; a dozen large caches evict each other and hit rate collapses. Check usage with gh cache list --sort size_in_bytes.
  • A cache restore slower than the install it replaces is a loss. Measure both.

Parallelism and test sharding

Shard by recorded timing, not by file count; file-count splits leave one shard running 3x longer than the others.

RunnerBuilt-in sharding
Playwrightnpx playwright test --shard=2/4, merge with npx playwright merge-reports
Vitestvitest run --shard=2/4
Jestjest --shard=2/4 (Jest 28+)
pytestpytest-xdist (-n auto) within a machine; pytest-split with stored durations across machines
Gogo test parallelizes packages; split packages across jobs with go list ./...
jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - run: npx vitest run --shard=${{ matrix.shard }}/4 --reporter=junit --outputFile=junit-${{ matrix.shard }}.xml
  • Within one machine, use all cores first (-n auto, Jest workers) before adding machines; each extra machine pays the full setup cost.
  • Stop adding shards when the setup time is more than about a third of the shard duration.
  • Tests that share a database or port fail when parallelized; give each worker its own schema or port via the worker index (PYTEST_XDIST_WORKER, VITEST_POOL_ID).
  • For a static split from timings, use the script’s --shards N --json output to write a file list per shard and commit it; refresh it monthly.

Affected-only work

Repo shapeTool
JS/TS monorepo, Nxnpx nx affected -t lint test build --base=origin/main --head=HEAD
JS/TS monorepo, Turboreponpx turbo run test --filter="...[origin/main]" plus remote cache
Polyglot, largeBazel or Pants with remote cache
Few independent foldersPath filters, with an always-running aggregator job for the required check
  • Affected detection needs history: actions/checkout with fetch-depth: 0, or at least the merge base.
  • On main, run everything (or rely on a remote cache) so a missed dependency edge is caught before release.
  • Required checks plus path filters strand PRs (the check never reports). Put the filter inside the job, or add a final job with if: always() that every PR runs and that fails when any needed job failed.

Cheap wins

  • Cancel superseded PR runs: concurrency: { group: ci-${{ github.ref }}, cancel-in-progress: true } (never on main or deploy workflows).
  • Shallow checkout (fetch-depth: 1) when history is not needed.
  • Skip CI for docs-only changes with filters that still satisfy required checks.
  • Run lint and type-check in parallel with tests, not as a prerequisite stage.
  • Bigger runners: a 4-core runner often halves test time for less than double the cost; check the user’s billing plan before switching.
  • timeout-minutes on every job so a hung test cannot hold a runner for 6 hours.

Verification

  • Re-run the same commit before and after (or compare 10 runs each) and report p50 and worst-case duration of the required checks.
  • Confirm the same number of tests executed: sharding and affected-only filters can drop tests silently. Compare test counts in JUnit totals.
  • Confirm cache hit on the second run (cache-hit output or log line).
  • If a change does not save at least 10% on the critical path, revert it; complexity is a cost.

Output template

## CI speedup: <repo>
| Segment | Before | After | Change |
|---|---|---|---|
| Install | 2m40s | 35s | npm cache on lockfile |
| Test | 11m10s | 3m20s | 4 timing-balanced shards |
| Required checks p50 | 16m | 6m | |
Tests executed: 2,431 before / 2,431 after
Cost impact: <runner minutes per run before/after>
Follow-ups: <slow tests to fix, remote cache to evaluate>

Gotchas

  • Caching build outputs keyed only on the branch serves stale artifacts; key on source hashes.
  • Parallel shards each upload coverage; merge reports or coverage thresholds fail per shard.
  • fail-fast: true (the matrix default) cancels sibling shards and hides how many failed.
  • Retrying flaky tests makes CI look faster on average and slower at p95; fix flakes separately (see flaky-test-triage).

What the zip contains

Codex reads bundled files only when a step points to them. Every one is shown here, so you can review it before installing.

agents/openai.yamlgenerated from this page

Gives the Codex app a display name, a one-line description and a starter prompt for this skill.

interface:
  display_name: "CI Speedup"
  short_description: "Measure the critical path, then cut it"
  default_prompt: "$ci-speedup Our CI takes 25 minutes per PR, find where the time goes and cut it"
scripts/junit_timings.py89 lines, 3.0 KB 
#!/usr/bin/env python3
"""Summarise JUnit XML timings and propose timing-balanced test shards.

Usage:
  python3 junit_timings.py <file-or-dir> [--top 15] [--shards 4] [--json]

Reads every *.xml under the path, aggregates testcase time per test and per file
(uses the 'file' attribute, else the classname), prints the slowest entries and a
greedy longest-first split of files into N shards. Exits 1 if no testcases found.
"""
import argparse
import json
import sys
import xml.etree.ElementTree as ET
from collections import defaultdict
from pathlib import Path


def load(paths):
    tests = []
    for p in paths:
        try:
            root = ET.parse(p).getroot()
        except ET.ParseError as exc:
            print(f"warn: skipping {p}: {exc}", file=sys.stderr)
            continue
        for tc in root.iter("testcase"):
            try:
                t = float(tc.get("time") or 0)
            except ValueError:
                t = 0.0
            group = tc.get("file") or tc.get("classname") or "(unknown)"
            tests.append((f"{group}::{tc.get('name', '?')}", group, t))
    return tests


def split(files, n):
    shards = [[0.0, []] for _ in range(n)]
    for name, secs in sorted(files.items(), key=lambda kv: -kv[1]):
        target = min(shards, key=lambda s: s[0])
        target[0] += secs
        target[1].append(name)
    return shards


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("path")
    ap.add_argument("--top", type=int, default=15)
    ap.add_argument("--shards", type=int, default=0)
    ap.add_argument("--json", action="store_true", help="print shard file lists as JSON")
    a = ap.parse_args()

    root = Path(a.path)
    paths = [root] if root.is_file() else sorted(root.rglob("*.xml"))
    tests = load(paths)
    if not tests:
        print(f"error: no testcases found under {root}", file=sys.stderr)
        return 1

    per_file = defaultdict(float)
    for _, group, t in tests:
        per_file[group] += t
    total = sum(t for *_, t in tests)

    if a.json and a.shards:
        out = [{"shard": i + 1, "seconds": round(s[0], 2), "files": s[1]}
               for i, s in enumerate(split(per_file, a.shards))]
        print(json.dumps(out, indent=2))
        return 0

    print(f"{len(tests)} tests in {len(per_file)} files, {total:.1f}s total ({len(paths)} report files)\n")
    print(f"Slowest {a.top} tests:")
    for name, _, t in sorted(tests, key=lambda x: -x[2])[: a.top]:
        print(f"  {t:8.2f}s  {name}")
    print(f"\nSlowest {a.top} files:")
    for name, t in sorted(per_file.items(), key=lambda kv: -kv[1])[: a.top]:
        print(f"  {t:8.2f}s  {100 * t / total if total else 0:5.1f}%  {name}")
    if a.shards:
        shards = split(per_file, a.shards)
        longest = max(s[0] for s in shards)
        print(f"\nBalanced split into {a.shards} shards (critical path {longest:.1f}s vs {total:.1f}s serial):")
        for i, (secs, names) in enumerate(shards, 1):
            print(f"  shard {i}: {secs:7.1f}s  {len(names)} files")
    return 0


if __name__ == "__main__":
    sys.exit(main())

About this skill

Written and maintained by the Codex Skills editors to the same editorial standard as every skill here: a selective description, a real workflow, known pitfalls and a way to verify the work. First published , last updated . MIT licensed. Something wrong or out of date? Tell us.

Related skills

All 40 DevOps & Infrastructure skills
esc