Node.js Profiling skill for Codex

Done well means: the symptom is classified (CPU-bound, event-loop blocked, waiting on I/O, or memory) before any profiler runs, the profile is captured under realistic load, the hot path is named down to a function and line, and the fix is proven with the same load and the same measurement.

Classify the symptom first

Signal	Likely class	Tool
Process CPU near 100% of one core, latency rises with load	CPU-bound on the main thread	--cpu-prof, flame graph
Event loop delay p99 > ~50 ms, CPU moderate	Blocking sync calls, big JSON, regex, GC pauses	ELU + delay histogram, CPU profile
CPU low, latency high	Waiting on DB, HTTP, pool, locks	Tracing / APM, not a CPU profiler: $database-performance-triage, $latency-budgets
RSS or heap grows steadily	Leak or unbounded cache	Heap snapshots: $memory-leak-hunting
Saw-tooth heap with long pauses	GC pressure from allocation churn	--heap-prof, --trace-gc

Measure the loop before anything else; it is cheap enough for production:

import { monitorEventLoopDelay, performance } from 'node:perf_hooks';

const h = monitorEventLoopDelay({ resolution: 10 });
h.enable();
let last = performance.eventLoopUtilization();
setInterval(() => {
  const elu = performance.eventLoopUtilization(last);
  last = performance.eventLoopUtilization();
  console.log(JSON.stringify({
    elu: +elu.utilization.toFixed(3),
    p50ms: h.percentile(50) / 1e6, p99ms: h.percentile(99) / 1e6, maxms: h.max / 1e6,
  }));
  h.reset();
}, 10_000).unref();

ELU near 1.0 means the loop is saturated: adding concurrency will not help, only less CPU per request or more processes.

Workflow

- [ ] 1. Read AGENTS.md / package.json: Node version, start command, framework
- [ ] 2. Reproduce with a load generator against a production build (NODE_ENV=production)
- [ ] 3. Capture ELU, loop delay, CPU and memory baseline under that load
- [ ] 4. Capture the right profile for the class (below) for 30-60 s of steady load
- [ ] 5. Find the top self-time and top inclusive frames in your code
- [ ] 6. Fix one hotspot; re-run the identical load; compare p50/p99 and req/s

Load: npx autocannon -c 50 -d 60 http://localhost:3000/api/orders (needs network to install). Or reuse the project’s k6 scripts ($load-testing). Never profile against production traffic or a shared environment without the user’s approval.

CPU profile

# Writes a .cpuprofile on exit; open in Chrome DevTools > Performance or speedscope
node --cpu-prof --cpu-prof-dir=./profiles --cpu-prof-interval 500 dist/server.js
# then: run the load, stop with Ctrl+C (SIGINT) so the profile is flushed

# Attach to a running process instead
node --inspect=127.0.0.1:9229 dist/server.js   # then chrome://inspect > Profiler

• Keep --inspect bound to 127.0.0.1. Binding to 0.0.0.0 gives anyone on the network code execution.
• In containers, start with --cpu-prof and stop the process gracefully; kill -9 loses the profile.
• Built-in V8 tick profiler (no DevTools needed): node --prof app.js, then node --prof-process isolate-*.log > ticks.txt.

Flame graph in one step: npx 0x -o dist/server.js (needs network to install). Clinic.js (clinic doctor, clinic flame) gives guided diagnosis but sees little maintenance and may lag new Node releases; prefer --cpu-prof when it misbehaves.

Reading the profile

• Sort Bottom-Up by self time. Frames from node_modules are not exempt: a slow serializer or validator called per request is your problem too.
• Common hot spots: JSON.parse/JSON.stringify on large payloads, synchronous fs.*Sync, crypto.pbkdf2Sync/scryptSync, regex with catastrophic backtracking, Array.prototype.find inside loops (O(n²)), logging with pretty-printing in production, schema compilation per request instead of once.
• (garbage collector) above ~10% of samples means allocation churn: use a heap profile.
• (program) / (idle) dominant means the CPU profiler is the wrong tool; the time is spent waiting.

Heap and GC

node --heap-prof --heap-prof-dir=./profiles dist/server.js   # sampling allocation profile
node --trace-gc dist/server.js 2>&1 | head -50                 # GC events and pauses
node --heapsnapshot-signal=SIGUSR2 dist/server.js              # kill -USR2 <pid> writes a snapshot

Allocation profiles answer “who allocates most”; heap snapshots answer “what is retained”. Snapshots pause the process and can double memory use; never take them on a production instance near its memory limit without the user’s go-ahead.

Fix patterns

Hot path	Fix
Sync crypto, compression, image work	Async APIs (crypto.scrypt, zlib.gzip), or worker_threads pool (Piscina)
Large JSON per request	Paginate, stream (JSON lines), select fewer fields, cache serialised output
Repeated computation on same inputs	Memoise with a bounded LRU; precompute at startup
Validation/schema compile per call	Compile once at module load (Ajv, Zod schemas as constants)
Logging cost	Structured logger (pino) with async transport; no pretty-printing in production
Regex backtracking	Rewrite with anchors and atomic structure; bound input length
Loop saturated after all fixes	More processes (cluster, containers) behind the load balancer

Hard rules

• Profile production builds with production NODE_ENV; dev mode changes frameworks’ hot paths (React SSR, Express view caching).
• Same Node version as production: V8 optimisations differ between majors.
• A CPU profile of an idle process proves nothing. Drive load during the capture.
• Remove --inspect flags and profiling hooks from production start scripts after the investigation unless the team decides to keep them deliberately.

Verification

Re-run the identical load for the same duration and compare:

| Metric | Before | After |
|---|---|---|
| req/s | 1,850 | 3,400 |
| p99 latency | 420 ms | 95 ms |
| ELU | 0.98 | 0.61 |
| loop delay p99 | 180 ms | 22 ms |
| hotspot share | buildReport 41% self | 3% |

If throughput did not change, the fixed function was not on the critical path, or the bottleneck moved (DB, pool); re-classify before the next change.

Report template

## Node profile: <service> (<Node version>, <load profile>)
Class: CPU-bound on main thread (ELU 0.98, loop delay p99 180 ms)
Hotspots (self time, CPU profile 60 s):
1. src/reports/build.ts:88 `formatRows` 41% - JSON stringify per row
2. node_modules/ajv compile 12% - schema compiled per request
Fixes applied: ...   Results: <table above>
Profiles saved: ./profiles/<file>.cpuprofile