Memory Leak Hunting skill for Codex

Gives the Codex app a display name, a one-line description and a starter prompt for this skill.

interface:
  display_name: "Memory Leak Hunting"
  short_description: "Snapshot, repeat, compare, follow the retainer"
  default_prompt: "$memory-leak-hunting Our Node service gets OOM-killed every 8 hours, find the memory leak"

# Runtime recipes for leak hunting

Read only the section for the runtime that leaks. Every recipe follows the same shape:
force GC, capture, repeat the action, capture again, compare, follow retainers.

## Browser (Chrome DevTools)

1. Open the page in a clean profile (extensions allocate too). DevTools > Memory.
2. Click the trash icon (collect garbage), take a **Heap snapshot** (A).
3. Repeat the action N times (navigate in the SPA and back, open/close a modal). GC.
   Snapshot B. Repeat N times. GC. Snapshot C.
4. Select C, Summary view, filter dropdown "Objects allocated between Snapshot A and
   Snapshot B". Sort by `# New` or Retained Size.
5. Type `Detached` in the class filter to list DOM nodes no longer in the document but
   still referenced from JS. Recent Chrome versions also offer a "Detached elements"
   profile type that lists them directly.
6. Click an instance; the Retainers pane shows the path. Look for your module names,
   event listener arrays, closures (`(closure)` / `system / Context`) and framework
   stores.

Other views:
- **Allocations on timeline**: blue bars that stay blue after GC are retained
  allocations; select a bar to see what was allocated at that moment.
- **Performance panel with "Memory" checked**: JS heap, DOM node and listener counts over
  time. Rising node or listener counts across repeated actions confirm a DOM leak quickly.

Automate with memlab (Meta, open source; needs network to install and Puppeteer's
Chromium):

```js
// scenario.js
module.exports = {
  url: () => 'http://localhost:5173/orders',
  action: async (page) => { await page.click('[data-testid="open-drawer"]'); },
  back: async (page) => { await page.click('[data-testid="close-drawer"]'); },
};
```

```bash
npx memlab run --scenario scenario.js     # reports leaked objects with retainer traces
```

Framework notes: React effects must return cleanup for subscriptions and timers;
Angular components should unsubscribe (`takeUntilDestroyed`); Vue `onUnmounted` for
manual listeners. Third-party widgets (maps, charts, editors) usually expose a `destroy()`
method that must be called on unmount.

## Node.js

```bash
node --inspect=127.0.0.1:9229 dist/server.js            # chrome://inspect > Memory tab
node --heapsnapshot-signal=SIGUSR2 dist/server.js        # kill -USR2 <pid> writes Heap.*.heapsnapshot
node --heapsnapshot-near-heap-limit=2 dist/server.js     # up to 2 snapshots as heap nears the limit
node --expose-gc dist/server.js                          # allows global.gc() in a debug route or script
```

```js
import v8 from 'node:v8';
global.gc?.();
const file = v8.writeHeapSnapshot();   // synchronous; blocks the event loop
console.log('snapshot', file, process.memoryUsage());
```

- Load `.heapsnapshot` files in Chrome DevTools Memory panel (Load button) and use the
  Comparison view between two snapshots.
- `process.memoryUsage()`: `heapUsed` growth = JS objects; `external`/`arrayBuffers` growth =
  Buffers and native-backed memory; `rss` growth with flat heap = native or fragmentation.
- Common Node leaks: EventEmitter listeners (watch for `MaxListenersExceededWarning`),
  per-request data pushed into module arrays, `setInterval` in request handlers, unbounded
  in-memory caches (`Map` keyed by user or URL), unconsumed HTTP response bodies with
  `fetch` (always read or cancel the body), AsyncLocalStorage stores held by long-lived
  callbacks.
- Snapshot size is roughly heap size; ensure disk space and memory headroom first.

## JVM (Java, Kotlin, Scala)

```bash
jcmd <pid> GC.class_histogram > histo-1.txt     # quick per-class instance counts (triggers full GC)
# ... repeat the action N times ...
jcmd <pid> GC.class_histogram > histo-2.txt
diff <(head -40 histo-1.txt) <(head -40 histo-2.txt)

jcmd <pid> GC.heap_dump /tmp/app.hprof          # live objects; large file, pauses the JVM
```

Startup flags worth having in every service:
`-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/dumps/`.

Analyse `.hprof` in Eclipse MAT (desktop app):
- **Leak Suspects** report for a first guess.
- **Dominator tree**: objects retaining the most memory.
- On a suspect: right click > Path to GC Roots > exclude weak/soft references.
- Typical roots: static collections, caches without eviction, `ThreadLocal` values in
  pooled threads, listeners registered on singletons, classloaders kept alive after
  redeploy (Metaspace growth).

Low-overhead option in production, JFR old-object sampling:

```bash
jcmd <pid> JFR.start name=leak settings=profile
# ... let it run through several leak cycles ...
jcmd <pid> JFR.dump name=leak filename=/tmp/leak.jfr path-to-gc-roots=true
jfr print --events jdk.OldObjectSample /tmp/leak.jfr | head -100
```

Native (off-heap) growth: start with `-XX:NativeMemoryTracking=summary`, then
`jcmd <pid> VM.native_memory baseline` and later `jcmd <pid> VM.native_memory summary.diff`.
Direct ByteBuffers, thread stacks (too many threads) and Metaspace are the usual sources.

## Python

```python
import gc, tracemalloc
tracemalloc.start(25)
gc.collect(); s1 = tracemalloc.take_snapshot()
for _ in range(50): handle_request(sample)
gc.collect(); s2 = tracemalloc.take_snapshot()
for stat in s2.compare_to(s1, "traceback")[:5]:
    print(stat); print("\n".join(stat.traceback.format()))
```

```python
import objgraph                     # pip install objgraph (graphviz needed for images)
objgraph.show_growth(limit=15)      # call twice around the repeated action
objgraph.show_backrefs(objgraph.by_type("Session")[-1], max_depth=6, filename="refs.png")
```

```bash
PYTHONMALLOC=malloc memray run -o leak.bin app.py
memray flamegraph --leaks leak.bin      # allocations never freed by the end of the run
memray attach <pid>                      # live process; needs gdb or lldb and ptrace rights
```

Common Python leaks: module-level dicts/lists used as caches, `functools.lru_cache` with
`maxsize=None` on methods (also retains `self`), reference cycles involving objects with
`__del__` in old code, C extensions not releasing memory, Django `DEBUG=True` storing
every query in `connection.queries`, Celery workers accumulating state (mitigate with
`--max-tasks-per-child` while fixing the cause).

## Go

```bash
go tool pprof -sample_index=inuse_space -diff_base=heap-1.pprof heap-2.pprof
curl -s 'http://127.0.0.1:6060/debug/pprof/goroutine?debug=1' | head -40
```

Capture `/debug/pprof/heap` twice, several minutes apart under steady load, and diff
`inuse_space`. Growing goroutine counts are the most common Go leak (each holds its
stack and everything it references): look for goroutines parked on channel operations
with no `ctx.Done()` case. Details in the go-pprof skill.