WebSockets Realtime skill for Codex

Done well means: every connection is authenticated before it is accepted, dead peers are detected within a minute, a slow client cannot exhaust server memory, clients reconnect with jitter and resume without losing or duplicating messages, and a broadcast reaches subscribers on every instance, not only the one that received it.

First, confirm WebSockets are the right transport

Need	Transport
Server pushes updates, client rarely sends (feeds, notifications, LLM tokens)	Server-Sent Events: use $server-sent-events
Bidirectional, low-latency, frequent client messages (chat, cursors, games)	WebSocket
Must work through hostile corporate proxies with fallbacks	Socket.IO (WebSocket with HTTP long-polling fallback)
Request/response with typed contracts	Plain HTTP or gRPC streaming
Audio/video or unreliable datagrams	WebRTC / WebTransport, out of scope here

Inspect before changing anything

• Read AGENTS.md for the runtime, package manager and test command.
• Find the current server: rg -n "WebSocketServer|socket\.io|websockets\.serve|@app.websocket|upgrade" --type-add 'web:*.{ts,js,py}' -t web.
• Find what sits in front: nginx/Caddy config, ALB/Cloud Run/Cloudflare settings, ingress annotations. Most “random disconnects” are proxy idle timeouts, not code.
• Check how many instances run in production. One instance hides every fan-out bug.

Workflow checklist

- [ ] Authenticate during the HTTP upgrade (cookie + Origin check, or a short-lived ticket)
- [ ] Set maxPayload and per-connection rate limits
- [ ] Define a versioned message envelope with type, id and seq
- [ ] Add server ping/pong heartbeat and terminate dead sockets
- [ ] Respect backpressure (bufferedAmount) and cap per-client queues
- [ ] Add pub/sub fan-out if more than one instance can run
- [ ] Client: reconnect with full-jitter backoff and resume from last seq
- [ ] Graceful shutdown: close with 1012, drain, then exit
- [ ] Proxy config: Upgrade headers, read timeout above heartbeat interval
- [ ] Tests: auth rejection, heartbeat timeout, reconnect + resume, two-instance broadcast

Authentication at the upgrade

Browsers cannot set an Authorization header on new WebSocket(). Pick one:

Option	When	Must also
Session cookie sent with the upgrade	Same-site app already using cookie sessions	Check Origin against an allowlist (cross-site WebSocket hijacking)
Short-lived ticket in the query string	Token auth, cross-origin clients	Ticket single-use, 30-60 s TTL, issued by an authenticated HTTP call; never put the long-lived token in the URL (it lands in logs)
First message carries the token	Legacy clients	Close with 1008 if no valid auth message within ~5 s; accept nothing else before it

Reject before the handshake completes so unauthenticated sockets never exist:

import { createServer } from "node:http";
import { WebSocketServer, type WebSocket } from "ws";

const server = createServer(app);
const wss = new WebSocketServer({ noServer: true, maxPayload: 64 * 1024 });
const ALLOWED_ORIGINS = new Set(["https://app.example.com"]);

server.on("upgrade", async (req, socket, head) => {
  try {
    if (!ALLOWED_ORIGINS.has(req.headers.origin ?? "")) throw new Error("origin");
    const user = await authenticate(req);          // cookie session or one-time ticket
    wss.handleUpgrade(req, socket, head, (ws) => wss.emit("connection", ws, req, user));
  } catch {
    socket.write("HTTP/1.1 401 Unauthorized\r\nConnection: close\r\n\r\n");
    socket.destroy();
  }
});

Long-lived connections outlive tokens. Store the token expiry on the connection and close with code 4001 (app-defined) when it passes, so the client re-authenticates. Authorize every subscribe/join message against the user, not just the connection.

Message envelope

{ "v": 1, "type": "message.created", "id": "01J9Z...", "seq": 4182, "ts": "2026-03-02T10:15:00Z", "data": { } }

• id: client-generated for client-to-server messages so the server can dedupe retries.
• seq: per-channel monotonic number from the server; the client tracks the last one seen.
• v: protocol version, so old clients get a clear close (4002) instead of silent breakage.
• Validate every inbound message with a schema (Zod, Pydantic) before acting on it.

Heartbeats and dead-peer detection

TCP does not tell you a phone went into a tunnel. Ping from the server:

const HEARTBEAT_MS = 30_000;
wss.on("connection", (ws: WebSocket & { isAlive?: boolean }) => {
  ws.isAlive = true;
  ws.on("pong", () => { ws.isAlive = true; });
});
const timer = setInterval(() => {
  for (const ws of wss.clients as Set<WebSocket & { isAlive?: boolean }>) {
    if (!ws.isAlive) { ws.terminate(); continue; }
    ws.isAlive = false;
    ws.ping();
  }
}, HEARTBEAT_MS);
wss.on("close", () => clearInterval(timer));

Keep the heartbeat below every idle timeout on the path (nginx proxy_read_timeout default 60 s, AWS ALB idle timeout default 60 s, Cloudflare ~100 s).

Backpressure

ws.send() queues in memory when the client reads slowly. Check before sending:

const MAX_BUFFER = 1 * 1024 * 1024;
function safeSend(ws: WebSocket, payload: string) {
  if (ws.bufferedAmount > MAX_BUFFER) {
    ws.close(1013, "client too slow");   // or drop non-critical messages / coalesce state
    return;
  }
  ws.send(payload);
}

For high-frequency state (cursors, prices) send the latest snapshot on a tick instead of every change.

Scaling past one instance

• A WebSocket is one long TCP connection, so it does not need sticky sessions. Socket.IO does, because its handshake starts with HTTP polling, unless you force transports: ["websocket"].
• Fan-out: publish to a broker, and every instance delivers to its local subscribers. Redis pub/sub (or the @socket.io/redis-adapter), NATS, or Postgres LISTEN/NOTIFY for low volume. Redis pub/sub is fire-and-forget: anything published while an instance is reconnecting is lost, which is why clients resume from seq.
• For resume, keep a bounded per-channel history (Redis Streams with MAXLEN, or a DB table) so resume(channel, lastSeq) can replay the gap; if the gap is too old, tell the client to refetch full state over HTTP.
• Budget connections per instance (file descriptors, memory ~ tens of KB each); raise ulimit -n and load-test before launch.

Client reconnection

function backoff(attempt: number) {
  const cap = 30_000, base = 500;
  return Math.random() * Math.min(cap, base * 2 ** attempt);   // full jitter
}

• Reset attempt only after the connection has stayed open ~10 s, or a server that accepts then crashes causes a tight reconnect loop.
• On open: re-authenticate, re-subscribe, send resume with the last seq per channel.
• Do not reconnect on 1008 (policy) or 4001 (auth expired) without refreshing credentials first; do reconnect on 1001, 1006, 1011, 1012, 1013.
• Queue outbound messages while disconnected only if they are idempotent (carry id).

Close codes that matter

Code	Meaning	Client reaction
1000	Normal closure	Do not reconnect unless the app wants to
1001	Going away (page or server leaving)	Reconnect with backoff
1006	Abnormal, no close frame (never sent explicitly)	Reconnect with backoff
1008	Policy violation (auth, Origin)	Stop; surface an error
1009	Message too big	Fix the client, do not retry the same payload
1011	Server error	Reconnect with backoff
1012	Service restart	Reconnect with extra jitter
1013	Try again later / overloaded	Reconnect with longer backoff
4000-4999	Application-defined	Document each one

Graceful shutdown and deploys

On SIGTERM: stop accepting upgrades, close each socket with 1012, wait a few seconds for in-flight writes, then exit. Clients spread reconnects with jitter, so a rolling deploy does not stampede the new instances. Set the orchestrator’s termination grace period longer than the drain time.

Proxy configuration

location /ws/ {
  proxy_pass http://app;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "upgrade";
  proxy_set_header Host $host;
  proxy_read_timeout 75s;     # above the 30 s heartbeat
  proxy_send_timeout 75s;
}

Other stacks

• Python websockets (v14+): use the websockets.asyncio.server.serve API; it sends pings by default (ping_interval=20, ping_timeout=20) and has max_size for payloads.
• FastAPI/Starlette: @app.websocket("/ws"), authenticate from websocket.cookies or query params before await websocket.accept(); close with code=1008 otherwise.
• Cloudflare Durable Objects: use the WebSocket Hibernation API; see $cloudflare-durable-objects.

Verification

• Unit/integration: start the server on an ephemeral port and connect a real client (Node 22+ ships a global WebSocket; Python websockets.connect). Assert: missing or bad auth fails the handshake, a foreign Origin is rejected, oversize messages close with 1009, a client that stops answering pings is terminated.
• Two-instance test: run two server processes against one Redis, connect a client to each, publish via one, assert delivery on both.
• Resume test: disconnect mid-stream, publish N messages, reconnect with lastSeq, assert exactly N messages arrive in order with no duplicates.
• Manual: websocat wss://host/ws (install may need network approval in the sandbox).

Gotchas

• ws emits error on the socket; an unhandled error event crashes the Node process. Attach ws.on("error", ...) to every connection.
• Logging full upgrade URLs leaks query-string tickets. Redact before logging.
• JSON.parse on untrusted input throws; wrap it and close with 1003 or 1008 on garbage.
• Broadcasting with for (client of wss.clients) on one instance passes tests and fails in production with three replicas.
• Serverless platforms without WebSocket support (most function runtimes) kill the connection at the request timeout. Use a managed gateway or a long-running service.

Output template

## Realtime change: <feature>
- Transport: WebSocket (why not SSE: ...)
- Auth: <cookie+Origin | ticket>, expiry handling: <...>
- Envelope: v<N>, types added: <...>
- Heartbeat: <interval>; proxy timeouts verified: <nginx/ALB values>
- Fan-out: <Redis pub/sub | NATS | none (single instance, documented)>
- Resume: <history store, retention>
- Tests added: <list>
- Not verified: <e.g. load test, real mobile network>