Redis Data Structures skill for Codex

Done well means: every key has a documented type, naming scheme, owner and expiry; each operation on the hot path is O(1) or O(log N) on a bounded collection; multi-step updates are atomic; and memory per key is measured, not assumed.

Before designing

1. Read AGENTS.md; find the client (ioredis, node-redis, redis-py, go-redis, Lettuce/Jedis) and whether the target is standalone, Sentinel or Cluster.
2. Check the server: redis-cli INFO server (version, redis_mode) and CONFIG GET maxmemory*. Redis 8 bundles JSON, time series, probabilistic types and the query engine; Valkey and older Redis may need modules for those. Check before using JSON.*.
3. Is the data a cache (rebuildable) or the source of truth? If source of truth, confirm persistence (appendonly yes, snapshots) and that maxmemory-policy is noeviction.

Cache-aside strategy, TTL tuning and stampede protection belong to $redis-caching; this skill covers the data model.

Pick the type

Need	Type	Key commands	Complexity
Single value, counter, flag, lock	String	SET k v EX 60 NX, INCR, INCRBY, GETEX	O(1)
Object with fields read/updated separately	Hash	HSET, HGET, HMGET, HINCRBY, HEXPIRE (7.4+)	O(1) per field
Ranking, priority, time window, delayed jobs	Sorted set	ZADD, ZINCRBY, ZRANGE ... REV LIMIT, ZRANGEBYSCORE, ZREMRANGEBYSCORE	O(log N)
Membership, dedup, tags	Set	SADD, SISMEMBER, SMISMEMBER, SINTERCARD	O(1) per member
Simple FIFO between few processes, capped recent list	List	LPUSH, LTRIM, BLMOVE, LRANGE 0 9	O(1) at ends
Durable event log, work queue with acks and replay	Stream	XADD, XREADGROUP, XACK, XAUTOCLAIM	O(1) append
Per-id boolean over dense integer ids	Bitmap	SETBIT, GETBIT, BITCOUNT	O(1)
Approximate unique counts	HyperLogLog	PFADD, PFCOUNT, PFMERGE	~12 KB, ~0.81% error
Nearby search	Geo	GEOADD, GEOSEARCH ... BYRADIUS	O(log N)
Nested document with partial updates	JSON	JSON.SET, JSON.GET, JSON.NUMINCRBY	depends on path

Rules of thumb:

• A hash of many small fields beats many string keys: less per-key overhead, one TTL.
• Lists are not queues you can recover from: a consumer that crashes after LPOP loses the item. Use BLMOVE to a processing list, or a stream with consumer groups.
• Sorted set scores are doubles: integers above 2^53 lose precision. Use ms timestamps, not ns.
• Do not store a large JSON blob in a string and rewrite it for one-field updates; use a hash or JSON type.

Key naming and expiry

<app>:<entity>:<id>[:<facet>]
shop:cart:9f2c            hash   TTL 7d, owner: cart-service
shop:lb:weekly:2026-W44   zset   TTL 21d
shop:events:orders        stream MAXLEN ~1000000

• Colons as separators; ids last so SCAN MATCH shop:cart:* works.
• Every non-permanent key gets a TTL at creation (SET ... EX, or EXPIRE in the same MULTI or script). Keys without TTL in a cache are a slow memory leak.
• Hash field TTL (HEXPIRE, Redis 7.4+) expires individual fields; check server support.
• Time-bucketed keys (...:2026-W44) let old data expire whole instead of trimming.

Recipes

Leaderboard with tie-break by earliest score:

ZADD shop:lb:weekly:2026-W44 GT <points> user:42
ZRANGE shop:lb:weekly:2026-W44 0 9 REV WITHSCORES
ZREVRANK shop:lb:weekly:2026-W44 user:42

For ties, encode points * 1e10 + (MAX_TS - ts_seconds) into the score while staying under 2^53.

Sliding-window counter (per user, last 60 s):

ZADD rl:user:42 <now_ms> <now_ms>-<random>
ZREMRANGEBYSCORE rl:user:42 0 <now_ms - 60000>
ZCARD rl:user:42
PEXPIRE rl:user:42 60000

Run these atomically (script below). Full rate-limiter design: $api-rate-limiting.

Stream work queue:

XGROUP CREATE shop:events:orders billing $ MKSTREAM
XADD shop:events:orders MAXLEN ~ 1000000 * type paid orderId 123
XREADGROUP GROUP billing worker-1 COUNT 50 BLOCK 5000 STREAMS shop:events:orders >
XACK shop:events:orders billing 1730000000000-0
XAUTOCLAIM shop:events:orders billing worker-2 60000 0-0 COUNT 50   # reclaim stuck messages
XPENDING shop:events:orders billing                                  # monitor backlog

• Process idempotently: a message can be delivered again after a crash or reclaim.
• Trim with MAXLEN ~ or MINID ~ on every XADD; untrimmed streams grow forever.
• Track delivery counts in XPENDING and move poison messages to a dead-letter stream.
• For heavy fan-out, retention measured in days, or replay across services, consider Kafka ($kafka-streaming) instead.

Atomic multi-step logic

MULTI/EXEC queues commands but cannot branch on values read inside. Use a script:

-- KEYS[1] = rate key, ARGV = now_ms, window_ms, limit, member
redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, ARGV[1] - ARGV[2])
local n = redis.call('ZCARD', KEYS[1])
if n >= tonumber(ARGV[3]) then return 0 end
redis.call('ZADD', KEYS[1], ARGV[1], ARGV[4])
redis.call('PEXPIRE', KEYS[1], ARGV[2])
return 1

• Load once and call by SHA (EVALSHA, with fallback on NOSCRIPT) or register with FUNCTION LOAD (Redis 7+) so scripts survive as named functions.
• Pass every key through KEYS, never build key names inside the script; Cluster routing depends on it.
• Scripts block the server while running. Keep them O(small N); no loops over big collections.

Cluster

• Multi-key commands, transactions and scripts need all keys in one slot. Use hash tags: shop:{cart:9f2c}:items and shop:{cart:9f2c}:meta share the slot of cart:9f2c.
• Do not put a whole tenant behind one hash tag if the tenant is large: that creates a hot shard.
• KEYS and unscoped SCAN hit one node in Cluster; iterate each primary.

Memory

redis-cli INFO memory                     # used_memory, fragmentation ratio
redis-cli MEMORY USAGE shop:cart:9f2c     # bytes for one key
redis-cli --bigkeys                       # largest key per type (SCAN-based, safe-ish)
redis-cli --memkeys                       # largest keys by memory
redis-cli OBJECT ENCODING shop:cart:9f2c  # listpack / hashtable / intset / skiplist

• Small hashes, sets and sorted sets use compact encodings (listpack, intset) below thresholds such as hash-max-listpack-entries (default 128) and hash-max-listpack-value (64 bytes). Crossing them can multiply memory per key.
• Estimate: measure MEMORY USAGE on 1 000 representative keys, multiply out, add 30-50% headroom for fragmentation and replication buffers.
• Big keys (a hash or zset with millions of members) cause latency spikes on delete and migration. Split them by bucket, and delete with UNLINK (non-blocking) instead of DEL.
• Run --bigkeys, --memkeys and SCAN against production only with user approval and ideally on a replica; they add load.

Hard rules

• Never KEYS * in production; use SCAN with COUNT.
• FLUSHDB, FLUSHALL, mass UNLINK and CONFIG SET on shared instances require explicit user confirmation.
• No unbounded collections: every list, stream, set and zset has a trim or expiry plan.
• Serialize consistently (one format per key type); mixing JSON and msgpack across services breaks readers.

Verification

• Integration test against a real Redis (container) for each script and recipe, including concurrency (parallel clients) for atomic paths.
• MEMORY USAGE on sample keys matches the sizing estimate.
• redis-cli --latency and SLOWLOG GET 20 show no new slow commands after deploy.
• INFO keyspace shows expires close to keys for cache databases.

Output

| Key pattern | Type | Example ops | TTL / trim | Est. size per key | Owner |
|---|---|---|---|---|---|
| shop:cart:<id> | hash | HSET, HGETALL | 7d | 1.2 KB | cart-service |
Atomic paths: <scripts/functions and the keys they touch>
Cluster: <hash tags used, hot-slot risk>