Background Jobs skill for Codex

Gives the Codex app a display name, a one-line description and a starter prompt for this skill.

interface:
  display_name: "Background Jobs"
  short_description: "Jobs that survive retries, crashes and deploys"
  default_prompt: "$background-jobs Move PDF generation out of the request into a background job"

# Library-specific settings

Read only the section for the library the project uses. Check the installed version in
the lockfile first; option names below match current major versions (BullMQ 5, Sidekiq
7/8, Celery 5.x).

## BullMQ (Node, Redis)

```ts
import { Queue, Worker, UnrecoverableError } from 'bullmq';
import IORedis from 'ioredis';

// Workers use blocking commands: maxRetriesPerRequest must be null.
const connection = new IORedis(process.env.REDIS_URL!, { maxRetriesPerRequest: null });

export const emailQueue = new Queue('email', {
  connection,
  defaultJobOptions: {
    attempts: 8,
    backoff: { type: 'exponential', delay: 2_000 },
    removeOnComplete: { age: 24 * 3600, count: 10_000 },
    removeOnFail: { age: 14 * 24 * 3600 },
  },
});

await emailQueue.add('receipt', { invoiceId }, { jobId: `receipt-${invoiceId}` });

const worker = new Worker(
  'email',
  async (job) => {
    const invoice = await loadInvoice(job.data.invoiceId);
    if (!invoice) throw new UnrecoverableError('invoice not found');
    await sendReceipt(invoice);
  },
  { connection, concurrency: 10, limiter: { max: 50, duration: 1_000 } },
);

worker.on('failed', (job, err) => log.warn({ jobId: job?.id, err }, 'job failed'));

for (const sig of ['SIGTERM', 'SIGINT']) {
  process.on(sig, async () => { await worker.close(); process.exit(0); });
}
```

Traps:

- A custom `jobId` that already exists is silently ignored while the old job is kept
  (that is the dedupe), including completed jobs not yet removed. Use the `deduplication`
  option when you need a time-bounded dedupe window instead.
- Custom job IDs must not be plain integers; avoid `:` in them too, since BullMQ uses it
  as a Redis key separator.
- `removeOnComplete: true` deletes immediately, which also disables `jobId` dedupe for
  repeats; keep a count or age window.
- Long CPU-bound processors block lock renewal and get marked stalled, then run twice.
  Use a sandboxed processor file or keep the event loop free.
- Use a separate Redis (or at least `maxmemory-policy noeviction`) for queues; eviction
  silently deletes jobs.
- Repeatable/cron jobs: use job schedulers (`upsertJobScheduler`) in current BullMQ
  versions rather than the older repeat API.

## Sidekiq (Ruby, Redis)

```ruby
class SendReceiptJob
  include Sidekiq::Job
  sidekiq_options queue: "critical", retry: 8

  sidekiq_retry_in do |count, exception|
    case exception
    when PermanentError then :kill          # straight to the Dead set
    else (count ** 4) + 15 + rand(10 * (count + 1))
    end
  end

  sidekiq_retries_exhausted do |job, ex|
    Rails.logger.error("receipt job exhausted #{job['jid']}: #{ex.message}")
  end

  def perform(invoice_id)
    invoice = Invoice.find_by(id: invoice_id)
    return if invoice.nil? || invoice.receipt_sent_at?
    ReceiptMailer.with(invoice:).receipt.deliver_now
    invoice.update!(receipt_sent_at: Time.current)
  end
end

SendReceiptJob.perform_async(invoice.id)   # JSON-native args only
```

```yaml
# config/sidekiq.yml
:concurrency: 10
:timeout: 25
:queues:
  - [critical, 6]
  - [default, 3]
  - [bulk, 1]
```

Traps:

- Default retry count is 25 (about three weeks of retries); set it per job.
- Sidekiq 7+ rejects non-JSON-native arguments (symbols, objects) by default; pass IDs.
- Open-source Sidekiq fetch removes the job from Redis when a worker picks it up; a
  hard kill (OOM, SIGKILL) loses in-flight jobs. Graceful TERM pushes unfinished jobs
  back after `:timeout`. Reliable fetch is a Sidekiq Pro feature.
- `concurrency` threads each need a database connection: size the Active Record pool
  at least to concurrency.
- Weighted queues still starve under load; run a dedicated process for `critical`.

## Celery (Python)

```python
# celery.py
app.conf.update(
    task_acks_late=True,                 # ack after the task finishes
    task_reject_on_worker_lost=True,     # requeue if the worker process dies
    worker_prefetch_multiplier=1,        # do not hoard tasks on one worker
    task_time_limit=300,                 # hard kill
    task_soft_time_limit=240,            # raises SoftTimeLimitExceeded first
    broker_transport_options={"visibility_timeout": 3600},  # Redis/SQS: > longest task + ETA
    task_routes={"billing.*": {"queue": "critical"}, "reports.*": {"queue": "bulk"}},
)
```

```bash
celery -A proj worker -Q critical -c 8 --max-tasks-per-child 500
celery -A proj worker -Q default,bulk -c 4
```

Traps:

- With `acks_late`, tasks must be idempotent; that is the point of the setting.
- On Redis or SQS brokers, tasks with ETA/countdown longer than `visibility_timeout`
  are redelivered and run more than once. Keep long delays in a scheduler instead.
- `retry_backoff=True` without `retry_jitter=True` synchronises retries across workers.
- Passing model instances pickles stale data; use the JSON serializer and pass IDs.
- `--max-tasks-per-child` mitigates memory leaks in long-lived worker processes.

## Postgres-backed queues

Solid Queue (Rails), River (Go), Oban (Elixir), pg-boss and graphile-worker (Node),
Procrastinate (Python) store jobs in Postgres and claim them with
`FOR UPDATE SKIP LOCKED`.

- Big advantage: enqueue in the same transaction as the business write, so a job exists
  if and only if the data committed.
- Throughput is lower than Redis queues (low thousands of jobs per second is typical,
  depending on hardware); fine for most product workloads.
- Completed jobs bloat tables; configure the library's pruning and watch autovacuum on
  the jobs table.
- Use a separate connection pool or database for job polling if the primary pool is
  tight.