OpenAI Fine-Tuning skill for Codex

Done well means: fine-tuning was chosen because prompting plus evals hit a ceiling, the account and base model are confirmed eligible, every JSONL line validates, and the fine-tuned checkpoint beats the base model with the best prompt on a held-out eval.

Check eligibility first (the platform is winding down)

OpenAI’s docs state the fine-tuning platform is being wound down: it is closed to new users, eligible existing customers can create jobs until January 6, 2027 (or until the base model shuts down, if earlier), and fine-tuned models serve inference until their base model shuts down. Published shutdowns: gpt-4.1-nano-2025-04-14 and its fine-tunes on October 23, 2026; gpt-4.1-2025-04-14, gpt-4.1-mini-2025-04-14 and their fine-tunes on January 14, 2027. Re-check the deprecations page before every plan; dates move.

Situation	Route
Org has run fine-tuning before, base model still live, job finishes before the cutoff	Continue with this skill
Org never fine-tuned on OpenAI	Not available. Improve prompts and few-shot examples ($prompt-engineering) or fine-tune open weights ($lora-fine-tuning)
Existing fine-tuned model in production	Plan migration now: rebuild the eval, then test prompting on a current model or an open-weights LoRA against it
Want reinforcement fine-tuning (RFT)	It targeted o4-mini, which is on the shutdown list; treat as unavailable unless the dashboard shows otherwise

Network is required for every API call; in a Codex sandbox with network off, prepare and validate data locally and ask before making API calls.

When fine-tuning is the right tool

Goal	Fine-tune?
Consistent output format, tone or label set that prompting gets 90% right	SFT, good fit
Pick between two acceptable styles (“concise and warm”, not “long”)	DPO on preference pairs
Teach new facts or private documents	No. Use retrieval ($rag-pipelines)
Cut prompt length/cost of a long system prompt at high volume	SFT can help; measure total cost including higher per-token fine-tuned prices
Task fails badly with the best prompt	Fix the task definition first; 50 good examples should show movement or the task is wrong

Workflow

- [ ] 1. Confirm eligibility, base model snapshot and its shutdown date
- [ ] 2. Build an eval set (held out, never trained on) and score base model + best prompt
- [ ] 3. Assemble training examples; dedupe; split train/validation
- [ ] 4. Validate JSONL with scripts/validate_jsonl.py
- [ ] 5. Upload files, create job, monitor events
- [ ] 6. Evaluate final model and earlier checkpoints on the eval set
- [ ] 7. Report win/loss vs baseline, cost, and the model's shutdown date

Read AGENTS.md for where datasets and secrets live. Never commit training data that contains customer PII; ask the user how it was consented and scrubbed.

Data format

SFT: one JSON object per line with a messages array in chat format. Include the same system prompt you will use in production; the model learns the mapping from that prompt.

{"messages": [{"role": "system", "content": "Classify the ticket. Reply with one label: billing, bug, how_to, other."}, {"role": "user", "content": "I was charged twice this month"}, {"role": "assistant", "content": "billing"}]}

Tool-calling examples may also carry tools, parallel_tool_calls and assistant tool_calls. Multi-turn examples train on every assistant turn; add "weight": 0 to an assistant message to exclude it from loss.

DPO: one-turn examples with a shared input and two outputs (text only):

{"input": {"messages": [{"role": "user", "content": "Summarise this release note: ..."}]}, "preferred_output": [{"role": "assistant", "content": "Adds SSO; fixes export timeout."}], "non_preferred_output": [{"role": "assistant", "content": "This release contains several exciting improvements..."}]}

Data rules:

• Minimum 10 examples; start with 50-100 high-quality ones, then scale if the eval moves.
• Hold out the eval set before anything else; dedupe train against it (exact and near-duplicate).
• Cover the real distribution: edge cases, refusals, long inputs, every label in useful proportion.
• One assistant voice: contradictory examples teach averaging, not the policy you want.
• DPO pairs should differ in the property you care about, not in unrelated length or format.

Validate before upload (stdlib only, works offline):

python3 scripts/validate_jsonl.py train.jsonl --mode sft
python3 scripts/validate_jsonl.py prefs.jsonl --mode dpo --max-tokens 65536

It checks JSON per line, roles, empty content, final assistant turn, DPO structure, duplicates and an approximate token count, and exits non-zero on any error.

Create and monitor the job

from openai import OpenAI

client = OpenAI()  # OPENAI_API_KEY from env; never hard-code it

train = client.files.create(file=open("train.jsonl", "rb"), purpose="fine-tune")
valid = client.files.create(file=open("valid.jsonl", "rb"), purpose="fine-tune")

job = client.fine_tuning.jobs.create(
    model="gpt-4.1-mini-2025-04-14",
    training_file=train.id,
    validation_file=valid.id,
    suffix="ticket-router-v3",
    seed=42,
    method={
        "type": "supervised",
        "supervised": {"hyperparameters": {"n_epochs": "auto"}},
    },
)
print(job.id, job.status)

DPO uses method={"type": "dpo", "dpo": {"hyperparameters": {"beta": 0.1}}}; beta ranges 0-2, higher stays closer to the reference model. A common pattern is SFT first, then DPO starting from the SFT model.

for ev in client.fine_tuning.jobs.list_events(job.id, limit=20):
    print(ev.created_at, ev.level, ev.message)

for cp in client.fine_tuning.jobs.checkpoints.list(job.id):
    print(cp.step_number, cp.fine_tuned_model_checkpoint, cp.metrics)

Creating a job costs money: state the estimated training tokens x epochs and get the user’s go-ahead before calling jobs.create. Cancel with client.fine_tuning.jobs.cancel(id).

Hyperparameters

Symptom on validation	Change
Output ignores the format, loss still falling at end	More epochs (+1 or +2) or more data
Repeats training phrasing, worse on unusual inputs	Fewer epochs; pick an earlier checkpoint
Loss noisy and no improvement	Lower learning_rate_multiplier; check label consistency
Small dataset (< 100) with diverse outputs	Leave auto; it already raises epochs for small sets

Only the last three epoch checkpoints are kept; each is a usable model ID.

Evaluate

Score base model with the production prompt and every kept checkpoint on the same held-out set, with the same graders ($llm-evals). Call fine-tuned models like any other model:

resp = client.responses.create(
    model="ft:gpt-4.1-mini-2025-04-14:acme:ticket-router-v3:abc123",
    input=[{"role": "system", "content": SYSTEM}, {"role": "user", "content": text}],
)

Decide with numbers: accuracy or rubric pass rate, failure categories, latency and cost per 1k requests. A fine-tune that ties the base model is a loss (extra cost, shutdown risk).

Gotchas

• Training without the production system prompt, then calling with it (or vice versa), silently degrades results.
• moderation_checks events can block deployment of a finished model; read the events before assuming success.
• A fine-tuned model dies with its base snapshot. Record the shutdown date in the deployment config and in the report.
• Validation loss going down while eval pass rate stays flat means the loss measures wording, not correctness; trust the eval.
• Token counts in the validator are approximate (chars / 4); the API’s own count governs billing.
• Do not train on outputs from another provider’s model in violation of its terms; ask where generated examples came from.

Report template

**Base:** gpt-4.1-mini-2025-04-14 (shuts down 2027-01-14) | **Method:** SFT, 3 epochs (auto)
**Data:** 420 train / 80 valid / 150 held-out eval; deduped; validator clean
**Result:** eval pass 93.3% vs 86.0% base+prompt; billing/bug confusion down from 14 to 3
**Cost:** training ~1.1M tokens; inference price per 1k requests: $X vs $Y
**Model ID:** ft:gpt-4.1-mini-2025-04-14:acme:ticket-router-v3:abc123 (checkpoint step 1260)
**Risk:** platform wind-down; migration plan: open-weights LoRA evaluated by 2026-12