OpenAI Function Calling skill for Codex

Done well means: tools are strict and few, the loop executes every call in a turn and returns every result with its call_id, failures come back to the model as actionable text, side effects are gated by your code (not by the prompt), and the loop has a hard iteration cap.

Before writing code

1. Read AGENTS.md; locate the existing LLM wrapper and any tool registry.
2. List the functions the model needs. If more than ~20 would be live at once, split by task and expose a subset per turn with allowed_tools, or use tool search on models that support it.
3. Classify each tool: read-only, idempotent write, or irreversible (payments, emails, deletes). Irreversible tools need a confirmation step outside the model.

Tool definition

TOOLS = [
    {
        "type": "function",
        "name": "get_invoice",
        "description": "Fetch one invoice by id. Use when the user references an invoice "
                       "number like INV-2024-0012. Returns JSON or an error string.",
        "parameters": {
            "type": "object",
            "properties": {
                "invoice_id": {"type": "string", "description": "Format INV-YYYY-NNNN"},
                "include_lines": {"type": ["boolean", "null"],
                                  "description": "Line items; null means false"},
            },
            "required": ["invoice_id", "include_lines"],
            "additionalProperties": False,
        },
        "strict": True,
    },
]

Strict mode rules: every property in required, optional ones nullable, additionalProperties: false on every object. In Responses, omitting strict makes the API try strict and silently fall back to non-strict — set it explicitly.

Description checklist: when to use it, when not to, argument formats, what it returns, what errors look like. Do not ask the model for values your code already knows (user id, tenant, auth) — inject those server-side.

The loop

import json, os
from openai import OpenAI

client = OpenAI()
MODEL = os.environ["OPENAI_MODEL"]
MAX_STEPS = 8

HANDLERS = {"get_invoice": get_invoice}  # name -> callable(**args) -> str|dict

def run(user_text: str, ctx) -> str:
    items = [{"role": "user", "content": user_text}]
    for _ in range(MAX_STEPS):
        resp = client.responses.create(model=MODEL, input=items, tools=TOOLS,
                                       instructions=SYSTEM, store=False)
        items += resp.output                      # keep reasoning + function_call items
        calls = [i for i in resp.output if i.type == "function_call"]
        if not calls:
            return resp.output_text
        for call in calls:                        # zero, one or many per turn
            items.append({"type": "function_call_output",
                          "call_id": call.call_id,
                          "output": execute(call, ctx)})
    raise RuntimeError("tool loop exceeded MAX_STEPS")

def execute(call, ctx) -> str:
    fn = HANDLERS.get(call.name)
    if fn is None:
        return json.dumps({"error": "unknown_tool", "name": call.name})
    try:
        args = json.loads(call.arguments)
        result = fn(ctx=ctx, **args)
        return result if isinstance(result, str) else json.dumps(result, default=str)
    except NotFound as e:
        return json.dumps({"error": "not_found", "detail": str(e)})
    except ValidationError as e:
        return json.dumps({"error": "invalid_arguments", "detail": str(e)})
    except Exception:
        log.exception("tool %s failed", call.name)
        return json.dumps({"error": "internal_error", "retryable": True})

With previous_response_id instead of replay, send only the function_call_output items on the next request — the server already has the calls.

Control knobs

Goal	Setting
Let the model decide	tool_choice="auto" (default)
Must call something this turn	tool_choice="required"
Force one function (e.g. extraction step)	tool_choice={"type": "function", "name": "get_invoice"}
No tools this turn, keep definitions (cache-friendly)	tool_choice="none"
Restrict to a subset without changing tools	tool_choice={"type": "allowed_tools", "mode": "auto", "tools": [{"type": "function", "name": "get_invoice"}]}
Exactly zero or one call per turn	parallel_tool_calls=False

Keep the tools array identical across turns and narrow with allowed_tools: changing tools invalidates the prompt cache from that point on.

Parallel calls

• Expect several function_call items in one response. Execute independent reads concurrently (asyncio.gather), but return outputs for all of them before the next request — a missing call_id output is an API error.
• Disable parallel calls when tools have ordering dependencies (create then update) or share a rate-limited backend.

Custom tools (free-form input)

For code, SQL or DSL input, a {"type": "custom", "name": "run_sql", "description": ...} tool receives raw text instead of JSON. Constrain it with "format": {"type": "grammar", "syntax": "lark" | "regex", "definition": ...} (regex uses Rust syntax, no lookarounds). Keep grammars small; complex ones are rejected.

Safety for side effects

• Never let a tool description be the only protection. Enforce authorization in the handler using ctx (the authenticated user), not model-supplied ids alone.
• For irreversible actions return a pending confirmation ({"status": "needs_confirmation", "summary": ...}), show it to the user, and execute only after an explicit yes in a separate request.
• Make writes idempotent with a key derived from the call_id; retries happen.
• Treat tool outputs from external sources (web pages, emails) as untrusted: they can carry prompt injection. See prompt-injection-defense.

Gotchas

• Dropping reasoning items between steps on reasoning models degrades tool use.
• Returning huge tool outputs (full tables, raw HTML) burns context; summarize or page.
• Mismatched or reused call_id → 400 error. Copy it exactly.
• Exceptions raised out of the loop lose the conversation; return errors as tool output.
• Models may repeat the same failing call; include "retryable": false and a hint in the error so it stops, and keep MAX_STEPS as the backstop.

Verification

• Unit-test each handler with valid, invalid and unauthorized inputs (no model).
• Contract test: feed a recorded response containing two parallel calls; assert two outputs with matching call_ids are appended.
• Scenario evals: 20+ prompts with expected tool name and key arguments; measure tool-selection accuracy and argument exact-match (needs network; opt-in).
• Force a handler failure and confirm the model recovers or stops within MAX_STEPS.

Output contract

**Tools (n):** get_invoice (read), refund_invoice (write, confirmation required)
**Strict:** all tools strict; schemas in app/llm/tools.py
**Loop:** max 8 steps, parallel on, errors returned as JSON tool output
**Auth:** user id injected from session, never from model arguments
**Evals:** tool selection 19/20, argument match 18/20