Gemini API skill for Codex

A Gemini integration is done when the model id lives in config, every call goes through one wrapper on the current SDK, conversation state is owned on purpose (server-side interactions or client-replayed history, not both), large media goes through the Files API, tool loops terminate, and token usage including cached and thinking tokens is logged.

Before you write code

1. Read AGENTS.md, then grep for the SDK in use: rg -n "google.generativeai|google.genai|@google/genai|@google/generative-ai" . The legacy packages (google-generativeai, @google/generative-ai) are deprecated; migrate them to google-genai / @google/genai.
2. Check which API surface the code uses. The Interactions API (client.interactions.create) is Google’s recommended interface for new work; the older client.models.generate_content (generateContent) remains supported and is still needed for explicit context caches. Do not mix both in one conversation flow.
3. Model ids change often (Gemini 2.5 access is restricted to existing users; Gemini 3.x and 4 are current at the time of writing). Read the model id from config and check ai.google.dev/gemini-api/docs/models before hardcoding anything.
4. Installs need network: pip install -U google-genai or npm install @google/genai. The SDK reads GEMINI_API_KEY from the environment. Never commit the key.

First call

from google import genai
client = genai.Client()                      # GEMINI_API_KEY from env

interaction = client.interactions.create(
    model=MODEL,                             # e.g. "gemini-3.8-flash", from config
    system_instruction="You are a concise assistant for an internal ops tool.",
    input="Summarize the incident in two sentences: ...",
    generation_config={"thinking_level": "low", "max_output_tokens": 800},
)
print(interaction.output_text)
print(interaction.usage.total_input_tokens, interaction.usage.total_output_tokens,
      interaction.usage.total_cached_tokens, interaction.usage.total_thought_tokens)

import { GoogleGenAI } from "@google/genai";
const ai = new GoogleGenAI({});
const interaction = await ai.interactions.create({ model: MODEL, input: "Hello" });
console.log(interaction.output_text);

Output is a list of typed steps (model output, function calls, tool results). output_text concatenates only the final text block; iterate steps when text and tool calls interleave.

State

Mode	How	Use when
Server-side (default store=true)	Pass previous_interaction_id=prev.id with only the new input	Chat and agents; simplest; thought signatures handled for you
Stateless (store=false)	Resend full history each turn, including every thought and tool block exactly as received	Data-retention rules forbid storage, or you need full control

In stateless mode, dropping or editing thought blocks (thought signatures) degrades reasoning and can fail tool calls. Pass them back verbatim, even when switching models.

Multimodal input

import base64
img = base64.b64encode(open("receipt.jpg", "rb").read()).decode()
interaction = client.interactions.create(
    model=MODEL,
    input=[
        {"type": "text", "text": "Extract merchant, date and total."},
        {"type": "image", "mime_type": "image/jpeg", "data": img},
    ],
)

# Large or reused media: upload once, reference by URI
f = client.files.upload(file="meeting.mp4")
interaction = client.interactions.create(
    model=MODEL,
    input=[{"type": "video", "uri": f.uri, "mime_type": f.mime_type},
           {"type": "text", "text": "List action items with timestamps."}],
)

• Use the Files API when the total request exceeds ~100 MB, for PDFs over 50 MB, or when the same file is used in several requests. Files are deleted after 48 hours; 2 GB per file, 20 GB per project. Re-upload rather than storing the URI long term.
• Video and long audio cost tokens per second of media; sample or trim before sending.
• Uploaded files are processed asynchronously; check the file’s state with files.get before using a large video, or the request may fail.

Function calling

import json
tools = [{
    "type": "function",
    "name": "get_order",
    "description": "Look up an order by id. Use for any question about a specific order.",
    "parameters": {"type": "object",
                   "properties": {"order_id": {"type": "string"}},
                   "required": ["order_id"]},
}]

interaction = client.interactions.create(model=MODEL, input=question, tools=tools)
for _ in range(8):                                         # hard cap on tool rounds
    calls = [s for s in interaction.steps if s.type == "function_call"]
    if not calls:
        break
    results = []
    for c in calls:                                        # parallel calls arrive together
        out = dispatch(c.name, c.arguments)                # validate args before executing
        results.append({"type": "function_result", "name": c.name, "call_id": c.id,
                        "result": [{"type": "text", "text": json.dumps(out)}]})
    interaction = client.interactions.create(
        model=MODEL, tools=tools, input=results, previous_interaction_id=interaction.id)
print(interaction.output_text)

• Control calling with generation_config={"tool_choice": "auto" | "any" | "none" | "validated"}.
• Return errors as results ({"error": "order not found"}) so the model can recover.
• Side-effecting tools (refunds, emails, deletes) need a user confirmation step in your code before dispatch runs them. See agent-tool-design for schema design.
• Built-in tools are declared by type, e.g. {"type": "google_search"}; grounded answers carry citations you must display per Google’s terms.

Structured output

from pydantic import BaseModel
class Receipt(BaseModel):
    merchant: str
    date: str
    total_cents: int

interaction = client.interactions.create(
    model=MODEL, input=[...],
    response_format={"type": "text", "mime_type": "application/json",
                     "schema": Receipt.model_json_schema()},
)
receipt = Receipt.model_validate_json(interaction.output_text)

Keep schemas flat and small; deeply nested optional fields increase refusals and latency. Validate anyway: schema support covers a subset of JSON Schema.

Streaming

for event in client.interactions.create(model=MODEL, input=prompt, stream=True):
    if event.event_type == "step.delta" and event.delta.type == "text":
        send_to_client(event.delta.text)
    elif event.event_type == "interaction.completed":
        log_usage(event)

Event types include step.start, step.delta, step.stop and interaction.completed. For SSE delivery to browsers and cancellation, see llm-streaming.

Thinking and cost

• generation_config.thinking_level: minimal, low, medium, high (allowed values vary by model; defaults vary too). Start with low for extraction and classification, raise only where evals improve.
• thinking_summaries: "auto" returns readable summaries; never show raw thoughts as facts.
• Implicit caching is automatic on Gemini 2.5 and newer when a request repeats a long prefix (minimum 2,048-4,096 input tokens depending on model). Put stable content (system instruction, documents) first and the variable question last. Check usage.total_cached_tokens.
• Explicit caches (client.caches.create(...) with a TTL, referenced through cached_content in generate_content) exist only on the generateContent API and only for supported Gemini 3 and 2.5 models. Use them for a large document queried many times within the TTL; you pay storage per hour.
• service_tier (flex, standard, priority) trades latency for price on supported models; batch backfills belong on flex or batch modes.

Errors and limits

Symptom	Cause	Fix
429 RESOURCE_EXHAUSTED	Rate or quota limit	Exponential backoff with jitter; check tier quotas
Empty text, finish reason safety	Safety filter	Inspect safety ratings; adjust safety_settings only within policy
400 on file reference	File expired (48 h) or still processing	Re-upload; wait for active state
Tool loop never ends	Model keeps calling tools	Cap rounds; tool_choice: "none" on the final turn
Lost context across turns	Mixed stateful and stateless	Pick one state mode per conversation

Workflow

• Read AGENTS.md; identify SDK, API surface and model ids in use
• Migrate legacy SDK imports to google-genai / @google/genai
• Centralize client creation, model id and usage logging
• Choose state mode; implement tool loop with caps and confirmations
• Route large media through the Files API
• Add structured output and validation where code consumes the result
• Measure tokens (cached, thought) and latency on representative inputs

Verification

• Run a smoke script against the real API (needs network and a key; ask before running it in CI with spend) covering text, one image, one tool round trip and structured output.
• Log usage per call and confirm cached tokens appear on repeated long-prefix requests.
• Unit-test the tool loop with a fake client that returns two parallel function calls, then text.