OpenAI Embeddings skill for Codex

Done well means: one model and dimension per vector space, recorded next to every vector; text normalized and hashed so nothing is embedded twice; requests batched within API limits with retries; and a small labeled query set proving that retrieval actually improves.

Pick model and dimensions

Model	Native dims	Max input	Use when
text-embedding-3-small	1536	8,192 tokens	Default: cheap, good quality, most search and dedup
text-embedding-3-large	3072	8,192 tokens	Quality matters more than cost; multilingual or subtle domains
text-embedding-ada-002	1536	8,192 tokens	Legacy only; migrate away

Check the embeddings guide for newer models before starting a new index.

The dimensions parameter shortens v3 embeddings server-side while keeping them normalized. Rules of thumb:

• 3-large at 1024 or 256 dims often beats 3-small at full size; measure on your data.
• Pick dims to fit your store: pgvector HNSW indexes on vector cap at 2,000 dimensions, so 3-large needs dimensions<=2000 (or halfvec) there.
• Storage per vector ≈ dims × 4 bytes (float32): 1536 dims ≈ 6 KB; 1M vectors ≈ 6 GB before index overhead.

Workflow

• 1. Read AGENTS.md; find the vector store, schema and existing embedding code
• 2. Fix the vector-space contract: model, dims, text preprocessing version
• 3. Implement batched, cached embedding calls
• 4. Store vectors with model/dims/hash metadata
• 5. Query with the same model, dims and preprocessing
• 6. Evaluate on labeled queries before shipping

2. The vector-space contract

Store this with every vector (column or collection metadata):

embedding_model = text-embedding-3-small
embedding_dims  = 1536
prep_version    = 2          # bump when chunking/cleaning changes
content_hash    = sha256(prepared_text)

Vectors from different models or dims are not comparable. Mixing them returns plausible but meaningless similarity scores with no error.

3. Batched, cached embedding

import hashlib, os
from openai import OpenAI

client = OpenAI(max_retries=5)   # SDK retries 429/5xx with backoff
MODEL = os.environ.get("EMBED_MODEL", "text-embedding-3-small")
DIMS = int(os.environ.get("EMBED_DIMS", "1536"))
MAX_ITEMS, MAX_TOKENS = 2048, 300_000   # per-request API limits

def prep(text: str) -> str:
    return " ".join(text.split())[:30_000]   # collapse whitespace; hard cap characters

def batches(texts, count_tokens):
    batch, tokens = [], 0
    for t in texts:
        n = count_tokens(t)
        if n > 8192:
            raise ValueError("chunk exceeds 8192 tokens; re-chunk before embedding")
        if batch and (len(batch) == MAX_ITEMS or tokens + n > MAX_TOKENS * 0.9):
            yield batch
            batch, tokens = [], 0
        batch.append(t)
        tokens += n
    if batch:
        yield batch

def key(text: str) -> str:
    return hashlib.sha256(f"{MODEL}:{DIMS}:{text}".encode()).hexdigest()

def embed_all(texts, cache, count_tokens):
    prepared = [prep(t) for t in texts]
    missing = list(dict.fromkeys(p for p in prepared if p and key(p) not in cache))
    for chunk in batches(missing, count_tokens):
        resp = client.embeddings.create(model=MODEL, input=chunk, dimensions=DIMS)
        for item in resp.data:                    # item.index points into chunk
            cache[key(chunk[item.index])] = item.embedding
    return [cache.get(key(p)) if p else None for p in prepared]

Count tokens with tiktoken (cl100k_base for v3 models) if installed; otherwise estimate ~4 characters per token and keep a 10% margin. Empty strings are rejected — filter them out first.

For more than ~50k texts, use the Batch API (/v1/embeddings, half price) instead of synchronous calls — see openai-batch-api.

4–5. Similarity

• OpenAI embeddings are unit-length, so cosine similarity equals the dot product. Use the store’s inner-product operator for speed (<#> in pgvector returns the negative inner product; <=> is cosine distance).
• If you truncate vectors yourself instead of using dimensions, renormalize: v = v[:256]; v = v / np.linalg.norm(v). Prefer the API parameter.
• Raw scores are not probabilities and shift between models. Never hard-code a threshold like > 0.8 without calibrating it on labeled pairs for the exact model and dims.
• Queries and documents must use identical preprocessing. Short queries vs. long documents is normal; asymmetric cleaning is not.

6. Evaluate

Build 30–100 queries with known relevant document ids. Report recall@10 and MRR for the current setup and for any change (model, dims, chunking). For the metric harness see rag-evaluation.

Re-embedding migrations

Changing model or dims means a new vector space:

1. Add a new column/collection (embedding_v2); never overwrite in place.
2. Backfill with the Batch API; track progress by content_hash.
3. Dual-write new content to both spaces during backfill.
4. Evaluate v1 vs v2 on the labeled query set; switch reads behind a flag.
5. Drop the old space only after the user confirms — deletion is irreversible.

Gotchas

• Embedding whole documents: anything over a few hundred tokens dilutes meaning. Chunk first (see rag-chunking).
• Forgetting dimensions on the query side after setting it on documents produces a dimension-mismatch error at best, wrong results with padded stores at worst.
• Re-embedding unchanged text on every deploy; hash-and-skip saves most of the bill.
• Storing vectors as JSON text in a relational column, then computing similarity in app code at scale. Use a vector index.
• Sending PII to the API without checking the data policy; strip or tokenize it in prep.
• Model knowledge cutoff does not matter much for embeddings, but domain jargon does — test with real queries, not synthetic ones only.

Verification

• Unit test: same text twice → one API call (cache hit); batch splitter respects 2,048 items and the token cap.
• Assert len(vector) == DIMS and abs(norm - 1) < 1e-3 for a sample.
• Labeled-query eval shows recall@10 at or above the previous setup.
• Check stored metadata: SELECT embedding_model, embedding_dims, count(*) ... GROUP BY 1,2 returns exactly one row per space.

Output contract

**Vector space:** text-embedding-3-small, 1536 dims, prep_version 2
**Store:** pgvector halfvec(1536) HNSW, inner product
**Volume:** 312k chunks, 1.9 GB; cache hit rate on re-run 97%
**Eval:** recall@10 0.81 → 0.86, MRR 0.62 → 0.69 (64 labeled queries)
**Migration:** embedding_v2 backfilled via Batch API; reads switched behind EMBED_V2 flag