Form Data Extraction skill for Codex

Extraction is done when every output record matches a declared schema, amounts are exact decimals that reconcile (lines to subtotal, subtotal plus tax to total), each field carries its source and confidence, and accuracy is measured on a labelled sample rather than eyeballed on three documents.

Pick the cheapest reliable source first

Check these in order; stop at the first that yields the data:

Order	Source	How to detect	Reliability
1	Embedded e-invoice XML (Factur-X / ZUGFeRD in PDF/A-3, or standalone UBL / CII / XRechnung)	PDF attachments named factur-x.xml, zugferd-invoice.xml, xrechnung.xml; or the file itself is XML	Exact; parse XML, no ML
2	AcroForm field values	reader.get_fields() returns values	Exact for filled forms
3	Native text layer with stable layout (one vendor, one template)	pdfplumber text is non-empty and ordered	High with anchors and regions
4	Varied layouts, many vendors	Text exists but layouts differ	LLM structured output or cloud document AI
5	Scans and phone photos	No text layer	OCR first (see ocr-documents), then 3 or 4, or cloud document AI directly

from pypdf import PdfReader

reader = PdfReader("invoice.pdf")
print("attachments:", list(reader.attachments))          # {name: [bytes, ...]}
print("form fields:", {k: v.get("/V") for k, v in (reader.get_fields() or {}).items()})
print("has text:", bool((reader.pages[0].extract_text() or "").strip()))

Structured e-invoicing is mandatory or being phased in across several EU countries (in Germany, B2B businesses must be able to receive e-invoices since 1 January 2025, with issuing obligations phased in afterwards). Always look for the XML before parsing pixels.

Workflow

- [ ] 1. Define the output schema and acceptance rules
- [ ] 2. Build a labelled sample (at least 20-50 documents across vendors and formats)
- [ ] 3. Route each document to the cheapest reliable source
- [ ] 4. Extract with provenance
- [ ] 5. Normalize and validate
- [ ] 6. Measure accuracy, set thresholds, route the rest to review

1. Schema

from datetime import date
from decimal import Decimal
from pydantic import BaseModel, Field

class LineItem(BaseModel):
    description: str
    quantity: Decimal | None = None
    unit_price: Decimal | None = None
    amount: Decimal                       # net line amount as printed

class Invoice(BaseModel):
    vendor_name: str
    vendor_tax_id: str | None = None      # VAT/GST/EIN as printed
    invoice_number: str
    invoice_date: date
    due_date: date | None = None
    currency: str = Field(pattern=r"^[A-Z]{3}$")   # ISO 4217
    subtotal: Decimal | None = None
    tax_total: Decimal | None = None
    total: Decimal
    line_items: list[LineItem] = []

Decide up front: are amounts net or gross per line, how are credit notes signed, which fields are required for the downstream system, and what happens to a document that fails.

3–4. Extraction methods

Layout anchors (single template, native text). Find a label, read the value to its right or below, inside a bounding box. Coordinates come from extract_words.

import pdfplumber

def value_right_of(page, label_pattern):
    """First token printed to the right of a label on the same line."""
    hits = page.search(label_pattern, regex=True, case=False)
    if not hits:
        return None
    h = hits[0]
    band = page.within_bbox((h["x1"], h["top"] - 2, page.width, h["bottom"] + 2))
    text = (band.extract_text() or "").strip()
    return text.split()[0] if text else None

with pdfplumber.open("invoice.pdf") as pdf:
    page = pdf.pages[0]
    invoice_no = value_right_of(page, r"invoice\s+(no|number)\.?:?")
    total_raw = value_right_of(page, r"total\s+due")
    tables = page.extract_tables({"vertical_strategy": "lines", "horizontal_strategy": "lines"})

Ruled tables: "lines" strategy. Unruled tables: "text" strategy, and check column boundaries on several samples. Keep per-vendor templates in config (anchors, regions, date format, decimal separator), not in code branches.

LLM structured output (many layouts). Send the page text (and the page image for scans or complex layouts) with a strict JSON schema, and ask for verbatim values plus the printed text each came from. With the OpenAI Python SDK, for example:

from openai import OpenAI
client = OpenAI()   # needs network and an API key from the environment
resp = client.responses.parse(
    model=MODEL,
    input=[
        {"role": "system", "content": "Extract invoice fields exactly as printed. Use null when a field is absent. Do not compute missing totals."},
        {"role": "user", "content": page_text},
    ],
    text_format=Invoice,
)
invoice = resp.output_parsed

Tell the model not to calculate or guess; derived values are your code’s job, so validation can catch disagreements. Sending documents to an external API is a data transfer: confirm the user is allowed to send them, and redact what is not needed.

Cloud document AI. Prebuilt invoice/receipt models (Azure AI Document Intelligence prebuilt-invoice and prebuilt-receipt, AWS Textract AnalyzeExpense, Google Document AI invoice parser) return fields with confidences and bounding boxes. Good default for high volumes of scans; still validate their output with the rules below.

Record provenance for every field: method, page, bounding box or matched text, and a confidence (model score, or 1.0 for XML and form fields).

5. Normalize and validate

Normalization:

• Amounts: strip currency symbols and spaces, detect decimal separator per document (1.234,56 vs 1,234.56); parse to Decimal, never float. Parentheses or trailing minus mean negative.
• Dates: parse with the vendor’s known format; 03/04/2026 is ambiguous without it. Reject dates more than a set window from the receipt date.
• IDs: keep as strings; normalize whitespace and case only.

Validation rules (failures route to review, they do not get “fixed” silently):

from decimal import Decimal

TOL = Decimal("0.02")

def check(inv: Invoice) -> list[str]:
    issues = []
    if inv.line_items:
        lines = sum((li.amount for li in inv.line_items), Decimal(0))
        if inv.subtotal is not None and abs(lines - inv.subtotal) > TOL:
            issues.append(f"lines {lines} != subtotal {inv.subtotal}")
    if inv.subtotal is not None and inv.tax_total is not None:
        if abs(inv.subtotal + inv.tax_total - inv.total) > TOL:
            issues.append("subtotal + tax != total")
    for li in inv.line_items:
        if li.quantity is not None and li.unit_price is not None:
            if abs(li.quantity * li.unit_price - li.amount) > TOL:
                issues.append(f"qty x price != amount for '{li.description[:30]}'")
    if inv.due_date and inv.due_date < inv.invoice_date:
        issues.append("due date before invoice date")
    return issues

Also check: duplicate detection (vendor + invoice number + total seen before), vendor tax ID format for its country, IBAN checksum when bank details are extracted, currency consistent with vendor country, totals not zero.

6. Measure and route

• Field-level accuracy on the labelled sample: exact match after normalization for IDs and dates; exact decimal match for amounts; line items by matched count and amount sum.
• Report per field and per source route, not one blended number.
• Set auto-accept rules: all required fields present, validation passes, every field’s confidence above the threshold chosen from the sample. Everything else goes to a human queue showing the page image with the extracted boxes highlighted.
• Re-run the sample on every prompt, template or model change and compare.

Gotchas

• Multi-page invoices: totals on the last page, line items spanning pages with repeated headers; dedupe header rows.
• Receipts print “Total” several times (subtotal, total, total paid, change); anchor on position and validate arithmetic.
• Discounts and shipping lines break sum(lines) == subtotal unless modelled as lines.
• Credit notes look like invoices with negative or positive amounts; classify document type first.
• PDF text order can differ from visual order; use coordinates, not raw text order, for layout parsing.
• Never log full documents or bank details while debugging; mask them.

Output

{
  "document": "inbox/2026-10/acme-4471.pdf",
  "route": "llm_structured",
  "status": "needs_review",
  "issues": ["subtotal + tax != total"],
  "data": { "vendor_name": "Acme Ltd", "invoice_number": "4471", "total": "1440.00", "currency": "GBP" },
  "provenance": { "total": { "page": 2, "text": "TOTAL £1,440.00", "confidence": 0.93 } }
}

Pipeline summary for the user: documents processed by route, auto-accepted vs review, field accuracy on the labelled sample, and the top three failure causes.