PDF Processing skill for Codex

Done well means: the output text or table matches what a human sees on the page, page operations are lossless (no re-rendering, no rasterising), and every claim about the result was checked against the file, not assumed from the code.

First, classify the PDF

Every decision depends on what kind of PDF it is. Inspect before writing any pipeline:

qpdf --check in.pdf                       # structural damage, encryption
qpdf --show-npages in.pdf
pdffonts in.pdf | head                    # poppler; empty list => no text layer
pdftotext -f 1 -l 1 -layout in.pdf - | head -40

from pypdf import PdfReader

r = PdfReader("in.pdf")
print("pages:", len(r.pages), "encrypted:", r.is_encrypted)
print("meta:", r.metadata)
print("form fields:", bool(r.get_fields()))
sample = r.pages[0].extract_text() or ""
print("chars on p1:", len(sample.strip()))

Finding	Meaning	Route
Text extracts, fonts listed	Born-digital PDF	Continue here
No text, no fonts, pages are images	Scanned	Hand off to $ocr-documents first
Text is garbled ((cid:12), mojibake)	Font without ToUnicode map	Try pdftotext; if still bad, OCR the page
get_fields() non-empty	AcroForm	Field values live in the form, not the text: use $pdf-forms
is_encrypted true	Needs password or only owner-locked	r.decrypt("") first; ask the user for a password otherwise

Never try to remove a password the user does not own or was not given. Owner-locked files (empty user password) open with decrypt(""); that is normal, not a bypass.

Tool choice

Task	Default	Why
Merge, split, rotate, reorder, decrypt	pypdf in code, qpdf on the CLI	Lossless object copy, keeps fonts and links
Plain text, reading order	pdfplumber or pdftotext -layout	Better spacing than pypdf’s plain mode
Tables	pdfplumber extract_tables with tuned settings	Explicit control over line vs text strategies
Word coordinates, cropping regions	pdfplumber extract_words, page.crop	Gives x0/top/x1/bottom per word
Render a page to PNG (to look at it)	pdfplumber page.to_image() or pdftoppm -r 150	Uses pypdfium2 / poppler

Install when missing: pip install pypdf pdfplumber (needs network; in a Codex sandbox with network off, ask for approval or check whether the project already vendors them). Read AGENTS.md for the project’s package manager and use it (uv, poetry, pip-tools).

Workflow

- [ ] 1. Classify the PDF (table above) and note page count
- [ ] 2. Render 1-2 representative pages to PNG and look at them
- [ ] 3. Run the operation on a small page range first
- [ ] 4. Run on the full file / folder, streaming page by page
- [ ] 5. Verify output (counts, spot checks, qpdf --check)
- [ ] 6. Report what was produced and any pages that need human review

Text extraction

import pdfplumber

with pdfplumber.open("in.pdf") as pdf:
    for i, page in enumerate(pdf.pages, start=1):
        text = page.extract_text(x_tolerance=1.5, y_tolerance=3) or ""
        print(f"--- page {i} ---\n{text}")

• Multi-column layouts: crop each column and extract separately, otherwise lines from both columns interleave. left = page.crop((0, 0, page.width / 2, page.height)).
• Headers and footers: crop them out by y-coordinate rather than regex-deleting lines afterwards (page.crop((0, 60, page.width, page.height - 50))).
• pypdf alternative with column-preserving spacing: page.extract_text(extraction_mode="layout").
• Hyphenated line breaks (infor-\nmation) and ligatures (fi) need a post-pass: unicodedata.normalize("NFKC", text) fixes ligatures.

Table extraction

Start by drawing what pdfplumber detects; tune settings until the overlay matches.

import csv
import pdfplumber

settings = {
    "vertical_strategy": "lines",     # ruled tables; use "text" for whitespace-aligned
    "horizontal_strategy": "lines",
    "snap_tolerance": 3,
    "intersection_tolerance": 5,
}

with pdfplumber.open("statement.pdf") as pdf:
    page = pdf.pages[2]
    page.to_image(resolution=120).debug_tablefinder(settings).save("p3-tables.png")
    rows = []
    for table in page.extract_tables(settings):
        rows.extend(table)

with open("p3.csv", "w", newline="", encoding="utf-8") as f:
    csv.writer(f).writerows(rows)

Symptom	Fix
No tables found on a table without ruling lines	vertical_strategy and horizontal_strategy = "text"
Cells merged across columns	Lower snap_tolerance; add explicit_vertical_lines=[x1, x2, ...] from word x0 values
A table spans pages	Extract per page, drop repeated header rows, then concatenate
Wrapped cell text split into two rows	Merge rows whose first column is None or empty into the previous row
Numbers as strings with 1,234.50 or (45.00)	Normalise after extraction; parentheses mean negative in finance tables

Page operations (lossless)

from pypdf import PdfReader, PdfWriter

# Merge whole files, or ranges: pages=(start, stop) is zero-based, stop exclusive
w = PdfWriter()
w.append("cover.pdf")
w.append("report.pdf", pages=(0, 10))
w.append("appendix.pdf", outline_item="Appendix")    # adds a bookmark
w.write("combined.pdf")

# Split into single pages
r = PdfReader("combined.pdf")
for n, page in enumerate(r.pages, start=1):
    single = PdfWriter()
    single.add_page(page)
    single.write(f"page-{n:03d}.pdf")

# Rotate pages 2 and 5 clockwise (multiples of 90 only) and reorder
r = PdfReader("scan.pdf")
w = PdfWriter(clone_from=r)
for idx in (1, 4):
    w.pages[idx].rotate(90)
w.write("fixed.pdf")

Equivalent qpdf commands, preferred for one-off CLI jobs and very large files:

qpdf --empty --pages a.pdf b.pdf 1-3 -- merged.pdf
qpdf in.pdf --pages . 5-1 -- reversed.pdf           # reverse pages 1-5
qpdf in.pdf --rotate=+90:2,5 out.pdf
qpdf --split-pages=10 in.pdf chunk-%d.pdf            # 10 pages per file
qpdf --decrypt --password='given-by-user' in.pdf plain.pdf

Bookmark editing, compression and linearisation belong to $pdf-merge-split.

Batch processing

• Process page by page and close files (with pdfplumber.open(...)); pdfplumber caches layout objects per page, so call page.flush_cache() (or page.close()) in long loops.
• One bad file must not kill the batch: catch pypdf.errors.PdfReadError per file, log the filename, continue, and list failures in the final report.
• Write outputs to a separate folder; never overwrite inputs in place.

Gotchas

• PdfWriter() plus add_page drops document-level things (outline, named destinations, form fields). Use PdfWriter(clone_from=reader) when editing a single file.
• Rotation via /Rotate is a viewer instruction; text extraction coordinates stay in the unrotated space. Extract first, or account for page.rotation.
• extract_text() returns None-ish or empty on image-only pages; check length per page and flag those pages instead of silently emitting blanks.
• Page labels (i, ii, 1, 2) differ from physical indexes. Confirm which one the user means by “page 3”; r.page_labels shows the mapping.
• Merging files with identically named form fields makes them share values. Flatten or rename fields first ($pdf-forms).
• Incremental-update PDFs can contain old, “deleted” content. Visible text removal is not redaction: use $document-redaction.

Verification

qpdf --check out.pdf && qpdf --show-npages out.pdf
pdftoppm -f 1 -l 1 -r 80 -png out.pdf preview      # look at the first page

• Page count of the output equals the sum you expected.
• For extraction: compare 3 random pages of extracted text against a rendered PNG.
• For tables: row count and a column total match a figure printed in the PDF.
• If a check fails, re-render the failing page with debug_tablefinder and adjust settings rather than patching the output by hand.

Report template

**Input:** statement-2025-09.pdf (14 pages, born-digital, not encrypted)
**Output:** tables/statement.csv (412 rows), text/statement.txt
**Method:** pdfplumber 0.11, lines strategy, header row de-duplicated across pages 3-12
**Checks:** closing balance in CSV (18,204.55) equals PDF page 14; qpdf --check clean
**Needs review:** page 7 is a scanned insert with no text layer (route to OCR)