PDF Batch Workflows skill for Codex

Done well means: inputs are never modified, every input has exactly one manifest row saying what happened to it, a crashed run resumes where it stopped, failures are listed with reasons instead of aborting the batch, and every output passes a structural check and a page-count check. Single-file PDF techniques live in $pdf-processing; this skill is about running them at scale.

Plan the batch

Question	Why
How many files, total size, largest file?	Sizes the worker count and disk space (OCR output can be 1 to 3x input)
Born-digital, scanned, or mixed?	Decides OCR mode (--skip-text vs --force-ocr vs --redo-ocr)
Encrypted, signed or form PDFs present?	Signatures break on any change; forms may flatten
Output naming and folder structure?	Must be deterministic so reruns overwrite the same output, never inputs
Order of operations?	OCR before stamping (stamps are not text you want OCRed), compress last

Profile first:

find in -type f -iname '*.pdf' | wc -l
du -sh in
find in -iname '*.pdf' | shuf -n 20 | while IFS= read -r f; do   # sample 20
  printf '%s\t%s\t%s\n' "$(qpdf --show-npages "$f" 2>/dev/null || echo ERR)" \
    "$(pdffonts "$f" 2>/dev/null | tail -n +3 | wc -l)" "$f"
done      # pages, font count (0 fonts = probably scanned), path

shuf is GNU coreutils; on macOS use gshuf or Python’s random.sample.

Workflow

- [ ] 1. Profile a sample: page counts, text layer, encryption, signatures
- [ ] 2. Write process_one(in_path, out_path) for a single file; test on 5 diverse files
- [ ] 3. Build the manifest (path, size, sha256) for all inputs
- [ ] 4. Run in parallel with per-file status written to the manifest as each finishes
- [ ] 5. Verify outputs (qpdf --check, page counts) and collect failures
- [ ] 6. Retry failures once with adjusted options; report the rest for human review

Tools: qpdf, ocrmypdf (needs Tesseract and Ghostscript), ghostscript, Python pypdf. Installing them needs network and possibly a package manager with elevated rights; ask for approval in a sandboxed Codex session or check what is already installed (command -v qpdf ocrmypdf gs tesseract).

Manifest-driven runner

import csv, hashlib, subprocess, time
from concurrent.futures import ProcessPoolExecutor, as_completed
from pathlib import Path

IN, OUT = Path("in"), Path("out")
MANIFEST = Path("manifest.csv")
FIELDS = ["path", "sha256", "pages_in", "pages_out", "status", "error", "seconds"]

def sha256(p: Path) -> str:
    h = hashlib.sha256()
    with p.open("rb") as f:
        for chunk in iter(lambda: f.read(1 << 20), b""):
            h.update(chunk)
    return h.hexdigest()

def npages(p: Path) -> int:
    return int(subprocess.run(["qpdf", "--show-npages", str(p)], capture_output=True,
                              text=True, check=True).stdout)

def process_one(src: Path) -> dict:
    rel = src.relative_to(IN)
    dst = OUT / rel
    dst.parent.mkdir(parents=True, exist_ok=True)
    row = {"path": str(rel), "sha256": sha256(src), "pages_out": "", "error": ""}
    t0 = time.monotonic()
    try:
        row["pages_in"] = npages(src)
        tmp = dst.with_suffix(".tmp.pdf")
        r = subprocess.run(["ocrmypdf", "--skip-text", "--rotate-pages", "--deskew",
                            "-l", "eng", "--jobs", "1", "--output-type", "pdf",
                            str(src), str(tmp)], capture_output=True, text=True)
        if r.returncode == 8:                          # encrypted: needs a password
            row.update(status="skipped", error="encrypted",
                       seconds=round(time.monotonic() - t0, 1))
            return row
        if r.returncode != 0:
            raise RuntimeError(f"ocrmypdf exit {r.returncode}: {r.stderr.strip()[-300:]}")
        subprocess.run(["qpdf", "--check", str(tmp)], capture_output=True, check=True)
        row["pages_out"] = npages(tmp)
        if row["pages_out"] != row["pages_in"]:
            raise RuntimeError("page count changed")
        tmp.replace(dst)                               # atomic publish of the output
        row["status"] = "ok"
    except Exception as exc:                            # one bad file never stops the batch
        row["status"], row["error"] = "failed", str(exc)[:500]
    row["seconds"] = round(time.monotonic() - t0, 1)
    return row

def main(workers: int = 4) -> None:
    done = set()
    if MANIFEST.exists():
        with MANIFEST.open() as f:
            done = {(r["path"], r["sha256"]) for r in csv.DictReader(f) if r["status"] == "ok"}
    todo = [p for p in sorted(IN.rglob("*.pdf"))
            if (str(p.relative_to(IN)), sha256(p)) not in done]
    new = not MANIFEST.exists()
    with MANIFEST.open("a", newline="") as f, ProcessPoolExecutor(workers) as pool:
        w = csv.DictWriter(f, FIELDS)
        if new:
            w.writeheader()
        for fut in as_completed(pool.submit(process_one, p) for p in todo):
            w.writerow(fut.result()); f.flush()        # progress survives a crash

if __name__ == "__main__":
    main()

• Workers x per-file threads must not exceed CPU cores: with ProcessPoolExecutor(4) give ocrmypdf --jobs 1; for a few huge files, use 1 worker and --jobs = cores instead.
• Write to a temp name and rename on success, so a crash never leaves a truncated file that looks finished.
• The manifest is append-only; the latest row per path wins. Resume skips (path, sha256) pairs already ok, so a changed input is reprocessed automatically.

Operation recipes

OCR (scans and mixed archives):

ocrmypdf --skip-text --rotate-pages --deskew -l eng+deu --output-type pdfa in.pdf out.pdf

Situation	Option
Mixed archive, keep existing text	--skip-text (pages with text are passed through)
Old bad OCR layer to replace	--redo-ocr
Text layer is garbage and must go	--force-ocr (rasterises those pages; output gets larger)
Archival requirement	--output-type pdfa (default); use pdf when PDF/A conversion fails

ocrmypdf exit code 6 means the file already has text (only without a skip, redo or force mode) and 8 means it is encrypted; record both as “skipped”, not failures.

Stamp a watermark or header on every page (stamp.pdf is one page, same size as the targets):

qpdf in.pdf --overlay stamp.pdf --repeat=1 -- out.pdf      # on top of content
qpdf in.pdf --underlay stamp.pdf --repeat=1 -- out.pdf     # behind content

Bates numbering (unique per page across the batch) with pypdf, after generating one numbered stamp page per output page with a PDF library such as reportlab:

from pypdf import PdfReader, PdfWriter

def bates(src, stamps_pdf, dst):
    w = PdfWriter(clone_from=src)
    stamps = PdfReader(stamps_pdf)
    for page, stamp in zip(w.pages, stamps.pages, strict=True):
        page.merge_page(stamp, over=True)
    w.write(dst)

Keep the Bates counter in the manifest (first_number, last_number per file) and assign ranges before parallel processing, so numbers stay sequential and gapless.

Compress large scans (lossy; run on a copy and compare):

gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.7 -dPDFSETTINGS=/ebook \
   -dNOPAUSE -dBATCH -dQUIET -sOutputFile=small.pdf big.pdf

/ebook is about 150 dpi images, /printer 300 dpi. Ghostscript rewrites the whole file: it can drop form fields, annotations and signatures, so never compress legal originals.

Merge per group (customer, case, month) from the manifest, not from a glob, so order is explicit: qpdf --empty --pages $(cat group-0042.txt) -- merged/0042.pdf. For bookmarks and splitting by outline, use $pdf-merge-split.

Gotchas

• Digitally signed PDFs: any rewrite invalidates the signature. Detect them first (pypdf field type /Sig) and route to a human.
• Encrypted files: process only with passwords the user supplies; list the rest as skipped.
• Mixed page sizes: a one-size stamp lands in the wrong place on A4 versus Letter or landscape pages. Generate stamps per page size.
• Tesseract language packs must be installed for each -l language; a missing pack fails every file the same way. Run one file before the batch.
• Disk space: OCR temp files live in $TMPDIR; thousands of parallel jobs can fill it.
• Path names with non-ASCII characters and spaces: always pass argument lists to subprocess, never a shell string.

Verification

• Every input has exactly one latest manifest row; ok + failed + skipped = total inputs.
• All ok outputs pass qpdf --check, and pages_out == pages_in for every row.
• Spot-check 10 random outputs: OCR text searchable (pdftotext out.pdf - | head), stamp visible and placed correctly on portrait and landscape pages.
• Rerun the batch: it reports 0 files to process.

Report

**Batch:** 3,412 PDFs (18.6 GB) from in/ -> out/, OCR (eng) + "CONFIDENTIAL" overlay
**Result:** 3,377 ok, 21 skipped (encrypted, list in manifest), 14 failed
**Failures:** 9 corrupt (qpdf --check), 5 Tesseract timeouts on 600 dpi scans (retry with --tesseract-timeout)
**Checks:** page counts equal for all ok rows; 10-file spot check passed
**Manifest:** manifest.csv (sha256 per input, resumable)