OCR Documents skill for Codex

Done well means: the output PDF looks identical to the scan but is searchable and copyable, text is extracted in reading order, low-confidence regions are flagged rather than trusted, and accuracy was measured on a sample instead of assumed.

First, confirm OCR is needed

pdffonts in.pdf                 # no fonts listed -> image-only pages
pdftotext -l 2 in.pdf - | head  # empty or garbage -> needs OCR
pdfimages -list in.pdf | head   # image resolution (x-ppi / y-ppi) per page

Born-digital PDFs with real text should be extracted directly with $pdf-processing; OCR would only add errors. Mixed files (some scanned pages) are handled with --skip-text below.

Check tools: ocrmypdf --version, tesseract --version, tesseract --list-langs. Installing them (brew install ocrmypdf tesseract-lang, apt install ocrmypdf tesseract-ocr-deu) needs network and usually admin rights; in a Codex sandbox, ask for approval or have the user install them.

Default: OCRmyPDF for PDFs

ocrmypdf -l eng --rotate-pages --deskew --output-type pdfa \
  --sidecar out.txt --jobs 4 scan.pdf searchable.pdf

Situation	Option
Multiple languages on the page	-l eng+deu (order: most common first; each must be installed)
Some pages already have text	--skip-text (OCRmyPDF 17 also accepts --mode skip)
Bad old OCR layer to replace	--redo-ocr
Text is vector or broken fonts, rasterise everything	--force-ocr (loses vector quality)
Speckled or dark scans	--clean (needs unpaper; cleans only the image used for OCR)
Smaller output	--optimize 2 or 3 (3 needs pngquant/jbig2enc)
Only some pages	--pages 1-3,7
Huge or slow pages	--tesseract-timeout 300, --skip-big 50 (megapixels)

Exit codes worth handling in scripts: 6 already has text (rerun with --skip-text, --redo-ocr or --force-ocr), 8 encrypted (ask for the password; never try to remove protection on a file the user does not control), 2 unreadable input, 3 missing dependency. Digitally signed PDFs refuse OCR unless you pass --invalidate-digital-signatures; tell the user the signature will no longer verify and get confirmation first.

Batch a folder, continuing past failures:

mkdir -p ocr logs
for f in scans/*.pdf; do
  out="ocr/$(basename "$f")"
  ocrmypdf -l eng --skip-text --rotate-pages --deskew -j 4 "$f" "$out" >>logs/ocr.log 2>&1
  rc=$?
  [ $rc -eq 0 ] || echo "$rc $f" >>logs/failed.txt
done

Images and fine control: Tesseract

tesseract page.png out -l eng --psm 6 --oem 1 --dpi 300      # -> out.txt
tesseract page.png out -l eng pdf                            # searchable single-page PDF
tesseract page.png out -l eng --psm 6 tsv                    # words with boxes and confidence

--psm	Use for
3 (default)	Full pages with mixed layout
4	Single column of text of varying sizes (receipts, statements)
6	One uniform block of text
7	A single line (cropped field)
11 / 12	Sparse text scattered on the page (forms, labels)

Language data: tessdata_fast (default packages) is quicker; tessdata_best is more accurate for difficult scans. Point Tesseract at a folder with TESSDATA_PREFIX.

Preprocessing that actually helps

Tesseract does best with about 300 DPI, dark text on white, and capital letters roughly 20-40 px tall. Fix the input before blaming the engine:

import cv2
import numpy as np
import pytesseract
from PIL import Image
from pytesseract import Output

def prepare(path: str) -> np.ndarray:
    img = cv2.imread(path, cv2.IMREAD_GRAYSCALE)
    h = img.shape[0]
    if h < 2500:                                   # roughly < 300 DPI for a letter/A4 page
        img = cv2.resize(img, None, fx=2, fy=2, interpolation=cv2.INTER_CUBIC)
    img = cv2.fastNlMeansDenoising(img, h=10)
    _, bw = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
    return bw

bw = prepare("receipt.jpg")
data = pytesseract.image_to_data(Image.fromarray(bw), lang="eng", config="--psm 4",
                                 output_type=Output.DICT)
words = [(w, float(c)) for w, c in zip(data["text"], data["conf"]) if w.strip()]
text = " ".join(w for w, _ in words)
low = [w for w, c in words if c < 60]
print(f"{len(words)} words, {len(low)} below 60 confidence: {low[:15]}")

• Uneven lighting (phone photos): use cv2.adaptiveThreshold(img, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 31, 15) instead of Otsu.
• Photos at an angle need perspective correction (find the page contour, then cv2.getPerspectiveTransform), not just deskew.
• Rasterise PDFs at 300 DPI for custom pipelines: pdftoppm -r 300 -gray -png in.pdf page.
• Do not sharpen or binarise for OCRmyPDF; it handles its own preprocessing and keeps the original image in the output.

Known failure modes

Symptom	Cause	Fix
S0 read as SO, 1 as l, 0 as O	Ambiguous glyphs in codes and IDs	Post-validate with patterns ([A-Z]\d+), or whitelist characters for that field: -c tessedit_char_whitelist=0123456789
Columns interleaved	Wrong segmentation	--psm 4 or crop columns first
Tables come out as word soup	OCR has no table model	OCR to TSV and rebuild rows from word boxes, or use $form-data-extraction
Empty output on a clear page	Inverted colours (white on dark) or tiny text	Invert the image; upscale
Handwriting mostly wrong	Tesseract is trained on print	Set expectations; cloud handwriting OCR only with the user’s consent to upload
Output PDF much larger	Images re-encoded losslessly	--optimize 2, or keep --output-type pdf

Scanned documents are often sensitive (IDs, medical, contracts). Keep OCR local; any cloud OCR service requires the user’s explicit approval to send the files.

Measure accuracy

Pick 3-5 representative pages, type (or obtain) the true text, and compute character error rate:

def cer(ref: str, hyp: str) -> float:
    ref, hyp = " ".join(ref.split()), " ".join(hyp.split())
    prev = list(range(len(hyp) + 1))
    for i, rc in enumerate(ref, 1):
        cur = [i]
        for j, hc in enumerate(hyp, 1):
            cur.append(min(prev[j] + 1, cur[j - 1] + 1, prev[j - 1] + (rc != hc)))
        prev = cur
    return prev[-1] / max(len(ref), 1)

Rough guide: under 2% CER is good print OCR; 2-10% is usable for search but needs review for data entry; above 10% means fix the input (resolution, lighting, language pack, --psm) before scaling up.

Verification

• pdffonts searchable.pdf now lists a font for the invisible text layer (the name varies by OCRmyPDF version); pdftotext returns text for every page.
• Search a few distinctive words from the scan in the output PDF.
• Page count unchanged; first and last page rendered and compared with the original.
• Batch: logs/failed.txt empty or each exit code explained.

Report

**Input:** 412 scanned PDFs (2,960 pages), 200-300 DPI, English and German
**Output:** ocr/ - PDF/A-2b, searchable; sidecar text per file
**Settings:** ocrmypdf -l eng+deu --skip-text --rotate-pages --deskew -O 2
**Accuracy:** CER 1.4% on a 5-page ground-truth sample; low-confidence words listed in review.csv
**Failures:** 3 files encrypted (exit 8) - need passwords