Document Redaction skill for Codex

A redaction is done when the removed content is gone from every layer of the file (text, images, vector art, annotations, form fields, metadata, earlier revisions) and a fresh extraction of the output finds nothing. A black rectangle drawn on top is not a redaction.

Hard rules

• Never overwrite the original. Write to a new file and keep the source untouched.
• Never save a redacted PDF incrementally. Incremental updates append to the file, so the original bytes stay recoverable after %%EOF.
• Never rely on drawing shapes, highlighting in black, setting text color to white, or covering with an image. All of these leave the text extractable.
• Treat the output as unverified until step 6 passes. Say so if you stop earlier.
• Redaction lists are sensitive. Do not echo the found values into chat or commit them; report counts and locations (page, field type) instead.

Workflow

- [ ] 1. Define scope: what to remove, for whom, which file types
- [ ] 2. Inspect the files (text layer? scans? forms? attachments? revisions?)
- [ ] 3. Detect candidates (explicit terms + PII patterns + review)
- [ ] 4. Apply true redaction
- [ ] 5. Scrub metadata and hidden content, rewrite the file
- [ ] 6. Verify by re-extracting and scanning the output
- [ ] 7. Write the redaction log

1. Define scope

Ask (or infer from the request) and write down:

• Categories: personal names, emails, phone numbers, postal addresses, government IDs, bank/card numbers, dates of birth, case numbers, faces/signatures in images.
• Exceptions: e.g. keep the company’s own name, keep the public officer’s name.
• Replacement style: solid box (default), box with label such as [REDACTED: EMAIL], or pseudonym (Person A) for anonymized analysis copies. Pseudonyms must be consistent across the whole document set; keep the mapping table out of the output folder.
• Legal basis, if any (FOIA exemption codes, GDPR article). Labels often must cite it.

2. Inspect before touching anything

Install note: pip install pymupdf needs network; ask for approval if the sandbox blocks it.

import pymupdf  # PyMuPDF >= 1.24; older code uses "import fitz"

doc = pymupdf.open("in.pdf")
print("pages", doc.page_count, "encrypted", doc.is_encrypted)
print("metadata", doc.metadata)
print("embedded files", doc.embfile_names())
print("form fields", sum(1 for p in doc for _ in p.widgets()))
for p in doc:
    text = p.get_text().strip()
    print(p.number + 1, "chars", len(text), "images", len(p.get_images()),
          "annots", len(list(p.annots())))

Decide from what you see:

Finding	Consequence
Page has images but no text	Scanned page. OCR to locate text (see ocr-documents), then redact the image pixels at those boxes
Embedded files / attachments	Inspect and delete or redact them separately; doc.scrub() can remove them
Form fields (widgets)	Field values live outside page text; flatten or delete fields that hold PII
Many %%EOF markers (grep -a -c %%EOF in.pdf)	Earlier revisions exist; a full rewrite in step 5 drops them
Encrypted	Need the password from the user; never try to crack it
.docx input	See “Word documents” below before choosing a route

3. Detect candidates

Combine three sources, then review:

1. Explicit terms supplied by the user (names, matter numbers). Search case-insensitively and include variants: initials, possessives, line-broken names, all-caps headers.

2. Pattern detection with validation, not bare regex. Card numbers must pass Luhn, IBANs must pass mod-97, SSNs must not start with 000/666/9xx. The bundled scanner does this on extracted text:

   python3 scripts/scan_pii.py extracted.txt            # human report
   python3 scripts/scan_pii.py --json extracted.txt     # machine-readable, values masked

3. NER for names and addresses when there is no term list: Microsoft Presidio (presidio-analyzer plus a spaCy model; both need network to install). Treat its output as candidates with scores; names are where false negatives hide.

Produce a candidate table (page, category, masked value, source, decision) and get user sign-off on ambiguous entries before applying. Over-redaction of non-sensitive context is also a defect when the document must stay useful.

4. Apply true redaction (PDF)

import pymupdf

TERMS = ["Jane Q. Example", "ACCT-99812"]  # from step 3, approved list

doc = pymupdf.open("in.pdf")
hits = 0
for page in doc:
    for term in TERMS:
        for rect in page.search_for(term):      # list of Rect; case-insensitive
            page.add_redact_annot(rect, fill=(0, 0, 0))
            hits += 1
    # images=PDF_REDACT_IMAGE_PIXELS blanks only the overlapped pixels of images;
    # the default removes any image that touches a redaction box entirely.
    page.apply_redactions(images=pymupdf.PDF_REDACT_IMAGE_PIXELS)
print("redacted areas:", hits)

Notes:

• search_for matches on one line; a phrase broken across lines returns nothing. Search for each part as well, or build rects from page.get_text("words") sequences.
• For labelled boxes pass text="[REDACTED]", fontsize=8, fill=(0,0,0) and text_color=(1,1,1) to add_redact_annot; check the label fits the box.
• For regions (signatures, photos, stamps) pass a pymupdf.Rect(x0, y0, x1, y1) in points from the top-left; derive it from page.get_image_rects(xref) or OCR boxes.
• Vector-drawn text (outlined fonts) is not found by search. If the source is outlined, redact by region and verify visually.
• Redacting form fields: delete the widget (page.delete_widget(w)) when its value is sensitive; redaction annots do not clear AcroForm values.

5. Scrub and rewrite

doc.set_metadata({})                      # title, author, subject, keywords, producer...
doc.del_xml_metadata()                    # XMP packet
doc.scrub(attached_files=True, clean_pages=True, embedded_files=True,
          hidden_text=True, javascript=True, metadata=True, redactions=True,
          remove_links=True, reset_fields=True, thumbnails=True, xml_metadata=True)
doc.save("out.pdf", garbage=4, deflate=True, clean=True)  # full rewrite, never incremental=True

scrub() is aggressive (it removes links and resets fields). If the recipient needs links or form fields, pass False for those flags and handle them by hand.

For maximum assurance (court filings, leaked-risk documents) rasterize the redacted pages to images at 200 to 300 dpi and rebuild the PDF; then OCR if the recipient needs search. This destroys hidden text, fonts and structure, and also accessibility, so offer it as a choice rather than a default.

Word documents (.docx)

Content hides in places a visible edit misses: tracked deletions (w:del), comments, headers/footers, footnotes, text boxes, alt text on images, hidden text (w:vanish), document properties (docProps/core.xml, docProps/app.xml, docProps/custom.xml), embedded objects, and the file name itself.

Recommended route:

1. Accept or reject all tracked changes and delete comments first (see word-tracked-changes).
2. Replace sensitive runs with the label text in every part: word/document.xml, header*.xml, footer*.xml, footnotes.xml, endnotes.xml, comments.xml. Note that one word can be split across several w:r runs, so match on joined paragraph text, then rewrite the runs.
3. Clear core and custom properties; remove docProps/thumbnail.* if present.
4. If the recipient does not need an editable file, convert to PDF and run the PDF workflow on the result. That is the safer default.

6. Verify

Run all of these against the output, never the input:

python3 -c "import pymupdf,sys; d=pymupdf.open(sys.argv[1]); print(''.join(p.get_text() for p in d))" out.pdf > out.txt
python3 scripts/scan_pii.py out.txt                     # expect: no findings
grep -i -c -f approved_terms.txt out.txt                # expect: 0
pdftotext out.pdf - | grep -i "jane"                    # second extractor (poppler), if installed
grep -a -c "%%EOF" out.pdf                                # expect: 1
qpdf --check out.pdf                                    # structural sanity, if installed

Also check: doc.metadata is empty, doc.embfile_names() is empty, page count unchanged, and render two or three pages to PNG (page.get_pixmap(dpi=100).save(...)) to confirm the boxes cover what they should and nothing more. For docx, unzip and grep -ril each term across all XML parts.

If any check finds a term: do not patch the output. Fix the candidate list or the method and rerun from step 4 on the original.

Gotchas

• Copy-pasting from a PDF viewer is the classic leak test; extraction tools are stricter.
• Ligatures and kerning can split words (fi, letter-spaced headings). Search for fragments and verify with the scanner.
• Redacting a scanned page without OCR leaves the invisible OCR text layer if one exists.
• Bates numbers, headers and footers repeat on every page; one miss there leaks everywhere.
• Spreadsheets embedded in docx or PDF portfolios carry full data behind a picture.
• File names and email subjects often contain the very name you redacted.

Output: redaction log

## Redaction log: contract-2026-04.pdf -> contract-2026-04.redacted.pdf
- Method: PyMuPDF redaction annotations, images=PIXELS, metadata scrubbed, full rewrite
- Categories: person names (14), emails (6), IBAN (2), signature images (3)
- Pages affected: 1, 2, 5, 9-11
- Exceptions kept: company name, public registry numbers (per request)
- Verification: re-extracted text scanned (0 findings), term list grep (0), 1 %%EOF,
  metadata empty, visual check of pages 1, 5, 10
- Residual risk: page 7 is a scan; redacted by OCR boxes, verified visually only