Programmatic SEO skill for Codex

Done well: every generated page answers a real query with data the visitor cannot get by swapping a word on a sibling page; pages without enough unique data are not indexed; and the rollout is staged so you learn from indexation before generating ten thousand more.

Is this a programmatic use case?

Proceed only if all are true:

• A repeating query pattern with demand exists ([tool] + [tool] integration, [service] in [city], [A] vs [B], [term] meaning, [currency] to [currency]).
• You own or can lawfully use a dataset that makes each page materially different (attributes, numbers, reviews, inventory, prices, steps, compatibility, examples).
• A template can answer the intent better than a hand-written page could at that scale.

If the only variable is the name in the H1, stop: that is the doorway pattern. Google’s spam policies (doorway abuse, scaled content abuse) target exactly this, regardless of whether pages are written by people, templates or models.

Workflow

- [ ] 1. Validate the pattern: demand, SERP format, who ranks
- [ ] 2. Audit the dataset: fields, coverage, freshness, rights
- [ ] 3. Define uniqueness and indexation thresholds
- [ ] 4. Design the template (sections fed by data, not filler)
- [ ] 5. URL scheme, hubs, internal links, sitemaps
- [ ] 6. Build with tests that enforce thresholds
- [ ] 7. Stage the rollout and measure
- [ ] 8. Prune and improve

1. Validate the pattern

Take 10–20 sample combinations across head, mid and long tail. For each, check the SERP (network needed): do template pages (marketplaces, directories, competitors) rank, or editorial guides? If editorial dominates, a template will struggle. Record volume per sample to estimate how much of the set has any demand at all; typically a small share of combinations carries most of it.

2. Audit the dataset

Profile it before designing anything:

import csv
from collections import Counter
rows = list(csv.DictReader(open("integrations.csv", encoding="utf-8")))
print(len(rows), "rows")
for field in rows[0]:
    filled = sum(1 for r in rows if (r[field] or "").strip())
    print(f"{field:30} {filled/len(rows):6.1%} filled, {len(set(r[field] for r in rows))} distinct")

Fields with low fill rates cannot carry uniqueness. Confirm licensing for scraped or third-party data and how often it refreshes.

3. Thresholds

Define, in code, what makes a page indexable. Example for integration pages:

Rule	Threshold	Below threshold
Distinct data fields populated	≥ 6 of 10	noindex, follow, excluded from sitemap
Unique text (not shared with siblings)	≥ 40% of body text	noindex until enriched
Real use cases / steps	≥ 3	noindex
Search demand or business value	demand > 0 or strategic	not generated at all

Measure uniqueness by comparing a page’s text shingles with its nearest siblings:

import re

def shingles(text: str, k: int = 5) -> set[str]:
    words = re.findall(r"\w+", text.lower())
    return {" ".join(words[i:i + k]) for i in range(max(len(words) - k + 1, 1))}

def unique_share(page: str, siblings: list[str]) -> float:
    mine = shingles(page)
    shared = set().union(*(shingles(s) for s in siblings)) if siblings else set()
    return len(mine - shared) / max(len(mine), 1)

4. Template design

Each section is either data-driven or omitted; never filler paragraphs with a swapped noun. Strong section types:

• Key facts table from structured fields (pricing, compatibility, limits, ratings).
• Computed insights: comparisons to category averages, rankings, trends.
• Steps or code specific to the combination (real API endpoints, real settings).
• First-party content: reviews, Q&A, usage stats, screenshots.
• Conditional copy: different sentences for different data states (free vs paid, supported vs not), written by a human once per state.
• Related entities: nearest siblings by real similarity, not random.

LLM-generated text per page is acceptable only when grounded in that page’s data, reviewed by sampling, and adding facts; ungrounded generated prose at scale is the scaled-content pattern.

5. URLs, hubs and links

• Stable, readable URLs: /integrations/slack/, /integrations/slack/jira/. Decide ordering for pairs (alphabetical or canonical order) and 301 the reverse order.
• Hub pages per dimension (/integrations/, /integrations/slack/) with crawlable links (paginated, not infinite scroll) to every indexable child.
• Each page links to its parent hub, siblings by similarity, and relevant editorial content. Every indexable page must be reachable within about 3–4 clicks.
• Sitemaps contain only pages that pass thresholds; split by section (see sitemaps-robots).

6. Build with enforced thresholds

Put the gate in the build so nobody can ship thin pages by accident:

// lib/pseo/gate.ts
export function indexable(p: IntegrationPage): { index: boolean; reasons: string[] } {
  const reasons: string[] = [];
  if (p.populatedFields < 6) reasons.push(`only ${p.populatedFields} fields`);
  if (p.useCases.length < 3) reasons.push("fewer than 3 use cases");
  if (p.uniqueShare < 0.4) reasons.push(`unique share ${p.uniqueShare.toFixed(2)}`);
  return { index: reasons.length === 0, reasons };
}

Feed index into robots metadata and sitemap inclusion, and write a test that fails the build if more than an agreed share of pages fall below threshold. Log reasons to a CSV for the content team.

7. Staged rollout

1. Launch 50–200 of the strongest pages (highest data completeness and demand).
2. Wait 3–6 weeks. In Search Console, track indexed vs submitted for that sitemap, impressions, and “Crawled - currently not indexed” counts.
3. If most are indexed and earning impressions, expand in batches; if not, improve the template before adding more pages. More pages never fix a weak template.

8. Prune and improve

Quarterly: pages with no impressions after 6 months and below-median data → enrich, merge into a hub, or noindex/remove (404/410, or 301 to the hub when there is a close match). Track the section’s indexed ratio as the health metric.

Gotchas

• Indexing everything on day one invites a sitewide quality reassessment; poor pages can drag down the section and the domain.
• Near-duplicate location pages (“Plumber in Springfield” × 500 with the same text) are doorway pages. Local pages need real local data (see local-seo).
• Pages generated for combinations that do not exist (an integration that is not built) mislead users; show “not supported” pages only if people search for that answer, and say so plainly.
• Faceted combinations explode. Generate only combinations with demand or data.
• Rendering thousands of pages at request time without caching can hammer the database when Googlebot crawls; prerender or cache with revalidation.

Output

**Pattern:** /integrations/[a]/[b] — 2,140 candidate pairs, 312 pass thresholds.
**Gate:** fields ≥ 6/10, use cases ≥ 3, unique share ≥ 0.40 (lib/pseo/gate.ts + test).
**Phase 1:** 150 pages live, sitemap-integrations-1.xml; hub /integrations/ links all.
**Measure at week 4:** indexed/submitted, impressions, crawled-not-indexed.
**Below threshold report:** pseo-gaps.csv (1,828 pairs with missing fields).