Full-Text Search skill for Codex

Gives the Codex app a display name, a one-line description and a starter prompt for this skill.

interface:
  display_name: "Full-Text Search"
  short_description: "Relevant results, an index that never drifts"
  default_prompt: "$full-text-search Add product search with typo tolerance and filters to this app"

# Engine specifics

Read the section for the engine in use. All HTTP examples need network access to the search
service; in a Codex sandbox with network off, write the code and ask before running it.
Replace hosts, keys and index names; load keys from env, never hard-code them.

## Meilisearch

Every write returns a task; poll it before assuming the change is live.

```bash
# Settings (applied asynchronously; changing filterable/sortable fields reindexes)
curl -s -X PATCH "$MEILI_URL/indexes/products_new/settings" \
  -H "Authorization: Bearer $MEILI_MASTER_KEY" -H 'Content-Type: application/json' \
  --data '{
    "searchableAttributes": ["title", "brand", "category_names", "description"],
    "filterableAttributes": ["tenant_id", "category_names", "price", "in_stock", "visibility"],
    "sortableAttributes": ["price", "created_at", "popularity"],
    "rankingRules": ["words", "typo", "proximity", "attribute", "sort", "exactness", "popularity:desc"],
    "synonyms": { "sneakers": ["trainers", "running shoes"], "tv": ["television"] },
    "typoTolerance": { "disableOnAttributes": ["sku"] },
    "displayedAttributes": ["id", "title", "price", "image_url", "slug"]
  }'

# Documents: upsert by primary key, in batches
curl -s -X POST "$MEILI_URL/indexes/products_new/documents?primaryKey=id" \
  -H "Authorization: Bearer $MEILI_MASTER_KEY" -H 'Content-Type: application/json' \
  --data-binary @batch-0001.json

curl -s "$MEILI_URL/tasks/<taskUid>" -H "Authorization: Bearer $MEILI_MASTER_KEY"

# Zero-downtime: swap contents of the live name and the rebuilt index
curl -s -X POST "$MEILI_URL/swap-indexes" \
  -H "Authorization: Bearer $MEILI_MASTER_KEY" -H 'Content-Type: application/json' \
  --data '[{ "indexes": ["products", "products_new"] }]'
```

Search request body: `{"q": "red runing shoes", "filter": "in_stock = true AND price < 100",
"sort": ["price:asc"], "facets": ["category_names"], "limit": 20}`.

Ranking: a custom rule such as `popularity:desc` placed last acts as a tie-breaker; placing it
before `words` lets popularity override text relevance, which is rarely what users want.

Multi-tenancy: issue tenant tokens. A tenant token is a JWT signed with a search API key, whose
payload contains `apiKeyUid`, an `exp`, and `searchRules` such as
`{"products": {"filter": "tenant_id = 42 AND visibility = 'public'"}}`. The official SDKs expose
a helper to generate it server-side; check the SDK version's README for the exact call. Give
tokens short expiries and mint them per session.

## Typesense

The schema is explicit; fields not in the schema are stored but not indexed unless `.*` auto
fields are configured.

```bash
curl -s -X POST "$TS_URL/collections" -H "X-TYPESENSE-API-KEY: $TS_ADMIN_KEY" \
  -H 'Content-Type: application/json' --data '{
    "name": "products_v2",
    "fields": [
      {"name": "title", "type": "string"},
      {"name": "brand", "type": "string", "facet": true},
      {"name": "category_names", "type": "string[]", "facet": true},
      {"name": "description", "type": "string", "optional": true},
      {"name": "sku", "type": "string"},
      {"name": "tenant_id", "type": "string", "facet": true},
      {"name": "price", "type": "float"},
      {"name": "in_stock", "type": "bool"},
      {"name": "popularity", "type": "int32"}
    ],
    "default_sorting_field": "popularity"
  }'

# Bulk upsert, JSON Lines body (one document per line)
curl -s -X POST "$TS_URL/collections/products_v2/documents/import?action=upsert" \
  -H "X-TYPESENSE-API-KEY: $TS_ADMIN_KEY" -H 'Content-Type: text/plain' \
  --data-binary @products.jsonl | grep -v '"success":true' | head   # print failures only

# Point the alias the app queries at the new collection
curl -s -X PUT "$TS_URL/aliases/products" -H "X-TYPESENSE-API-KEY: $TS_ADMIN_KEY" \
  -H 'Content-Type: application/json' --data '{"collection_name": "products_v2"}'
```

Search parameters: `q`, `query_by=title,brand,category_names,description`,
`query_by_weights=4,2,2,1`, `filter_by=in_stock:true && price:<100`,
`sort_by=_text_match:desc,popularity:desc`, `facet_by=brand,category_names`, `num_typos`,
`per_page`. Turn typos off for codes by searching SKUs in a separate request or field with
`num_typos=0`.

Multi-tenancy: create a search-only key once, then derive scoped search keys server-side with
an embedded `filter_by` (for example `tenant_id:=42`) and `expires_at`. The JS client offers
`client.keys().generateScopedSearchKey(searchOnlyKey, params)`. Embedded filters cannot be
removed by the client.

Memory: Typesense holds the index in RAM. Budget roughly 2-3x the raw size of indexed fields
and test with production-sized data.

## OpenSearch (and Elasticsearch)

Define mappings explicitly; never rely on dynamic mapping for searchable fields.

```json
PUT /products_v2
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0,
    "refresh_interval": "-1",
    "analysis": {
      "analyzer": {
        "folded_english": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": ["lowercase", "asciifolding", "english_stop", "english_stemmer"]
        }
      },
      "filter": {
        "english_stop": { "type": "stop", "stopwords": "_english_" },
        "english_stemmer": { "type": "stemmer", "language": "english" }
      }
    }
  },
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "title":       { "type": "text", "analyzer": "folded_english", "fields": { "raw": { "type": "keyword" } } },
      "description": { "type": "text", "analyzer": "folded_english" },
      "brand":       { "type": "keyword" },
      "sku":         { "type": "keyword" },
      "tenant_id":   { "type": "keyword" },
      "price":       { "type": "scaled_float", "scaling_factor": 100 },
      "in_stock":    { "type": "boolean" },
      "popularity":  { "type": "integer" },
      "created_at":  { "type": "date" }
    }
  }
}
```

Load with `_bulk` (NDJSON, 5-15 MB per request), then restore `refresh_interval` (for example
`"1s"`) and replicas before swapping:

```json
POST /_aliases
{ "actions": [
  { "remove": { "index": "products_v1", "alias": "products" } },
  { "add":    { "index": "products_v2", "alias": "products" } }
] }
```

Query shape: text relevance in `must`, hard constraints in `filter` (cached, not scored):

```json
GET /products/_search
{
  "query": {
    "function_score": {
      "query": {
        "bool": {
          "must": [{ "multi_match": { "query": "red runing shoes", "fields": ["title^3", "brand^2", "description"], "type": "best_fields", "fuzziness": "AUTO" } }],
          "filter": [{ "term": { "tenant_id": "42" } }, { "term": { "in_stock": true } }]
        }
      },
      "field_value_factor": { "field": "popularity", "modifier": "log1p", "missing": 0 },
      "boost_mode": "sum"
    }
  },
  "aggs": { "brands": { "terms": { "field": "brand", "size": 20 } } }
}
```

Use `_analyze` to see how text is tokenized and `"explain": true` on a search to see why a
document scored as it did. For mapping depth (nested vs object, multi-fields, reindex API),
use the elasticsearch-mappings skill.

## Postgres full-text search

```sql
ALTER TABLE products ADD COLUMN search tsvector
  GENERATED ALWAYS AS (
    setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
    setweight(to_tsvector('english', coalesce(brand, '')), 'B') ||
    setweight(to_tsvector('english', coalesce(description, '')), 'C')
  ) STORED;
CREATE INDEX CONCURRENTLY products_search_idx ON products USING GIN (search);

SELECT id, title, ts_rank_cd(search, q) AS rank
FROM products, websearch_to_tsquery('english', $1) AS q
WHERE tenant_id = $2 AND search @@ q
ORDER BY rank DESC, popularity DESC
LIMIT 20;
```

- `websearch_to_tsquery` accepts user input safely (quotes, `-exclude`, `or`).
- Adding a stored generated column rewrites the table: schedule it like any heavy migration
  (zero-downtime-migrations), or add a plain column filled by trigger and backfill in batches.
- Typo tolerance needs `pg_trgm`: `CREATE INDEX ... USING GIN (title gin_trgm_ops)` and
  `WHERE title % $1 ORDER BY similarity(title, $1) DESC` as a fallback when FTS returns nothing.
- Prefix autocomplete: `to_tsquery('english', 'runn:*')` built from sanitized last-word input.