Text to Speech skill for Codex

Done well means: the text is normalized for speech before synthesis, pronunciation of names and terms is pinned, long content is chunked at sentence boundaries without audible seams, streaming starts playing within a second for interactive use, and listeners are told the voice is AI-generated.

Choose the provider

Need	Default
Natural voice with tone controlled in plain language	OpenAI gpt-4o-mini-tts with instructions
Precise control: pauses, phonemes, say-as dates/numbers	Azure, Google Cloud or Amazon Polly with SSML
Widest voice library, cloning, many languages	ElevenLabs
Offline or on-device	Open-weights TTS (Piper, Kokoro and similar) run locally; check each model’s license
Two-way spoken conversation	Not TTS alone: $openai-realtime-voice

Read AGENTS.md for the provider already in use and where API keys live. Every hosted API needs network; in a Codex sandbox ask before calling it, and estimate cost for batch jobs (characters x price per million) before starting them.

Prepare the text

TTS reads exactly what it is given. Normalize first:

- [ ] Expand abbreviations and symbols: "Dr." -> "Doctor", "&" -> "and", "%" -> "percent"
- [ ] Decide how numbers, dates, currencies, times and versions are spoken ("v2.3.1" -> "version two point three point one")
- [ ] Remove markdown, URLs, code blocks, emojis and footnote markers
- [ ] Spell out or phonetically respell brand names the voice gets wrong
- [ ] Split into sentences; keep paragraphs as natural pause points

Keep a pronunciation dictionary ({"Nginx": "engine x", "Kubernetes": "koo-ber-net-eez"}) in the repo and apply it before every request, so fixes persist across runs.

OpenAI TTS

from openai import OpenAI
client = OpenAI()

with client.audio.speech.with_streaming_response.create(
    model="gpt-4o-mini-tts",
    voice="coral",
    input="Your order has shipped and should arrive on Thursday.",
    instructions="Warm, upbeat customer-support tone. Medium pace. Clear pauses between sentences.",
    response_format="mp3",
) as resp:
    resp.stream_to_file("shipped.mp3")

• Voices include alloy, ash, ballad, coral, echo, fable, nova, onyx, sage, shimmer and verse (newer voices may be added; list the current set from the docs). Pick by listening to the same paragraph in 3-4 voices.
• instructions steers tone, pace, accent and emotion; it only works on the gpt-4o TTS models, not on tts-1 / tts-1-hd.
• Input limit is about 4,096 characters per request; chunk longer text.
• Formats: mp3 (default, broad support), opus (streaming, small), aac, flac, wav and pcm (raw 24 kHz 16-bit mono, lowest latency to play).
• OpenAI’s usage policies require telling end users the voice is AI-generated, not human.

SSML providers

<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis" xml:lang="en-US">
  <voice name="en-US-JennyNeural">
    Your appointment is on <say-as interpret-as="date" format="mdy">11/04/2026</say-as>
    at <say-as interpret-as="time" format="hms12">3:30pm</say-as>.
    <break time="400ms"/>
    Ask for <phoneme alphabet="ipa" ph="ˈʃɪvɔn">Siobhan</phoneme> at the front desk.
    <prosody rate="-10%">Please arrive ten minutes early.</prosody>
  </voice>
</speak>

• Element support differs per provider and per voice (neural voices ignore some prosody settings). Test each tag with the exact voice you will ship.
• Escape &, <, > in user text before inserting it into SSML, or synthesis fails.
• Azure also offers express-as styles via its mstts namespace; Polly neural voices support a subset of SSML; check the provider’s tag list.

Long-form audio

1. Chunk at sentence or paragraph boundaries, under the provider limit (about 3,000-4,000 characters).
2. Use the same voice, model, instructions and format for every chunk; include a short overlap of context only if the provider supports a previous-text parameter (ElevenLabs does).
3. Concatenate with a small natural pause and normalize loudness once at the end:

printf "file '%s'\n" chunk_*.wav > list.txt
ffmpeg -f concat -safe 0 -i list.txt -af "loudnorm=I=-16:TP=-1.5:LRA=11" -c:a libmp3lame -b:a 128k episode.mp3

-16 LUFS integrated suits podcasts and web audio; use the platform’s own target if it publishes one. Generate chunks as wav or pcm and encode once; concatenating mp3 files adds gaps and re-encoding loss.

Streaming for interactive apps

• Start synthesis per sentence as the LLM streams text: buffer tokens until ., ?, ! or a newline, then send that sentence to TTS while the next one generates.
• Use pcm or opus for the lowest time to first audio; play chunks as they arrive.
• Measure time to first audio byte and end-to-end delay from user message to first sound; under about 1 second feels responsive.
• Cancel in-flight TTS when the user interrupts, or stale audio keeps playing.

Caching and cost

• Cache by hash of (provider, model, voice, instructions, normalized text, format); static prompts like menu lines should synthesize once.
• Batch narration costs scale with characters: report the estimate and get approval before a large run.
• Store generated audio with its source text and settings so a fix re-renders only the changed chunk.

Voice cloning and consent

• Clone or design a voice resembling a real person only with that person’s documented consent; never imitate public figures or private individuals without it.
• Disclose AI-generated audio in the product (UI label, intro line, or metadata).
• Do not publish generated audio publicly without the user’s confirmation.

Gotchas

• Numbers read wrongly (“1984” as a year vs a quantity) unless normalized or marked with SSML say-as.
• Very short inputs (one word) can sound clipped or odd; pad with context or join short lines.
• Different chunks with slightly different instructions produce audible tone jumps.
• Browser autoplay policies block audio until a user gesture; start playback from a click.
• Sample-rate mismatches when concatenating (24 kHz vs 44.1 kHz) cause pitch changes; resample everything to one rate before joining.

Verification

• Listen to the first, a middle and the last chunk of every batch, plus every line containing a dictionary term.
• Check duration is plausible (about 150 words per minute for narration).
• ffprobe the output for codec, sample rate and duration; measure loudness with ffmpeg -i out.mp3 -af ebur128 -f null -.
• For streaming, log time to first audio byte over 20 requests and report p50/p95.

Report template

**Content:** 14 lessons, 61k characters | **Provider:** OpenAI gpt-4o-mini-tts, voice "sage"
**Instructions:** "Calm instructor, moderate pace" (identical for all chunks)
**Pronunciation dictionary:** 23 entries, all spot-checked
**Output:** 14 mp3 files, 128 kbps, -16 LUFS, total 6h12m; est. cost confirmed before run
**Disclosure:** "This narration is AI-generated" added to each lesson intro