Elixir OTP skill for Codex

A good OTP design uses processes only where there is state, concurrency or a failure boundary; everything else is plain functions. Every process is started under a supervisor, can crash and restart into a valid state, and cannot become a bottleneck that every request serialises through.

Before changing code

1. Read AGENTS.md, mix.exs (Elixir/OTP versions, deps), config/*.exs and the application module (lib/my_app/application.ex) to see the current supervision tree.
2. Run mix compile --warnings-as-errors and mix test to know the baseline.
3. Check whether Phoenix, Oban, Broadway or Nebulex are present; they already solve jobs, pipelines and caching and should not be rebuilt as GenServers.

Do you need a process?

Need	Use
Pure transformation, validation, formatting	plain module functions
Fire-and-forget or parallel work with results	Task.Supervisor.async_nolink/2 / Task.async_stream/3
Read-heavy shared data (config, lookup tables, caches)	ETS table (read_concurrency: true) owned by a process, or :persistent_term for rarely-changing data
Serialised access to a resource (rate-limited API, port, connection)	GenServer
One process per entity (game room, device session, upload)	DynamicSupervisor + Registry
Durable background jobs that survive restarts	Oban (Postgres-backed), not a GenServer
Periodic work	GenServer with Process.send_after/3, or Oban cron for at-least-once

A GenServer used as a cache that every request calls is a single-process bottleneck; put the data in ETS and let callers read it directly.

GenServer template

defmodule MyApp.RateLimiter do
  use GenServer
  require Logger

  @refill_ms 1_000

  # Client API: the only functions other modules call
  def start_link(opts) do
    name = Keyword.get(opts, :name, __MODULE__)
    GenServer.start_link(__MODULE__, opts, name: name)
  end

  def acquire(server \\ __MODULE__, timeout \\ 2_000) do
    GenServer.call(server, :acquire, timeout)
  end

  # Server callbacks
  @impl true
  def init(opts) do
    capacity = Keyword.fetch!(opts, :capacity)
    {:ok, %{capacity: capacity, tokens: capacity}, {:continue, :schedule}}
  end

  @impl true
  def handle_continue(:schedule, state) do
    Process.send_after(self(), :refill, @refill_ms)
    {:noreply, state}
  end

  @impl true
  def handle_call(:acquire, _from, %{tokens: 0} = state), do: {:reply, {:error, :rate_limited}, state}
  def handle_call(:acquire, _from, state), do: {:reply, :ok, %{state | tokens: state.tokens - 1}}

  @impl true
  def handle_info(:refill, state) do
    Process.send_after(self(), :refill, @refill_ms)
    {:noreply, %{state | tokens: state.capacity}}
  end

  def handle_info(msg, state) do
    Logger.warning("unexpected message #{inspect(msg)}")
    {:noreply, state}
  end
end

Rules this encodes:

• Keep init/1 fast. Slow setup (DB loads, connecting) goes in handle_continue/2, otherwise the whole supervisor start blocks and boot can time out.
• Client functions wrap call/cast; callers never send raw messages.
• Once you define any handle_info/2 clause, add a catch-all: an unmatched message raises FunctionClauseError and crashes the server.
• Never do slow I/O inside handle_call for a shared server; offload to a Task and reply later with GenServer.reply/2, returning {:noreply, state} meanwhile.

call vs cast

	call	cast
Backpressure	yes, caller waits	none; mailbox can grow without bound
Failure visibility	caller gets exit on timeout or crash	silent
Default timeout	5000 ms	n/a

Default to call. Use cast only when the sender truly must not wait and the rate is bounded. A timed-out call exits the caller; catching it is rarely right, and the server may still process the request, so make the operation idempotent.

Supervision trees

def start(_type, _args) do
  children = [
    MyApp.Repo,
    {Registry, keys: :unique, name: MyApp.SessionRegistry},
    {DynamicSupervisor, name: MyApp.SessionSupervisor, strategy: :one_for_one},
    {Task.Supervisor, name: MyApp.TaskSupervisor},
    {MyApp.RateLimiter, capacity: 50},
    MyAppWeb.Endpoint
  ]

  Supervisor.start_link(children, strategy: :one_for_one, name: MyApp.Supervisor)
end

Strategy	When
:one_for_one	children independent (default)
:rest_for_one	later children depend on earlier ones (e.g. a registry, then its users)
:one_for_all	children only valid together; restart all if one dies

• Order matters: children start top to bottom and stop in reverse. Start dependencies (Repo, Registry, PubSub) before their users.
• restart: :permanent (default) for servers, :transient for work that may finish normally, :temporary for processes never restarted. Set via use GenServer, restart: :transient or in the child spec.
• Restart intensity defaults to 3 restarts in 5 seconds, then the supervisor itself dies. Do not raise limits to hide a crash loop; fix the crash.
• Put unstable parts (external connections) in their own subtree so their crashes do not restart stable siblings.

Per-entity processes

def start_session(id) do
  spec = {MyApp.Session, id: id, name: {:via, Registry, {MyApp.SessionRegistry, id}}}
  case DynamicSupervisor.start_child(MyApp.SessionSupervisor, spec) do
    {:ok, pid} -> {:ok, pid}
    {:error, {:already_started, pid}} -> {:ok, pid}
    other -> other
  end
end

Registry is node-local. For clustered uniqueness you need :global, Horde or a different design; say so instead of assuming Registry covers it.

Let it crash, correctly

• Crash on programmer errors and impossible states; handle expected errors ({:error, :not_found}) as values.
• State that must survive a crash belongs outside the process (DB, ETS owned by a supervisor-level process, or recomputed in init).
• Use Process.flag(:trap_exit, true) only when the process must run terminate/2 cleanup on shutdown; set shutdown: in the child spec to give it time.
• Monitor (Process.monitor/1) rather than link when you only need to know a process died.

Testing

test "rejects when tokens are exhausted" do
  pid = start_supervised!({MyApp.RateLimiter, capacity: 1, name: nil})
  assert :ok = MyApp.RateLimiter.acquire(pid)
  assert {:error, :rate_limited} = MyApp.RateLimiter.acquire(pid)
end

• start_supervised!/1 ties the process to the test and cleans it up. Allow name: nil (or unique names) so async: true tests do not collide.
• Test the client API, not handle_call directly.
• For timing, send the message yourself (send(pid, :refill)) instead of sleeping.
• Use assert_receive with a timeout for async messages; never Process.sleep to wait.

Diagnosing a live system

:sys.get_state(MyApp.RateLimiter)                     # inspect state (debug only)
Process.info(pid, [:message_queue_len, :memory, :current_function])
Supervisor.which_children(MyApp.Supervisor)
:observer.start()                                     # GUI; needs wx, local only

A message_queue_len that keeps growing means producers outpace the server: switch to call, shard by key (PartitionSupervisor), move reads to ETS, or add a demand-driven pipeline (GenStage/Broadway). Attaching a remote shell to production is outward-facing; confirm with the user first and keep to read-only inspection.

Verification

• mix compile --warnings-as-errors, mix format --check-formatted, mix test.
• mix credo --strict and mix dialyzer if the project has them.
• Kill the new process in iex -S mix (Process.exit(pid, :kill)) and confirm the supervisor restarts it into a working state.
• Under load (a quick Task.async_stream loop), message_queue_len stays bounded.

Gotchas

• A named GenServer is a global singleton; two tests or two tenants sharing it will interfere.
• Large binaries and big maps in state are copied on every call reply; return only what the caller needs.
• Task.async links to the caller; a crash in the task kills the caller. Use Task.Supervisor.async_nolink when failure is acceptable.
• GenServer.call(self(), ...) from inside the same server deadlocks until timeout.
• ETS tables die with their owner process; create them in a long-lived process.