Time-Series Forecasting skill for Codex

Beat seasonal naive in a backtest, or do not ship. Codex frames the horizon and decision first, reshapes data into long format with explicit zero-versus-missing handling, and climbs a model ladder from seasonal naive through AutoETS and AutoARIMA to LightGBM lags. Every model is scored with rolling-origin backtests and a bundled metrics script that reports MASE, WAPE, bias and interval coverage.

AI & LLM Engineering
$time-series-forecasting
159 lines
3 files
~1,700 tokens
MIT
Updated Oct 28, 2026

Install

$ curl -fsSL codexskills.dev/install.sh \    | sh -s -- time-series-forecasting

Installs to ~/.agents/skills/time-series-forecasting/ for every project on this machine.

$ curl -fsSL codexskills.dev/cli.mjs -o codex-skills.mjs  node codex-skills.mjs install time-series-forecasting

The CLI also searches, updates and removes skills. Run it with no arguments to pick from a list. Needs Node 20.12+.

$ curl -fsSL codexskills.dev/install.sh \    | sh -s -- --project time-series-forecasting

Installs to .agents/skills/time-series-forecasting/ in the current repository. Commit it and everyone who runs Codex there gets it.

$ curl -fsSLO codexskills.dev/downloads/time-series-forecasting.zip  unzip -o time-series-forecasting.zip -d ~/.agents/skills/

The archive holds the skill folder with its generated agents/openai.yaml. Unzip into .agents/skills/ to scope it to a repo.

$ & ([scriptblock]::Create((irm codexskills.dev/install.ps1)))    time-series-forecasting

PowerShell 5.1 or 7. Installs to %USERPROFILE%\.agents\skills\. Add -Project for the current repo.

Download time-series-forecasting.zip

104-line POSIX script. No sudo, no telemetry. Read install.sh

Use it in Codex

Start a prompt with $time-series-forecasting, or pick it from /skills. You can also just describe the task: Codex reads the description below and loads the playbook when your request matches.

What Codex reads before loading it

Builds and backtests forecasts with Nixtla statsforecast (AutoETS, AutoARIMA, seasonal naive), mlforecast with LightGBM lags, and Prophet - long-format data prep, rolling-origin cross-validation, MASE, sMAPE and WAPE scoring against naive baselines, prediction intervals, hierarchical and intermittent demand, and exogenous regressors. Use when asked to forecast sales, demand, traffic or capacity, compare forecasting models, backtest a forecast, explain why a forecast is off, or produce prediction intervals.

Use it when

Sales, demand, traffic or capacity must be forecast for a fixed horizon
Several forecasting models need a fair comparison
A forecast needs prediction intervals for planning decisions
An existing forecast is biased or degrading

Use something else when

You are exploring trends, seasonality or anomalies without forecasting: use time-series-analysis
The data lives in a time-series database you need to query: use timescaledb

How to check the result

Rolling-origin backtests show the chosen model with MASE below the seasonal naive baseline across windows and interval coverage close to the nominal level.

The mistake it prevents

Validating with random K-fold splits or a single lucky cutoff, which lets the model peek at the future.

~/.agents/skills/time-series-forecasting/SKILL.md

--- name: time-series-forecasting description: Builds and backtests forecasts with Nixtla statsforecast (AutoETS, AutoARIMA, seasonal naive), mlforecast with LightGBM lags, and Prophet -… license: MIT ---

Done well means: every model is judged by rolling-origin backtests against a seasonal naive baseline on the horizon the business uses, the reported error is scale-free (MASE or WAPE), and the forecast ships with calibrated prediction intervals, not just a line.

Frame the problem first

Question	Why it matters
Horizon `h` and frequency (daily, weekly)	Backtest windows and lags depend on it
Decision the forecast drives (staffing, inventory, budget)	Picks the loss: over- vs under-forecast cost, quantiles
How many series, how long, how sparse	1-10 long series vs 10k short or intermittent ones need different tools
Known future inputs (price, promotions, holidays)	Only usable as regressors if known at forecast time
Data cutoff and latency	Last complete period; partial current week must be dropped

Read AGENTS.md, the data source and any existing forecast code. Check the series for gaps, duplicates and timezone issues ($time-series-analysis covers exploratory work).

Data format

Use long format: one row per series per timestamp.

import pandas as pd

df = (raw.rename(columns={"store_sku": "unique_id", "date": "ds", "units": "y"})
         [["unique_id", "ds", "y"]])
df["ds"] = pd.to_datetime(df["ds"])
# make every series regular: missing days are zero sales, not absent rows
full = (df.set_index("ds").groupby("unique_id")["y"]
          .resample("D").sum().reset_index())

Decide explicitly whether a missing period means zero (sales) or unknown (sensor); fill zeros only for the first.

Model ladder

Step	Model	Use when
0	`Naive`, `SeasonalNaive(season_length=7)`	Always: the baseline every model must beat
1	`AutoETS`, `AutoARIMA`, `AutoTheta` (statsforecast)	Few to thousands of regular series; fast and strong
2	`CrostonOptimized`, `ADIDA`, `IMAPA`	Intermittent demand (many zeros)
3	`mlforecast` with LightGBM + lags, date features, exogenous vars	Many related series, promotions/prices matter
4	Prophet	Strong yearly/weekly seasonality with holidays, analyst-friendly components; rarely the most accurate
5	Neural / foundation models (neuralforecast, TimeGPT-style APIs)	Only after 1-3 are tuned and the gain is backtested

Install with pip install statsforecast mlforecast lightgbm utilsforecast (network required; ask in a sandbox). Foundation-model APIs send data to a third party: confirm with the user before using one.

Backtest with statsforecast

from statsforecast import StatsForecast
from statsforecast.models import AutoETS, AutoARIMA, AutoTheta, SeasonalNaive

h = 28
sf = StatsForecast(
    models=[SeasonalNaive(season_length=7), AutoETS(season_length=7),
            AutoARIMA(season_length=7), AutoTheta(season_length=7)],
    freq="D", n_jobs=-1,
)
cv = sf.cross_validation(df=full, h=h, step_size=7, n_windows=6, level=[80, 95])
# columns: unique_id, ds, cutoff, y, SeasonalNaive, AutoETS, ..., AutoETS-lo-80, ...

Score per model with scale-free metrics (bundled script, stdlib only):

python3 scripts/backtest_metrics.py cv.csv --season 7 \
  --models SeasonalNaive,AutoETS,AutoARIMA,AutoTheta --train train.csv

It prints MAE, RMSE, WAPE, sMAPE, bias and (with --train) MASE per model, and the empirical coverage of any -lo-XX/-hi-XX interval columns. Export with cv.to_csv("cv.csv", index=False) and full.to_csv("train.csv", index=False).

Metrics: which and why

Metric	Use	Avoid when
MASE (vs seasonal naive in-sample)	Default model comparison across series; < 1 beats naive	Series with almost no history
WAPE = sum abs error / sum actuals	Business reporting on volumes	Series with total near zero
sMAPE	Comparisons with published benchmarks	Values near zero (unstable)
Bias = mean(forecast - actual)	Systematic over/under-forecast	-
Pinball loss / interval coverage	Quantile or interval forecasts	-

Never use MAPE on series containing zeros. Aggregate per-series errors weighted by volume when the business cares about volume.

ML approach with mlforecast

import lightgbm as lgb
from mlforecast import MLForecast
from mlforecast.lag_transforms import RollingMean

mlf = MLForecast(
    models={"lgbm": lgb.LGBMRegressor(n_estimators=500, learning_rate=0.05, verbose=-1)},
    freq="D",
    lags=[7, 14, 28],
    lag_transforms={7: [RollingMean(window_size=7)], 28: [RollingMean(window_size=28)]},
    date_features=["dayofweek", "month"],
)
cv_ml = mlf.cross_validation(df=full, h=28, n_windows=6, step_size=7)

Lags shorter than the horizon need recursive prediction; mlforecast handles it, but errors compound. Lags of at least h avoid that at some accuracy cost.
Exogenous columns in df must also be provided for the future via X_df at predict time; if a value is not known in advance (weather actuals), it cannot be a feature.

Prophet in brief

from prophet import Prophet
m = Prophet(interval_width=0.9, weekly_seasonality=True, yearly_seasonality=True)
m.add_country_holidays(country_name="US")
m.fit(series[["ds", "y"]])
fc = m.predict(m.make_future_dataframe(periods=28))

Backtest it with prophet.diagnostics.cross_validation(m, initial="730 days", period="28 days", horizon="28 days") and compare to the same naive baseline before adopting it.

Gotchas

Random K-fold on time series leaks the future. Only rolling-origin splits.
Partial last period (current week to date) looks like a crash; drop it.
Structural breaks (COVID, a pricing change, a new channel) make older data misleading; test shorter training windows or a break indicator.
Forecasting the sum of many series: reconcile hierarchy (hierarchicalforecast) rather than forecasting the total and each child independently.
Log transform for multiplicative growth must be inverted with bias care; forecast medians, not means, come back from exp.
Interval coverage well below nominal (80% interval covering 60%) means intervals are too narrow; use conformal intervals or widen before decisions rely on them.

Report template

**Series:** 1,240 store-SKU daily series, 2023-01-01 to 2026-09-30 | **Horizon:** 28 days
**Backtest:** 6 rolling windows, step 7 days
| Model | MASE | WAPE | Bias | 80% coverage |
|---|---|---|---|---|
| SeasonalNaive | 1.00 | 31.2% | +0.4 | 74% |
| AutoETS | 0.86 | 27.0% | -1.1 | 79% |
| LightGBM (mlforecast) | 0.79 | 24.6% | +0.6 | n/a |
**Chosen:** LightGBM point forecast + conformal 80/95% intervals
**Caveats:** 112 intermittent series use CrostonOptimized; promo calendar needed 4 weeks ahead

What the zip contains

Codex reads bundled files only when a step points to them. Every one is shown here, so you can review it before installing.

agents/openai.yamlgenerated from this page

Gives the Codex app a display name, a one-line description and a starter prompt for this skill.

interface:
  display_name: "Time-Series Forecasting"
  short_description: "Beat seasonal naive in a backtest, or do not ship"
  default_prompt: "$time-series-forecasting Forecast daily sales for each store and SKU for the next 4 weeks"

scripts/backtest_metrics.py113 lines, 4.3 KB

#!/usr/bin/env python3
"""Score rolling-origin forecast backtests (statsforecast/mlforecast cross_validation output).

Usage:
    python3 backtest_metrics.py cv.csv --models AutoETS,SeasonalNaive [--season 7] [--train train.csv]

cv.csv needs columns: unique_id, ds, y, and one column per model. Optional interval columns
named <model>-lo-<level> / <model>-hi-<level> are scored for empirical coverage.
train.csv (unique_id, ds, y) enables MASE, scaled by the in-sample seasonal naive error.
Exits 1 if a requested model column is missing or no rows are scored.
"""
import argparse
import csv
import math
import re
import sys
from collections import defaultdict


def num(v):
    try:
        x = float(v)
        return x if math.isfinite(x) else None
    except (TypeError, ValueError):
        return None


def seasonal_scales(path, season):
    series = defaultdict(list)
    with open(path, newline="", encoding="utf-8") as fh:
        for row in csv.DictReader(fh):
            y = num(row.get("y"))
            if y is not None:
                series[row["unique_id"]].append((row["ds"], y))
    scales = {}
    for uid, pts in series.items():
        ys = [y for _, y in sorted(pts)]
        diffs = [abs(ys[i] - ys[i - season]) for i in range(season, len(ys))]
        if diffs and sum(diffs) > 0:
            scales[uid] = sum(diffs) / len(diffs)
    return scales


def main():
    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("cv_csv")
    ap.add_argument("--models", required=True, help="comma-separated model column names")
    ap.add_argument("--season", type=int, default=1, help="seasonal period for MASE scaling")
    ap.add_argument("--train", help="training data CSV for MASE")
    args = ap.parse_args()
    models = [m.strip() for m in args.models.split(",") if m.strip()]

    with open(args.cv_csv, newline="", encoding="utf-8") as fh:
        reader = csv.DictReader(fh)
        cols = reader.fieldnames or []
        rows = list(reader)
    missing = [m for m in models + ["y", "unique_id"] if m not in cols]
    if missing:
        print(f"error: missing columns: {', '.join(missing)}", file=sys.stderr)
        return 1

    intervals = defaultdict(list)  # model -> [level]
    for c in cols:
        m = re.fullmatch(r"(.+)-lo-(\d+(?:\.\d+)?)", c)
        if m and m.group(1) in models and f"{m.group(1)}-hi-{m.group(2)}" in cols:
            intervals[m.group(1)].append(m.group(2))

    scales = seasonal_scales(args.train, args.season) if args.train else {}

    print(f"{'model':<20}{'n':>7}{'MAE':>10}{'RMSE':>10}{'WAPE%':>8}{'sMAPE%':>8}{'bias':>9}{'MASE':>7}  coverage")
    scored_any = False
    for model in models:
        n = 0
        abs_err = sq_err = err = abs_y = smape = 0.0
        smape_n = 0
        scaled = []
        cover = defaultdict(lambda: [0, 0])
        for r in rows:
            y, f = num(r["y"]), num(r[model])
            if y is None or f is None:
                continue
            n += 1
            e = f - y
            abs_err += abs(e)
            sq_err += e * e
            err += e
            abs_y += abs(y)
            if abs(y) + abs(f) > 0:
                smape += 2 * abs(e) / (abs(y) + abs(f))
                smape_n += 1
            s = scales.get(r["unique_id"])
            if s:
                scaled.append(abs(e) / s)
            for lvl in intervals.get(model, []):
                lo, hi = num(r[f"{model}-lo-{lvl}"]), num(r[f"{model}-hi-{lvl}"])
                if lo is not None and hi is not None:
                    cover[lvl][0] += int(lo <= y <= hi)
                    cover[lvl][1] += 1
        if n == 0:
            print(f"{model:<20}{0:>7}  no scorable rows")
            continue
        scored_any = True
        wape = 100 * abs_err / abs_y if abs_y else float("nan")
        smape_pct = 100 * smape / smape_n if smape_n else float("nan")
        mase = sum(scaled) / len(scaled) if scaled else float("nan")
        cov = " ".join(f"{lvl}%->{100 * c / t:.0f}%" for lvl, (c, t) in sorted(cover.items()) if t)
        print(f"{model:<20}{n:>7}{abs_err / n:>10.3f}{math.sqrt(sq_err / n):>10.3f}"
              f"{wape:>8.1f}{smape_pct:>8.1f}{err / n:>9.3f}{mase:>7.3f}  {cov}")
    return 0 if scored_any else 1


if __name__ == "__main__":
    sys.exit(main())

About this skill

Written and maintained by the Codex Skills editors to the same editorial standard as every skill here: a selective description, a real workflow, known pitfalls and a way to verify the work. First published Oct 28, 2026, last updated Oct 28, 2026. MIT licensed. Something wrong or out of date? Tell us.

Install

Use it in Codex

+Use it when

−Use something else when

How to check the result

The mistake it prevents

~/.agents/skills/time-series-forecasting/SKILL.md

Frame the problem first

Data format

Model ladder

Backtest with statsforecast

Metrics: which and why

ML approach with mlforecast

Prophet in brief

Gotchas

Report template

What the zip contains

About this skill

Related skills

Time Series Analysis

TimescaleDB

OpenAI Fine-Tuning

Financial Modeling

scikit-learn Pipelines

LLM Evals

Use it when

Use something else when