Recommended models

Quick start

# Use the top-ranked model for a specific topic
althing panel run \
  --personas examples/personas.yaml \
  --instrument pricing-discovery \
  --best-model-for "Economy & Work"

# Top-ranked model across a whole dataset (by SPS)
althing panel run ... --best-model-for ":globalopinionqa"

# Topic within a non-default dataset
althing panel run ... --best-model-for "Technology & Digital Life:globalopinionqa"

Before the run, Althing prints a recommendation line to stderr so you can cancel and override:

synthbench: best model for globalopinionqa/Economy & Work → claude-haiku-4-5-20251001 · SPS 0.850 · JSD 0.091 · n=100 · $0.032/100q · cached 0h ago · source=live

How it works

1. 1 On first use, Althing fetches https://synthbench.org/data/leaderboard.json and caches it at ~/.althing/synthbench-cache.json for 24 hours.
2. 2 Entries are filtered to the requested dataset (default globalopinionqa), then ranked — by the named topic's score when a topic is given, otherwise by overall SPS.
3. 3 The top entry's model field is resolved through Althing's alias table (so "haiku" becomes claude-haiku-4-5-20251001) and stamped onto --model for the rest of the pipeline.

Environment knobs

Graceful offline behaviour

• Stale cache + network error → stderr warning, use stale cache.
• No cache + network error → stderr "synthbench unavailable", fall through to whatever --model or default was already in effect.
• Empty entries after filter → same fall-through.

No recommendation is ever fatal. --best-model-for is advisory: a bad network day won't take the panel down.

Use-case → top-ranked model

Regenerated from the live leaderboard.json (generated_at 2026-07-13). Picks are the top-ranked runnable single model per filter (the 3-model ensemble leads several rows but isn't a plain --model value). The live data updates continuously — consult the CLI flag or synthbench.org for current picks. Several topics are thinly sampled (small n); treat those as suggestive.

Caveats

• Display labels, ensembles & product configs (gh-519). Some leaderboard entries are Althing product configs (framework=product, is_ensemble=true) or carry a human-readable display label (e.g. Althing (Gemini Flash Lite)) in their model field. Althing never stamps such a label onto --model. Instead it substitutes a runnable id, preferring (1) the row's runnable model_id published by SynthBench (e.g. google/gemini-2.5-flash-lite, joined with provider_id when model_id is a bare slug), then (2) for product/ensemble rows without a model_id, a base model inferred from the entry's config_id (adopted only when it resolves to a recognized provider id or alias). If neither yields a runnable id the recommendation is refused with an actionable stderr message and your existing --model/default is kept. A stderr note records any substitution.
• Sparse topics. When the top entry's run_count < 3, a low-confidence warning is emitted. Treat those recommendations as suggestive rather than authoritative.
• Provider/model strings vary. The leaderboard publishes the raw model string the run used — sometimes a canonical id, sometimes a short alias. Althing passes the string through the alias resolver so either shape works, but the raw value is preserved in the recommendation line as raw_model.

Scoping

--best-model-for picks a single model for the whole panel. It is mutually exclusive with --models (which splits the panel across multiple models) — mixing the two is rejected at parse time.

See also