The Model Table

This is the at-a-glance truth view used to pick a coordination panel. A model needs three things to make the panel: reliable on two axes — we can reach it (not throttled to death) and its replies are usable (they parse, not truncate) — comparable (within a few points of its peers), and diverse-lineage (a different tech tree, so errors decorrelate; rows are tinted by lineage so you can spot it — note that Gemini and Gemma share Google's lineage and are not independent for panel purposes). The columns let you see all of it at once — and where the gaps still are.

The only membership rule is $0 to call — not which gateway. The access path is noted in small print under each model name (OpenRouter, Google AI Studio, …) so the numbers are reproducible on the same endpoints — and where a model has several free endpoints, the one in bold is the path we benched and reliability-checked on (the most reliable); we add any model that is freely accessible by some path, not just via OpenRouter. The public benchmarks (MMLU-Pro, GPQA) are third-party scores; MMLU-Pro-Mini-196 is our own custom stratified benchmark, run identically across every model.

★ The top row is the bar, not a panel member. It shows the best public frontier score on each bench as of mid-2026 — MMLU-Pro ~89.5 (Claude Opus 4.5), GPQA ~94.3 (Gemini 3.1 Pro); different models lead each. Paid / public access only — it sits there purely to mark the ceiling the free panel is measured against, never as one of the free models.† Usable re-verified clean on the pinned harness this session — coverage = parseable ÷ answered, over the questions each model actually returned (gemma-31b 36, gemma-26b 78, gpt-oss-120b 77, gpt-oss-20b 75, nemotron-nano-30b 78, nemotron-nano-9b 41). nemotron-30b's MMLU-Pro-Mini-196 78.5 is accuracy on the 88% it answered (≈ its 78.3 public MMLU-Pro — full strength when it concludes); the 12% it repetition-collapses are excluded. Reach is the contact axis: qwen's ✗ 9/196 is exactly why it has no Usable number — unreachable, not unintelligent. On our MMLU-Pro-Mini-196, gemma-4-26b (82.1) edges the bigger gemma-4-31b (80.6) — the small 4B-active MoE is the strongest reliable free single, the best-single reference for a panel to beat. Rows are tinted by lineage. Public MMLU-Pro/GPQA are each model's own-report or most-cited value — eval methodologies vary across sources (0-shot · CoT · reasoning-mode), so the public columns are approximate; MMLU-Pro-Mini-196 is our one consistent measure, run identically across every model. Entries marked "capped / thin / no data / ⚠ low-%" hit a free-tier wall (daily caps, 429s, or 550B/675B timeouts) before a clean 196-run — the number shown is accuracy on the questions answered, with coverage flagged. Row order is smartest→dumbest by our MMLU-Pro-Mini-196 (accuracy-on-answered) where we have it, else by public MMLU-Pro mapped onto that scale (our bench runs ~3–4 pts below public-reported, so public-only and coverage-capped rows are approximate placements, not exact).

Table notes — why a flagged result can't be fairly compared

1. 1 · Truncation artifact. A verbose reasoner ran out of token budget before writing "ANSWER: X"; the strict parser then scored the cut-off answer as wrong. nemotron-super's real intelligence is ~82–87% (its clean MMLU-Pro-Mini-196 = 82.1); the 58.7 / 59.2 / 62.8 values are measurement failure, not capability. Fixed by strict-parse + re-ask + the reliability/intelligence split.
2. 2 · Rate-limit-invalidated. 429s mid-run left most answers empty → scored as wrong (gemma 33.2, nemotron-super 20.9, …). Not real accuracy — the source file is literally named ..._INVALID_ratelimited.
3. 3 · Small-N pilot. N=30 (domain-routing, 6 disciplines × 5) and N=32 (the "crossover" hard slice) — 95% CI ≈ ±10–13 pts, easy/hard-skewed subsets → directional only, not comparable to the stratified 196.
4. 4 · Sub-sample. Some fusion runs scored only the N=168 disagreement subset (nemotron-super 80.2 / gemma 78.6 / gpt-oss-120b 73.2) — a different, smaller sample than the full 196.
5. 5 · Run-to-run reproducibility variance. Same model + same frozen 196-Q sample, different run → different score (the ±7 swing: gemma 80.6 vs 73.0 vs 66.3; gpt-oss-120b 75.0 vs 74.0). Pre-provider-pinning; the new pinned harness is the fix.
6. 6 · Unverified public score. NVIDIA's nemotron-super/ultra MMLU-Pro/GPQA could not be confirmed against a primary source (tech-report PDF unparseable, build.nvidia digits unconfirmed) → treated as UNKNOWN, not guessed.

What the gaps say

The panel this picks

Preferred trio (when qwen is up): gemma-4-26b (82.6) · gpt-oss-120b (80.8) · qwen3-next (80.6) — three lineages within 2.0 points.
Backup trio — gemma-4-26b · gpt-oss-120b · nemotron-nano-30b, three lineages. Measured on MMLU-Pro-Mini-196: the majority vote (81.1%) loses to its best member gemma-26b (82.1%, lift −1.0) — the trio is dominated by gemma-26b, and voting a dominated panel hurts (§6.1). Yet the selection ceiling is +4.1 (the right answer sits in the panel 86.2% of the time): the diversity is real, plain voting just can't reach it — the case for synthesis over voting, measured on our own clean data.
Coordinator (for the orchestrator): nemotron-nano-30b — external, comparable, NOT smarter, a different tech tree.
Best-single reference to beat: gemma-4-31b (85.2) — reliable, but out-of-panel (shares Google's lineage with gemma-4-26b).

Where each model wins — and the best trio

Every model scored across all 14 MMLU-Pro disciplines (14 questions each). No model wins every discipline — different lineages own different domains, which is the whole precondition for coordination. The two ceiling columns show the best score reachable per discipline: ceiling trio = best of the three trio members (marked ★), ceiling all = best of all seven. Both tower over every single model — that gap is the coordination headroom.

Green intensity = accuracy on that discipline. gold outline = discipline winner. ★ marks the three trio members. ceiling trio = best of the three per discipline; ceiling all = best of all seven — the bar coordination chases.

Specialised models — not panel candidates

The other free models on OpenRouter — code, vision, and small edge models. Each is shown against the one benchmark that fits its specialty, with that benchmark's best-known score (any model, paid or free) and the gap — so you can see how far below the field's best each free specialist sits on its own turf. Kept out of the panel table above (a coordination panel wants general reasoning).

Best · any model = current SOTA on that benchmark by ANY model, paid or free (sourced 2026: MMLU-Pro 89.8 Gemini 3 · SWE-bench Verified 88.6 Claude Opus 4.8 · MMMU 86.0 Qwen3.6 · MTEB 74.3 Harrier · ASR-WER ~2%). Only the 2 router meta-models (openrouter/free, owl-alpha) are left out — they aren't real models. Everything else free is here.

1. 7 · Method-divergent. llama-3.2-3b's MMLU-Pro reports as 36.5 (HuggingFace community eval) vs ~20 (lower-shot) — shown as 36.5.
2. 11 · Lower is better. WER (word error rate) is an error metric, so lower = better — Whisper-large-v3 is the leading open ASR model (~2.7% on clean LibriSpeech, 8–12% on real-world audio), a hair off the best specialised systems.
3. 12 · Relative metric. Aya Expanse 32B is the leading open multilingual model — it beats Llama-3.1-70B (twice its size) on non-English evals; m-Arena-Hard is a head-to-head win-rate across 23 languages, not a vs-SOTA absolute.
4. 8 · Benchmark-variant. north-mini-code's 80.2% is SWE-bench Verified pass@10; the SOTA + the other coders are best-effort single-attempt — gap is approximate.
5. 9 · Vendor harness. the 88.6% SWE-bench Verified figure (Claude Opus 4.8) is a vendor agentic-harness number, not a neutral pass@1 — so the code-model gaps are config-approximate. (Fable 5 / Mythos 5 score higher ~95% but aren't openly accessible, so the best accessible model anchors the gap.)
6. 10 · Not truly free. the Lyria music models carry $0 token pricing but bill per generated clip (~$0.04–0.08), and audio quality has no standard benchmark.

The free-API landscape — every provider we tried

Every free LLM API we probed while assembling the panel, and what its free tier actually delivers in practice. The recurring lesson: "free" rarely means "usable at volume" — daily caps, IP blocks and silent free→paid churn make the access layer the real constraint (full account in /problems).

✓ usable for real work · ⚠ works but capped/limited · ✗ blocked for us · all free with no card (our hard rule); paid-only models noted but never used.