Composite Intelligence: From One Capability Tier to the Next, by Coordination Alone

Abstract

1 · Introduction

The frontier of language-model capability is usually advanced by training larger models. We are interested in an orthogonal axis: given a fixed set of already-trained models — specifically the free, rate-limited, often-erroring models available to anyone — how much capability can be recovered purely through coordination, at near-zero compute and no training?

Two empirical facts motivate the question. First, capability is not uniformly distributed: a model that leads on one discipline routinely trails on another, because benchmarks are discipline-specific and so is competence. Second — and this is the constraint that makes the problem interesting rather than trivial — the cheapest models are unreliable: any given call may time out or be refused. A practical composite system must therefore tolerate members that are individually flaky, leaning on the fact that competence need only be measured once to be exploited thereafter.

Our contribution in this draft is conceptual and empirical. Conceptually (§3) we separate two regimes that the literature often blurs — selection, which can only ever realize the best member per domain, and synthesis, which can exceed it. Empirically (§4) we run the selection regime end-to-end on free models and report an honest, mixed result that sharply localizes where the difficulty lives. The synthesis regime (§5) is the work this paper builds toward.

2 · Related Work

Merging and evolutionary composition. Sakana AI's evolutionary model-merging recipes ^[2] showed that combining the weights of existing open models — without further training — can yield a model stronger than any parent, including cross-domain transfer. This is composition at the parameter level; we study composition at the inference level, which requires no weight access and so applies to closed, API-only free models.

Mixture-of-Agents and answer synthesis. The Mixture-of-Agents (MoA) architecture ^[1] layers proposer agents whose drafts are iteratively refined and aggregated, reporting that an ensemble of open models can surpass a single GPT-4-class model — evidence that synthesis crosses the tier boundary, not merely matches it. LLM-Blender ^[5] ranks and fuses candidate generations; multi-agent debate ^[6] improves factuality by having models critique each other; self-consistency ^[7] and "more agents" sampling ^[8] show that aggregating many samples raises accuracy beyond a single draw.

Routing and cost-aware selection. FrugalGPT ^[4] and RouterBench ^[3] formalize sending each query to a chosen model to optimize cost/quality. Routing is the selection regime in our taxonomy: it can capture cross-domain specialization but cannot exceed the chosen model on its own turf.

Our setting differs from all of the above in one practical respect we treat as first-class: the members are free and unreliable. This reframes the engineering goal from "minimize cost" to "coordinate maximum capability from intermittently-available zero-cost models," and makes a one-time competence benchmark (not live reliability) the load-bearing assumption.

2.1 · A working taxonomy of model generations

To make "tier jump" measurable we adopt a constructed taxonomy of frontier generations — not an official standard, but a positioning device. Each generation is the cohort of models that defined the state of the art during a given window, with its characteristic band on the benchmark we use; the table is the yardstick against which we later place the orchestrator.

* Approximate MMLU-Pro and GPQA-Diamond accuracy from published reports/leaderboards, pending verification; both benchmarks postdate G1–G2, so those figures are retrospective estimates. GPQA separates the top tiers more cleanly than MMLU-Pro, which is near-saturated from G4 on. "Era at the frontier" = the window in which that cohort was state of the art — not the models' release dates.

The open mid-tier models we coordinate (e.g. gpt-oss-120b, gemma-4-31b) sit in the G3 band. The thesis, stated against this yardstick: coordination should lift a G3 ensemble toward the G4 band — closing some fraction of a generational gap with no larger model and at zero marginal cost. Two honest constraints on that claim: (i) our reported accuracies are computed on small samples and are not directly comparable to the published full-benchmark figures above — placing our system on this scale honestly requires a standardized, stratified MMLU-Pro run (the proper test we build toward); (ii) as §6 argues, the achievable lift likely shrinks as the starting tier rises, so a G3→G4 jump is a genuine empirical question, not an assumption.

The yardstick has a second axis the model table hides: the benchmark has generations too, and a coordination lift can only surface where the benchmark still leaves the coordinated models headroom. The two ladders must be read together.

A saturated benchmark (BG1) hides any coordination dividend — the models have no room to improve — so testing frontier coordination needs a BG3 ruler. We test on BG2 (MMLU-Pro), the right ruler for G3 mid-tier models.

2.2 · Model selection: the free-model pool

Before coordinating models we must choose them — and the choice is itself a measurement problem, not a matter of taste. We enumerate the full text-capable set on the OpenRouter free tier (26 free models; 12 general-purpose after excluding audio, vision, code-only, sub-3B, and safety/router variants), score each on the public benchmark with the widest cross-model coverage — MMLU-Pro, the one general metric all but two of them publish — and band them. Three constraints then define an admissible panel:

• Reliable — ≥90% answer-coverage on our harness (the reliability flag, §2.3); a model that truncates cannot silently join a panel.
• Comparable — within a few MMLU-Pro points of one another, so no member dominates (the §4.2 crossover rule: a dominant member gets out-voted by a correlated sub-majority).
• Diverse-lineage — no two members from the same tech tree, so their errors decorrelate (the §4.4 coverage engine).

Free-tier general-purpose models, sorted by published MMLU-Pro (primary sources: official model cards / Artificial Analysis; full sourced table at /problems). GPQA shown only as a corroborating signal — the public "Diamond" label is inconsistent across cards. The Artificial Analysis Intelligence Index was rejected as a banding axis: it appears on two non-comparable calibrations within this cohort.

The constraints select a clean trio almost mechanically. A tight cluster of four different lineages within ~4 MMLU-Pro points (gemma-4-26b 82.6 · gpt-oss-120b 80.8 · qwen3-next 80.6 · nemotron-nano-30b 78.3) supplies more diversity than we need. Our panel is gemma-4-26b + gpt-oss-120b + qwen3-next — three independent tech trees within 2.0 points, two already reliability-proven. The strongest reliable free single, gemma-4-31b (85.2), is the natural best-single reference to beat — but it shares Google's lineage with gemma-4-26b, so it cannot sit inside the panel.

2.3 · Measurement protocol: earning a trustworthy number

A coordination lift is only as believable as the measurement beneath it — and our earliest lifts (+7.6, +2.5) were not believable: re-running the same model over the same 196 questions swung ±7 points. Proving that voting beats a strong member by a point or two demands a measurement whose noise is smaller than the effect. We therefore pre-register the protocol below; the bug-hunt that motivated each clause is documented at /problems.

• Control the controllable. Frozen question sample (fixed seed); temperature 0 with a token budget generous enough that <1% of answers truncate; one standardized prompt and one strict answer-extractor for every model; and — the largest single fix — the OpenRouter provider pinned (allow_fallbacks:false), so a model is always served by one backend at one quantisation instead of a silently-rotating mixture. A stable content-hash cache makes every run replayable.
• Split reliability from intelligence. Coverage (% answered in budget) and accuracy-among-answered are reported as separate numbers; a truncated reply is an abstention, never a silent wrong. This is what stops a verbose reasoner from being undercounted into a false low.
• Test the difference, paired. The headline is never a vote score from one run minus a single score from another — it is the per-question, within-run difference (each member's answer and the panel's vote on the same item, same run), aggregated by paired bootstrap / McNemar. When a run is "lucky," member and panel rise together, so run-to-run luck cancels in the difference — a ±7 swing in absolute scores can sit atop a stable paired lift.
• Quantify the residual, then clear it. With the controllable sources removed, the irreducible non-determinism (~±1–2 pts) is measured, not assumed: the reference model is run 30–50 times and we report the run-to-run standard deviation. The significance bar is raised to match — a lift counts only if its interval's lower bound exceeds not zero but the model's own upside swing (≈2.3–2.4 σ from the repeat-run study). The lift must beat the model's noise band, not merely beat nothing.

The honest consequence, stated up front: at n = 196 this protocol can confirm the large +8.8 selection ceiling but not a small (~+2–3) realized vote lift — that sits inside the noise band and needs either a larger sample (~800–1500 questions) or the paired cancellation to resolve. The rigor does not manufacture a result; it tells us precisely which of our results we are entitled to claim.

3 · Two Regimes: Selection vs. Synthesis

Let a query q belong to a domain d, and let each member model m have a true accuracy a(m,d). Two ceilings bound any composite system built from these members.

Selection ceiling = mean_d [ max_m a(m,d) ]  ≥  max_m [ mean_d a(m,d) ] = best single model

The selection ceiling — what a perfect router achieves — is the average over domains of the best member in that domain. Because no member is best everywhere, this strictly exceeds the best single model overall. This is the rigorous form of the intuition that "routing to specialists beats any one generalist." But selection is hard-capped: in a domain where the best member scores 80%, no routing decision can score higher.

The synthesis ceiling is higher. If members make different errors, an aggregation step — voting, ranking-and-fusion, or draft-critique-revise — can return a correct answer in cases where the per-domain best member was wrong, because a second model supplies the correction. This is the only regime that can exceed max_m a(m,d) within a domain — the genuine "level-up" past the members' own envelope. The remainder of this program is to measure how far above the selection ceiling synthesis can reach on free models.

3.1 · A second lens: what role the LLM plays in the coordination

The selection/synthesis split classifies a system by what it can achieve. A complementary lens classifies it by who does the coordinating — and it is the more honest axis for a stack of methods that all use LLMs. Every rung here calls LLMs to answer; even the baseline "single model" is one. So the ladder does not measure how much LLM is present — it measures what role an LLM plays in the coordination itself. By that test the rungs fall into three tiers, which bracket the roadmap cleanly:

• Tier 1 — Mechanical harness. Coordination is pure arithmetic: pick the highest-calibrated member (routing), or tally the members' votes, weighted or not. The LLMs only answer; nothing intelligent does the combining. (routing, plain vote, balanced vote, weighted vote.)
• Tier 2 — LLM as aggregator. An LLM now does the combining: it reads the members' full drafts, locates where one goes wrong, and writes the final answer. Intelligence decides the answer — but the process (which members are on the panel, how many rounds) is still fixed by the harness. (critique fusion, multi-layer MoA.)
• Tier 3 — LLM as orchestrator. An LLM controls the process: which members to consult, how many rounds to run, when to stop, how to combine. Intelligence decides the process, not merely the answer. (an off-the-shelf model prompted to orchestrate; or a model trained for nothing but orchestration.)

Across the three tiers the LLM is, in effect, promoted: from worker (it answers) → editor (it combines others' work) → manager (it decides who does what). Two consequences are worth stating. First, the two most-advanced bars — an off-the-shelf orchestrator and a purpose-trained orchestrator — share Tier 3: in both, the LLM is the coordinating intelligence; they differ only in whether that capability is prompted or learned, not in the role it occupies. Second, this lens aligns with but refines §3: Tier 1 is selection (hard-capped at the selection ceiling), Tiers 2–3 are synthesis (the only regimes that can exceed it) — the role-lens adds the orthogonal question of who does the synthesizing.

4 · Experiment 1 — Domain-Specialization Routing

Setup. Three free models of different lineage — gpt-oss-120b (OpenAI), gemma-4-31b (Google), and nemotron-3-ultra-550b (NVIDIA), all called via the OpenRouter free tier — answered held-out multiple-choice questions from six MMLU-Pro ^[9] domains (mathematics, computer science, law, biology, economics, history). Each model answered raw and single-shot (no tools, no agentic loop). A small calibration split (4 questions/domain) measured each model's per-domain accuracy; a mechanical router then sent each held-out test question (5/domain, 30 total) to the calibration-best model for that domain. We score each model alone, the routed "orchestra," and the oracle (perfect) router.

Per-domain test accuracy (%). "Best member" = the selection ceiling. Bold row = overall single-model accuracy; rightmost = oracle router.

The specialization signal is real. Gemma leads mathematics and law; gpt-oss leads computer science and history; they tie on biology and economics; nemotron leads nothing. Crucially, no single model is best across all six domains — the precondition for composition to help. The selection ceiling is 96.7%, a full +6.7 points over the best single model (gemma, 90.0%): a perfect router would convert "best generalist" into "best-of-specialists" and clear the bar.

Three findings carry forward. (i) Cross-domain specialization is a real, exploitable signal. (ii) The selection ceiling beats the best single model, validating the floor. (iii) Naive selection is fragile to calibration noise — a result that is itself useful, because it tells us the first engineering lever is calibration sample size, and the second is moving from selection to synthesis (which does not depend on a single brittle routing decision). We treat this experiment as the floor measurement; it may end up a footnote once the synthesis results land, but it establishes the premise honestly.

4.1 · Aggregation, without calibration, reverses the result

Routing forces a single brittle choice per domain. Aggregation does not: it asks every reachable member and combines their answers. On the identical 30 held-out questions and the identical cached answers — no new model calls, no calibration step — we computed three aggregators: a plain majority vote, an accuracy-weighted vote, and a vote of the two strongest members.

4.2 · The crossover: when coordination hurts

Is the aggregation dividend universal? To probe its boundary we re-ran the protocol on a harder slice of the same benchmark (questions drawn past the easy head) with a deliberately different panel: gpt-oss-120b and gpt-oss-20b (both OpenAI) plus gemma-4-31b — note that two of the three members share a lineage. The result reverses cleanly.

Taken with §4.1, the two runs bracket a crossover: aggregation beats the best single only when members make decorrelated errors and no single member dominates (§4.1's panel was three distinct lineages of comparable strength → +6.7); it falls below the best single when a correlated sub-majority can overrule a dominant member (this panel → −6.3). The design rule falls straight out of the data: diverse lineages, no dominant member. This is the empirical face of the decorrelation argument in §6 — and, we believe, a more useful result than a clean win would have been, because it specifies when the technique works rather than asserting that it always does.

On statistical power (read this before believing any single number). Both runs are small — N = 30 and N = 32 — and the 95% confidence interval on a single accuracy at this size spans roughly ±10–13 points. The individual lifts (+6.7, −6.3, −9.4) are therefore directional and mechanism-revealing, not statistically conclusive; their intervals overlap. We report them as pilots that establish the sign and the mechanism, not the magnitude. A clean test of the crossover (and of difficulty vs. panel-correlation, which are confounded across these two runs) requires the larger, standardized, single-variable runs flagged throughout. We state this plainly rather than over-claim either direction.

4.3 · The stratified result: no selection method beats the best member

To settle the question at full breadth on a corrected pipeline, we ran a stratified sample of 196 held-out questions — 14 from each of all 14 MMLU-Pro disciplines — through a diverse three-lineage panel (nemotron-3-super-120b, NVIDIA; gemma-4-31b, Google; gpt-oss-120b, OpenAI), scoring each model alone and every calibration-free aggregation rule, with 2,000-resample bootstrap intervals. The result is unambiguous — and for selection, it is negative. (This clean, full-size measurement supersedes the small, noisy pilots of §4.1–4.2, whose ±13-point intervals could not separate signal from zero.)

Every selection rule — routing, plain vote, accuracy-weighted vote, and even a leave-one-out optimally-weighted vote — lands at or below the best single member. This is precisely what §3 predicts: selection can only ever output an answer some member already gave, so when a strong member is present, picking among answers cannot beat it. Two details sharpen the point. First, the plain and accuracy-weighted votes are identical (−1.9): with one member (nemotron) clearly strongest, weighting by competence never flips the majority on a single one of the 196 questions. Second, the optimally-weighted vote — weights fit to maximize accuracy — reaches only −0.4, and its leakage-free optimum collapses to "trust nemotron." Tuned as hard as is mathematically possible, weighting still cannot clear the best member. On this panel, selection is the wrong lever.

4.4 · The selection ceiling: the diversity needed to win is provably there

That selection fails does not mean the panel lacks the knowledge to win — only that selection cannot capture it. We measure the selection ceiling: the fraction of questions on which at least one member is correct — the score an oracle selector that always picked the right member would reach, and the hard upper bound on every Tier-1 method.

The winning answer is present in this three-model panel 90.8% of the time — +8.8 points above the best single, and above a current frontier model (Opus 4.8 scores 86.7%, 170/196, under identical blind, single-prompt conditions). The diversity needed to reach frontier-class accuracy is genuinely in the panel; what is missing is a method that can reach it. The ceiling comes from complementary coverage: across the 196 questions each member is the sole correct one on some, so the union of their correct sets far exceeds any single set. Panel composition matters — a 3-lineage panel reaches a +8.8 ceiling, while 2-lineage panels reach +5.6 to +7.7. But we flag a confound plainly rather than over-read it: with only four models (two of them same-lineage), lineage diversity and member strength are entangled — the 2-lineage panels are also the ones carrying our weakest member (gpt-oss-20b) — so we cannot cleanly attribute the ceiling gap to lineage rather than strength; isolating it needs a strength-matched panel (future work). And, controlling for base rates, all model pairs' errors are positively correlated (≈3× what independence predicts) — members fail together on the hardest items. The engine is therefore complementary coverage on the medium-hard items, where different members succeed — not statistically independent errors. This ceiling is the prize; reaching above the best single is the work of §5.

Provenance note (2026-07-07): the 90.8% value is pinned — our number-pinning guard checks it against the committed run artifact on every edit — but the artifact’s stored member labels name gemma-4-26b as the panel’s best member, while this section describes the §4.3 trio (best member nemotron-super, alongside gemma-4-31b). The guard carries that labelling ambiguity as an open WARN until the ceiling is re-derived with unambiguous panel metadata; the value is not in question, its trio label is.

Figure. Per-discipline accuracy of all seven benchmarked models, beside two ceilings — ceiling trio (best of the three trio members per discipline) and ceiling all (best of all seven). No model leads every discipline — different lineages own different domains — and both ceiling columns exceed every single model on almost every row. This is the complementary coverage that makes the achievable bar high: even a couple of models leading a couple of categories each materially raises it.

4.5 · A tighter, complementary trio — when selection finally clears the best member

New measurement (this revision). §4.3 delivered the discipline-loss verdict on the original looser panel: no mechanical selection method beat the best single member, and aggregating a panel dominated by one strong model actively hurt. §4.4's per-discipline map then named a better-composed panel — the three models whose errors most decorrelate while sitting within ~1 point of one another: glm-4.5 · gemma-4-26b · nemotron-super-120b (ceiling-trio 88.7%). We re-ran the entire selection tier on this trio — same frozen 196-question stratified sample, same provider-pinned harness, member answers served from the shared seed-0 cache — and the verdict flips. Against a best single member of 82.1% (gemma-4-26b, with nemotron-super tied), every selection method now lands above it: category routing +0.6, and plain, intelligence-weighted, and grid-optimized voting all +2.0 (84.2%) — where the original trio's same four methods every one scored below the best member (−0.5, −1.9, −1.9, −0.4). The selection ceiling sits at +7.7 (89.8% — the correct answer is present in the panel roughly nine times in ten), so most of the diversity is still uncaptured, but the sign reversal is the headline: mechanical selection clears the best member once the panel is composed for complementarity rather than for raw individual strength.

Two honest caveats. (i) The lift is small and not yet statistically significant — the 95% bootstrap CI on the vote lift is [−0.5, +5.1], including zero (vote tie-break rule, fixed a priori: a tied tally defers to the a-priori best member by published benchmark — the never-worse-floor convention of §8; under the alternative tie-to-strongest-measured rule the point estimate moves to 83.7/+1.6, so the tie rule is worth one question either way and neither variant is significant) — so this is a directional reversal, not a settled margin; a larger sample is the obvious next step. (ii) One member, glm-4.5-flash, is intermittently rate-limited on its free tier and was reachable on only 87% of items this run; the vote is taken over the answers available (graceful degradation, §8), so the coverage gap widens the CI rather than biasing the point estimate. The mechanism is exactly the one §4.3 named in the negative: voting helps when no single member dominates and the errors decorrelate — conditions the §4.4-selected trio satisfies and the original trio did not. A note on the metric, and why panel clustering is decisive. We score every lift against the best single member — deliberately the harshest bar, because the entire premise is to beat the smartest model available, not the panel average. That choice makes the intellect spread of the panel load-bearing: in a loose panel where one member is materially weaker, that member's errors drag the tally below the best member even while the ensemble still comfortably beats its own average — so lift-vs-best prints negative although coordination is genuinely adding value over the mean. The earlier trio sat in exactly that trap. Clustering the three members within ~1 point (this trio) is therefore not cosmetic: it is the precondition for visible lift in the selection tier specifically — the mechanical methods (routing, voting) have no way to recover from a weak member dragging the tally. Tiers 2–3 (synthesis, orchestration), where an LLM actively reasons over the drafts rather than counting them, can in principle lift even a loose panel; it is purely mechanical selection that requires the tight cluster to show value against the best member. (Measured against the panel average rather than the best member, even the looser trios showed positive lift — but average-beating is not the claim; tier-jumping past the strongest member is.) This is the cleanest free-coordination win in the paper that uses no model intelligence to combine — a purely mechanical tally — and it is the selection-tier counterpart to the synthesis-tier lift in §5.1. (Re-running the synthesis tier on this trio is in progress; with a strong judge it is confounded by the judge-quality law of §5.1, so the vote is reported here as the clean selection result.)

5 · Toward the Upper Envelope and Beyond

The next experiments target results above the 96.7% selection ceiling, where only synthesis can reach. The plan, in increasing order of ambition:

(a) Robust selection. Larger calibration sets + soft routing (probability-weighted rather than hard argmax) to recover the oracle's +6.7 reliably and remove the brittleness of §4. The principled form of this weighting — provably-optimal test-time ensemble weights, computed as a mixed-integer program — is given by Best-of-∞ ^[20], and is the upgrade path from the per-discipline accuracy heuristic we test next.

(b) Cross-model aggregation — validated in §4.1. Per-query, collect every available member's answer and aggregate (majority vote on objective items, ranking-and-fusion ^[5] on open ones). This is the robust path to the selection ceiling demonstrated above. The open question it leaves: can fusion push above the ceiling on items where the per-domain best member is itself wrong but a minority of members is right?

(c) Mixture-of-Agents synthesis. A layered draft → cross-critique → fuse pipeline ^[1] over the free pool, with the explicit, falsifiable target of exceeding the best member within a domain — the genuine level-up.

(d) The step further. Where this work aims past prior art is the free-and-unreliable regime: a coordinator that schedules a one-time competence benchmark, then composes whatever subset of intermittently-available models is reachable at query time, degrading gracefully as members drop out. The scientific claim we intend to test: a coordinated system of free models, none individually frontier and none reliably online, can be driven above the capability of the best single model any one of them represents — and the margin is a function of coordination design, not model scale.

5.1 · Localized-Critique Fusion — the first lift above the best member

First measurement (this revision). We ran it. A single capable judge (gpt-oss-120b) reads all three panel drafts on the 52 questions where they disagree and reasons to a final answer (on the 144 unanimous questions the agreed answer stands). The outcome is the first method in this paper to clear the best single member: 82.7% — +0.6 over the best single, +2.6 over the plain vote. On the 52 contested items the judge answers 28 correctly to the vote's 23 — genuinely reaching answers the vote misses, not echoing the majority. Three honest caveats: (i) the margin is small and not yet statistically significant (95% bootstrap CI on the lift [−3.6, +5.6], includes zero) — a directional first result, not a settled one; (ii) it captures only a sliver of the +8.8 headroom to the selection ceiling; (iii) the judge is a panel member, but the panel's weakest (75.0%), so any bias toward its own draft would lower the result — the lift survives that. Reusing a panel model as the judge is also a sensible practical choice (one fewer model to run), so this is a realistic, lower-overhead configuration rather than merely a caveat; a stronger or fully-independent judge is future work. The operative variable is judge quality: the same fusion with a weak judge (gpt-oss-20b, 69.9%) produced no lift, matching the vote exactly. The load-bearing point: where every selection method sat below the best member, the first synthesis step sits above it — exactly the regime boundary §3 draws. This also refines SLM-MUX's warning ^[19] that unfocused discussion fails on small models — a weak judge gains nothing, but a capable judge with a localized critique crosses the bound.

Upper-bound probe — a frontier judge (paper-only, deliberately off the roadmap chart). The fusion above is capped by a single variable: judge quality. To bound the best case, we replaced the free-tier judge with the strongest model on hand — Claude Opus 4.8 — and had it adjudicate the same 52 contested items, reading the identical three cached panel drafts and blind to the answer key (zero new calls on the panel side: the drafts are served from the cached bank). This is deliberately not a free-coordination result — the judge sits far above the panel and is a paid frontier model — so it lives here as a ceiling measurement and is excluded from the headline bar chart, which tracks only what free models reach together. There is also a deeper, structural reason it cannot count toward the headline claim: in a genuine frontier-coordination setting the model capable of adjudicating the panel does not yet exist — by definition you are coordinating the current frontier, so no superior next-generation model is available to referee it. Borrowing tomorrow's model to judge today's panel is an instructive upper bound, but counting it would misrepresent a win you could not actually have at the frontier. On the 52 contested items the frontier judge scores 63.5% (33/52), against the plain vote's 44.2% (23/52) and the best single member's 50.0%: a strong model reasoning over three drafts beats mechanically counting them by +19 points. Full-bench, the 144 unanimous taken free (134/144) plus the judge's 33/52 gives 85.2% (167/196) — a hair under the same model answering all 196 solo (86.7%, 170/196). Read on its own that says "a strong judge matches but cannot beat the soloist." A first draft of this section stopped there and blamed the gap on the locked unanimous block. The control below shows that diagnosis was wrong, and inverts the lesson.

The control that matters — do the drafts actually help the judge? The fusion number conflates two things: the strong model's own competence, and whatever the drafts add or subtract. To separate them we ran the missing control — the same model (a fresh Opus 4.8, uncontaminated) answering those identical 52 items cold: no drafts, no key. The result inverts the naive story. Cold, Opus scores 69.2% (36/52) — higher than the 63.5% it scored with the three drafts in front of it. The drafts did not help the judge; they cost it three questions net — flipping its answer on 13 of 52, seven from right-to-wrong against only four from wrong-to-right. Shown three weaker models' workings, a frontier reasoner talked itself out of correct answers more often than into them: the same "deliberation amplifies a wrong consensus" failure we logged for the free-tier MoA in §4.1, exactly what SLM-MUX ^[19] reports for small models — now seen to bite a strong one too. Three consequences follow, each stated against our own earlier framing. (i) This configuration is not fusion; it is a cascade. If the drafts subtract value, the right design is not "show the judge the drafts" but agreement-gated escalation: keep the cheap panel's answer where it is unanimous, escalate only the disagreements to the strong model answering solo. That cascade scores 134 + 36 = 170/196 = 86.7% — statistically level with Opus answering everything itself (170/196 on the committed solo run), while invoking the expensive model on only 27% of items and spending zero tokens on draft-reading. The honest result is an inference-cost one — match a strong model at roughly a third of its calls — not a capability gain, and not a Mixture-of-Agents win. (ii) The binding constraint was never "the judge can't beat its own ceiling." It is simpler: the drafts are a net distraction to a model already stronger than their authors. (iii) The genuinely pretty result is in the agreement block. On the 144 items where all three free models agreed, the panel scored 134; the same Opus, answering those 144 alone, scores 134–136 depending on run (a fresh full-196 pass gets 136/144 directly; subtracting its 36 cold-contested from the committed 170 total gives 134). Statistically level with the panel either way: when a diverse cheap panel reaches unanimity, that unanimity is a correctness signal on par with a frontier model — the part of this probe that genuinely supports the broader thesis. And the two judge results are not in tension: they bracket the judge-quality axis. Drafts help a judge that is capable but weaker than the panel's best — the free-tier gpt-oss-120b (75%) gained +0.6 by being shown stronger members' work — and hurt a judge at or above that best, for which the drafts are at-or-below-level noise. The benefit of handing a judge the committee's drafts accrues only when the judge is not already as strong as the committee's best member; at or above that line, you give the model the question, not the committee — a curve we map precisely just below. Caveat, as everywhere: the −3 draft penalty and the cascade-vs-solo margin both sit inside the bootstrap noise — indeed inside the frontier model's own run-to-run variance, which we measured directly: two cold Opus runs on the identical 52 contested items disagree on 19 of them (netting 34 vs 36 correct), so any single-question comparison against the frontier reference is below its own reproducibility floor. The defensible claim is not "drafts significantly hurt" but "drafts did not help, and a draft-free cascade is at least as accurate and far cheaper."

The judge-quality curve, mapped — it is non-monotonic. We have now run four judges across three lineages; ordered by their own solo accuracy, the fusion lift over the best member (nemotron, 82.1) traces a clear inverted-U: gpt-oss-20b (69.9%) → no lift (collapses to the vote); gpt-oss-120b (75.0%) → +0.6 (the only method to clear the best member); nemotron (82.1%, the best member judging itself) → −1.4 (back below it, level with the plain vote); Opus 4.8 (frontier, ~86%) → below its own solo (the probe above). The peak is in the middle, and the two tails fail for opposite reasons: too weak a judge cannot adjudicate and defaults to counting votes; the best member — or a frontier model above the whole panel — is dragged down by the two weaker drafts it must read. The lift lives only in a narrow band: capable enough to recognise a correct answer, yet weaker than the panel's best, so the stronger members' drafts are genuinely new information to it. The actionable rule, and the one practical takeaway of §5.1, is counter-intuitive: for draft-fusion, pick a mid-tier judge, not your strongest model. Honest bounds — the single positive (+0.6) is not yet significant, and this is four points on one mini-bench: a shape to test at scale, not a law. (The nemotron-as-judge row had 2 of 52 contested fuses unreachable under upstream rate-limiting, filled with the panel vote.)

The synthesis step in (c) follows the standard Mixture-of-Agents ^[1] shape: a member drafts, others critique the whole answer, a fuser merges. We study a sharper variant we call localized-critique fusion — and we are deliberately careful below about which part of it is, and is not, ours. Rather than judge a draft as a whole (a coarse, end-of-trajectory signal), a critic model locates the single step where the draft first goes wrong — a mis-stated lemma, a wrong unit, an unjustified leap — and emits a short, targeted correction at that point; the drafter then revises from there rather than regenerating blind. The intuition is the credit-assignment one: a global "this is wrong" tells a model nothing about where, whereas a pin-pointed hint does.

What we build on — and the honest boundary of our claim. Our system coordinates already-trained models at inference. None of the combining mechanism is ours, and we say so plainly. Scoring the specific wrong step is process supervision ^[15]; a model critiquing its own trajectory in language is Reflexion ^[16] / Self-Refine ^[17]; and a separate critic emitting step-level feedback that an actor model then revises has been done directly ^[21][23], including with an entire ensemble of critics ^[22]. So we do not claim the critique-and-revise loop as novel — a prior-art check (this draft, §5.1) surfaced direct precedents, and we cite them here rather than paper over them. What is open, and all we claim, is empirical and regime-specific: whether this localized form works at inference over a pool of free, rate-limited, intermittently-available models — a setting in which the existing critique work assumed reliable, often frontier, models. The systems framing — compose whatever subset is reachable at query time, degrade gracefully as members drop out (§5d) — is the part of the regime the established technique was never built for.

Why it can exceed the selection ceiling. Voting and routing are bounded by the best member's answer per item (§3): if every member is wrong, the composite is wrong. Localized-critique fusion is not so bounded — a correction supplied by one model at the exact failure point can carry a draft to an answer no member produced alone. That was the falsifiable claim, now measured (§5.1, above): a capable judge clears the best member by a small, not-yet-significant margin where selection cannot, and a weak judge does not — with the same bootstrap intervals and the same candour about negatives as everything else here.

The evidence that makes this a genuine test, not a foregone one. We flag the strongest result against us. SLM-MUX ^[19] reports that discussion-based composition — debate, Mixture-of-Agents, multi-agent verification — lifts frontier models by ~2 points but fails on small models, dropping accuracy by up to 5.5 points across all three methods and both benchmarks tested. Our voting result (§4) is untouched by this, because voting is selection, not discussion; but localized-critique fusion is a discussion-class method, so on that evidence it should be expected to fail in our regime. That is exactly why it is worth running. The open question is whether localizing the critique to a single flagged step — rather than open-ended multi-agent debate — recovers the dividend where unfocused discussion destroys it. A plausible cause of the small-model failure is that weak models make agreeable, low-information critics ^[6]; we therefore force the critic to be adversarial and to commit to one concrete step. Either outcome is a result worth having: a lift that beats the best member, or an honest negative that corroborates ^[19] and tells us discussion-class synthesis simply does not survive at this tier. The caution is broader than small models: a controlled ICML-2024 benchmark found that multi-agent debate does not reliably beat far simpler self-consistency and is unusually sensitive to hyper-parameters ^[24] — so any positive result here must clear that bar, not just beat a single model.

Scope of the claim — stated explicitly. Everything above is measured on multiple-choice MMLU-Pro: a decomposable task with an objective answer key, which is precisely why it is cleanly verifiable. We make no claim that the dividend transfers to open-ended generation, long-horizon agentic work, or tasks without a ground-truth checker — settings where verification is itself unreliable and the effect could differ in either direction. Establishing whether localized-critique fusion survives off multiple-choice is explicit future work (§9), not something this draft asserts.

Confirmation on the new trio (this revision). We re-ran the synthesis step on the tighter §4.5 trio (glm-4.5 · gemma-4-26b · nemotron-super), with nemotron-super itself as the reconciling judge — the only strong, reliably-reachable model left once the external free judges (qwen3-next, gpt-oss-120b) were rate-limited out. The result lands exactly where the judge-quality law predicts: synthesis scores 82.1% (accuracy-among-answered 85.2% ≈ the judge's own solo intelligence), a +0.0 lift over the best single member. Because the judge sits above the panel's best, it reverts to its own ability and the drafts add nothing — so the purely mechanical plain vote (+2.0) beats this strong-judge synthesis (+0.0) on the very same trio. It is the cleanest demonstration yet of this section's thesis: a free-coordination lift is carried by the panel's agreement, not by a strong judge's verdict. We then ran exactly the clean test the law calls for — an external judge weaker than the panel's best, the regime §5.1 shows is where drafts actually help: nemotron-nano-omni (a multimodal-reasoning model, 77.3 solo, a different training lineage from the panel). The drafts did help it — shown the three panel drafts it scores 82.7% on the questions it answers, well above its 77.3% solo, confirming the prediction that a sub-panel-strength judge gains from the committee. But the gain is not enough to win: its full-bench accuracy is 78.1% (it abstains on ~6% it cannot resolve, coverage 94.4%), trailing both the best single member (82.1%) and the plain vote (84.2%). So across the judge-strength spectrum — a strong panel-member judge that reverts to its own ability (+0.0), and a weak external judge lifted by the drafts but capped below the vote — no synthesis configuration beats the mechanical plain vote for this trio. Amendment (this revision), and it is a lesson from our own §8: that verdict partly scored the judge’s unreliability as wrongness — the strong-judge run abstained on ~4% of items and those nulls were counted as errors, the precise bug class the reliability/intelligence split exists to prevent. Re-scored with the degradation ladder the paper itself prescribes (judge abstains → fall back to the panel vote), the identical cached judge verdicts reach 85.2% (+3.1) — one point above the plain vote and the highest realized (non-oracle) coordinator we have measured on this trio (exploratory; lift CI [−0.5, +6.6], still spanning zero — see §11.1). So the honest statement is narrower than the original: a strong panel-member judge adds nothing on the items it answers (it reverts to its own ability), but a stack that locks unanimity, takes the judge on contested items, and falls back to the vote when the judge fails converts the judge’s intelligence into full-bench accuracy better than either the vote or the judge alone. Here, the agreement of a tight, diverse panel is a stronger correctness signal than any single free judge's verdict; the synthesis win remains the open frontier, contingent on a judge that is both reliable and at least panel-strength while staying genuinely independent.

This is not idiosyncratic to our setup — it instantiates documented properties of LLM-as-a-judge systems. The strong panel-member judge's failure to improve on its own draft is textbook self-preference bias ^[26] — judges favour outputs from their own family — which is exactly why a judge drawn from the panel is the wrong default and an independent judge is preferred. The broader judge-bias literature ^[25] (position, verbosity, self-preference) also motivates our deliberately bias-resistant protocol: we never have the judge compare full free-text responses — it returns a single structured ANSWER: X letter, which neutralises the position- and verbosity-bias that dominate pairwise LLM judging. And that a tight, diverse trio is what unlocks the selection lift in the first place echoes Mixture-of-Agents ^[27], whose own analysis attributes aggregation gains to proposer diversity rather than raw individual strength — the same principle behind the §4.4 trio selection. The open question our data leaves is sharp and literature-aligned: does a judge that is simultaneously independent, panel-strength, and free-tier-reliable exist? If it does, the MoA result predicts it should clear the vote — and that is the orchestration frontier §11 turns to.

5.2 · The full ladder, traced — two laws, and where the headroom actually sits

Filling the rungs §5.1 skipped (this revision). §5.1 measured one synthesis method — agreement-gated localized-critique fusion. We now trace the two rungs between it and the plain vote: a judge that selects among the three drafts, and a judge that re-solves the question from scratch having read them. Both run on the same G2 panel (best member 82.1, selection ceiling 90.8, headroom +8.7) and the identical 196-item stratified sample, the judge's answers cached. Both lose to the best member — and how each loses is the contribution.

Law one — a re-solving judge reverts to its own ability. Handed all three drafts and asked to reason to its own final answer on every item (no agreement gate), a single external judge lands within a point of its solo score, whatever that score is. Two judges chosen to bracket the panel: qwen3-next-80b (solo 75.0) returned 76.0; nemotron-super-120b (solo 82.1) returned 81.9 on the 96% it answered. The drafts move neither judge meaningfully. The mechanism is plain once named: a judge told to re-answer discards the panel's single most useful signal — where its members agree — and substitutes its own competence, so the composite collapses to the judge alone. §5.1's frontier-judge control saw this at the top of the range (the drafts a net distraction to Opus); we now see it holds across the entire range. Full re-solve ≈ judge-solo, weak judge or strong.

Law two — a selecting judge performs at chance among the drafts. The lighter ask — do not re-solve; just pick which member to trust — fares no better. qwen3-next scored 79.1; nemotron-super, after a labelling artifact was corrected, ~78. Both sit at the mean of the three members (≈79.2) — statistically indistinguishable from picking a draft at random. From the drafts alone, the judge cannot identify which member is right on the contested items; its "selection" carries no information. (Measurement-hygiene note, in the spirit of /problems: the first nemotron selection run scored an impossible 62.2 — below the weakest member. The fault was ours: we had labelled the drafts Expert A/B/C while the answer options are also A–J, and the literal reasoner returned the expert label, not the option letter, on 17% of items. Relabelling the experts 1/2/3 restored ~78 — a figure that, as of 2026-07-07, is not yet artifact-pinned: the committed run file still holds the invalid 62.2 pass, and a clean re-run is queued; our number-pinning guard carries it as an open WARN until then. A score beneath the single-model floor is a measurement bug until proven otherwise.)

Why §5.1's lift survives where these fail: the signal is agreement, not adjudication. The reconciliation is exact. The one method that clears the best member — §5.1's localized-critique fusion (+0.6) — leaves the 144 unanimous answers untouched and sends only the 52 disagreements to the judge; it wins on the consensus block, not on the judge's verdicts. A tiebreak composite confirms it directly: take the majority where one exists, escalate only the all-disagree items to the strong judge, and the result is 82.4 — level with the best member again. The decomposition shows why nothing here clears it by much. A majority exists on 87% of items and is correct 85.3% of the time — agreement is a powerful correctness signal. The all-disagree remainder is only 13% of items; the right answer sits among the drafts on 80.8% of them, but the judge recovers just 53.8%. Even a perfect judge on the split items would cap the tiebreak at ~84.7% — the splits are simply too few to move the aggregate far. The rest of the +8.7 ceiling does not live in the splits at all. It lives in the wrong-majority items — two members agreeing on a wrong answer, the lone dissenter right — which no vote, selection, or agreement-gated rule can reach. Capturing them means overruling a confident majority: the calibrated-override gate of §8, and the genuine open frontier.

A coda that raises the bar. Benchmarking judge candidates turned up a single free model — nemotron-3-ultra-550b — scoring 86.8% on the 96% it answers on a clean re-bench of the current sample (an earlier run read 90.9% but at only 84% coverage — inflated; an interim fill read 87.0 at 94%; 86.8 at 96.4% coverage is the current, artifact-pinned number — full-bench 83.7 with unanswered scored wrong). That lands just below the trio's selection ceiling (90.8), yet comfortably above the best single panel member (82.1). The caveat that remains: the 550B still times out on ~6% of free-tier calls. But the direction is unambiguous, and it relocates the most compelling version of the thesis. A single free model now sits near the ceiling we are coordinating a weaker trio to reach. This does not retire the question — the coordination headroom is real at the tier we measured, and §6's compression argument predicts this as members strengthen — but it sharpens it: the live question is no longer "can we lift a weak trio to where one strong free model already sits?" but "does coordination still pay among models in that model's weight class?" — the frontier-panel test of §11.3, now with a concrete, free, 90-tier member to build from. (Caveat throughout §5.2: a single mini-bench, 196 items; the −0.5 to −6 gaps and the +0.6 lift all sit within bootstrap noise — these are shapes and mechanisms, not yet laws at scale.)

5.3 · The whole ladder, measured on the tight trio — and the plain vote is the high-water mark

Every remaining rung, now run (this revision). §5.2 traced the ladder on the older G2 panel; we have since run each richer aggregator on the tight §4.5 trio itself (glm-4.5 82.7 · gemma-4-26b 82.1 · nemotron-super 81.6), the identical frozen 196-item stratified sample, member drafts served from the shared cache — so every number below is measured against this run's own best single member (glm, 82.7) and its own plain vote (84.2). The result is a clean, honest sweep with a single headline: no synthesis or orchestration method beats the plain vote on this trio. Mechanical majority is the high-water mark; everything more elaborate lands at or below it. We report the whole ladder because the shape of the losses is the finding — and because burying a negative would betray exactly the standard the rest of this paper holds.

Tier 2 — synthesis (an LLM reasons over the drafts). Three methods, all at or below the vote. (i) Mixture-of-agents — one shared-revision round (each member re-answers having seen the others' drafts) then a vote — scores 83.2, +0.5 over the run's best single (glm 82.7; paired-lift CI [−3.6, +4.6]) but below the trio's plain vote (84.2): the revision round moves a couple of items and nets almost nothing, echoing §4.1's finding that shared deliberation can amplify a wrong consensus as easily as correct it. (ii) Localized-critique fusion — an external fuser (gpt-oss-120b, deliberately weaker than the panel's best per §5.1's law) rewrites a single answer from all three drafts on every item — scores 81.1: above the fuser's own 75.0 solo (the drafts genuinely lift a sub-panel-strength judge, exactly as §5.1 predicts) yet −1.6 under the best member and −3.1 under the vote. Reconciled with §5.1's winning +0.6: that win was agreement-gated (only the ~52 contested items went to the judge; the consensus block was preserved) on a panel whose vote sat at only 80.1 — here the fusion is ungated, so it overwrites the very consensus block §5.2 shows carries the lift, and the trio's vote bar it must clear is 4 points higher. (iii) Unanimity-else-judge — pass the 146 unanimous items through untouched, send only the 50 contested to the judge — scores 82.1, −0.6 vs best and −2.1 vs the vote, and its coverage split is the cleanest single view of why the ladder caps here: the 146 unanimous items score 95.9%, the 50 contested only 42.0%, and the external gpt-oss-120b judge recovers those contested items worse than the vote does (42% vs the vote's ~50% on the same block). This is not in tension with §11.1's offline 85.2 for the same gate: that figure used the panel's own best member (nemotron-super) as adjudicator with a vote-fallback on abstention; swapping in a weaker, fully-external judge and taking its verdict directly costs the 3.1 points — a direct traverse of the §5.1 judge-quality curve, not a contradiction.

Tier 3 — orchestration (an LLM runs the panel). The most ambitious rung, and the most negative. A bounded LLM orchestrator reads each question and chooses per item among solo / vote / consult / debate / fuse. With a low-capability conductor (nemotron-nano-30b) it scores 79.6, −3.1 vs best (CI [−7.7, +2.0]) — below even its own run's plain vote — and its move mix (82 solo, 79 vote, 28 fuse, 5 consult, 2 debate) shows a manager that mostly delegates down the ladder yet still pays a coordination tax on the items it does touch. We then tested the natural fix — a smarter manager — and it was rejected: a panel-strength, diverse-lineage conductor (qwen3-next-80b) scored 74.5, −8.2 vs best (CI [−14.3, −2.0] — the ladder's first interval that excludes zero, i.e. a statistically significant loss). The competent manager intervened far more (92 fuse, 81 consult, 22 vote, 1 solo) and paid the tax on every intervention. The lesson is the ladder's, sharpened: discretion to adjudicate is a liability, not an asset, on a well-composed panel — agreement carries the lift and every act of adjudication risks overturning it.

Tier 1 revisited — reliability-weighted voting (DFPE-style). The one selection-tier rung we had not run: weight each member's vote by its per-category accuracy (leave-one-out, so a member's weight on an item never sees that item), the closest analogue to DFPE ^[30] (which lifted a spread 10-model panel 70.8→73.5 this way). Computed offline from the cached matrix across a full weight sweep, it tops out at 84.7 (+2.0 vs best, CI [−1.0, +5.6]) — a mere +0.5 over the plain vote, and only at an in-sample-tuned weight (so even that is optimistic). The mechanism is transparent and it is the whole point of a tight panel: with all three members within ~1 point, there is almost no reliability spread to exploit, so reliability-weighting collapses onto the plain vote. DFPE's larger gain came precisely from a wide panel where the weak members could be down-weighted hard — the opposite of the complementary trio §4.5 is built to be.

The synthesis, stated plainly. Across the whole ladder on a well-composed free trio — routing (+0.6), plain / weighted / reliability-weighted vote (+2.0), mixture-of-agents (+0.5), critique-fusion (−1.6), unanimity-else-judge (−0.6), and both LLM orchestrators (−3.1, −8.2) — the plain majority vote is the best aggregator, and every method that lets a model adjudicate rather than count lands at or below it. This is the §5.2 laws made complete: the lift is carried by the panel's agreement, and the unclaimed headroom lives entirely in the wrong-majority block — items where two members confidently agree on a wrong answer and the lone dissenter is right — which no vote, fusion, or agreement-gate can reach and only a calibrated override (§8) or a genuinely stronger, independent judge (§11.5) could. It is worth stating that this negative is field consensus, not an artifact of our harness: an independent literature check confirms the pattern directly — Self-MoA ^[28] finds that aggregating the single best model beats every mixed-panel MoA configuration (a weak member drags the tally down, our loose-panel losses exactly), and a controlled 2025 study ^[29] finds majority voting accounts for essentially all of multi-agent debate's gains, debate adding no expected correctness. Our contribution is to have measured the entire ladder at zero cost on a deliberately optimal free panel and located, precisely, where the remaining headroom is not.

6 · Does the Technique Generalize Across Tiers?

Our experiment sits at a single point on the capability ladder. Academic honesty demands asking whether the coordination dividend we measured would survive a move up or down that ladder — and there are principled reasons it might not. The mechanism behind both aggregation and fusion is complementary coverage: members must each be correct on different questions, so that a correct minority can outvote or correct a wrong majority. (A measured caveat we owe the reader: on our panel, controlling for base rates, model errors are positively correlated — every pair fails together on the hardest items ≈3× more than independence would predict — so the dividend does not come from statistically independent errors, but from complementary coverage on the medium-hard items, where different members succeed.) Two distinct failures threaten this as the tier changes.

Moving up (stronger members). As models approach the ceiling on a task, their residual errors concentrate on the same genuinely-hard items — error correlation rises and headroom shrinks, so the dividend compresses toward zero. A committee of near-perfect models agrees on the easy cases and shares blind spots on the hard ones; little is left for coordination to recover. We see this in microcosm already: in the domains where every member scored ~100%, the orchestra could add nothing. So the technique's marginal value likely decreases as the tier rises — though its floor is preserved (it does not do worse than the best member, modulo aggregation noise). A true generational jump from a very strong tier would require fusion that manufactures correctness neither member possessed — a strictly harder bar than the error-correction we rely on here.

Moving down (weaker members). Errors decorrelate further (more headroom), but a new failure appears: if members are individually so weak that the majority is wrong on most items, voting amplifies the wrong consensus instead of correcting it — the "three bad answers" problem. Reasoning-fusion can still help if at least one member is right and the others can recognise sound reasoning when shown it, but pure voting degrades sharply. So the technique is bounded below by a minimum member competence.

The honest synthesis. The coordination dividend is largest in the middle of the ladder — capable-but-imperfect models that are usually right yet make decorrelated errors — and it compresses at both ends: saturation above, incompetence below. A genuine tier jump is therefore most plausible exactly where members are good enough to usually be correct but diverse enough to cover each other's gaps. Whether the tier we tested sits in that sweet spot, and how wide the spot is, our single-point experiment cannot settle; it requires running the identical protocol across several tiers, which we flag as the central follow-up. We make no claim that the gain is tier-invariant — only that it is real at the tier we measured, and we have given the theoretical reasons it may shrink elsewhere. This revision adds a first second point on that ladder (§6.1) — and it already revises the theory: the variable that controls the dividend proves to be the panel's internal balance, not its height on the ladder.

6.1 · A first measurement at a second tier — and a confound the theory missed

We ran it (this revision). The protocol above coordinates a panel of mid-to-large free models — call it the G2 tier: nemotron-3-super-120b 82.1, gemma-4-31b 80.6, gpt-oss-120b 75.0 (best member 82.1, selection ceiling 90.8, headroom +8.7). To put a second point on the ladder we benched a lineage-matched smaller panel — the same three lineages one size tier down, the G3 tier: nemotron-3-nano-30b 73.5 (76.2 among the 96% it answers), gemma-4-26b-a4b 82.1, gpt-oss-20b 69.9 — on the identical 196-item stratified sample, computing its coordination from the cached answers. (Correction, this revision: an earlier draft of this section was computed before the answer bank was fully gap-filled and read the a4b at 83.3, the ceiling gap at +1.9 and the vote at 77.6; the figures here are recomputed from the completed, committed matrix — the argument survives, the magnitudes moved.) The result cuts against §6's naive prediction. Moving down the ladder, §6 argued, should decorrelate errors and widen headroom; instead the smaller panel's selection ceiling is 85.2 — only +3.1 over its best member, barely a third of the larger panel's +8.7. The downward move shrank the coordination headroom, it did not grow it.

The resolution is a variable the up/down framing hid: intra-panel balance. The G3 panel is not a balanced-weak committee; it is dominated by one member — gemma-4-26b-a4b at 82.1 towers over its peers (73.5, 69.9), a 12-point spread, where the G2 panel is comparatively level (82.1 / 80.6 / 75.0, a 7-point spread). A dominated panel has little to coordinate: the two weaker members rarely hold a correct answer the leader missed, so the ceiling barely clears it — and naive voting still hurts, the small panel's vote (80.6) sitting −1.5 below its best member — like the imbalanced G2 panel (−1.9) and unlike the balanced trio (+2.0) — because two weaker votes can outvote the one strong member. §6's "moving down → more headroom" holds only if the panel stays balanced as it descends; when the smaller tier is instead uneven, the controlling variable is the within-panel spread, not the absolute tier. The honest correction to §6: the coordination dividend tracks intra-panel balance and complementary coverage more than it tracks the tier's height on the ladder — a balanced panel has headroom at either tier, a dominated one has little at either.

A striking single-model aside. The standout of the smaller tier is gemma-4-26b-a4b — a 4-billion-active-parameter mixture-of-experts model that scores 82.1 at full coverage, matching the larger panel's 82.1 leader and beating its other two members (an earlier partial-coverage read of 83.3 had it strictly ahead; on the completed bank it ties). A strong small model plainly exists — it is the same model the tight §4.5 trio is built around — and the lesson of §6.1 is that surrounding it with weaker lineage-mates coordinates poorly, while the §4.5 panel shows that surrounding the very same model with peers coordinates well. Capability and coordinability are different properties — the best soloist does not make the best committee; the committee makes the soloist coordinable.

Honest bounds. This is one comparison, and it is confounded: the G3 panel differs from G2 on two axes at once — smaller and less balanced — so we cannot cleanly attribute the headroom collapse to balance alone rather than tier alone; the clean test fixes balance and varies only size (and vice-versa), which we flag as follow-up. The coverage caveat the earlier draft carried here is now resolved: the gap-fill completed the a4b to 100% coverage (nano-30b to 96%), and the completed-data ceiling landed at +3.1 — inside the +3–4 band the partial-data draft predicted, still far below +8.7. And it is a single mini-bench. What survives the caveats is directional but clear: a one-model-dominated panel coordinates far worse than a balanced one, and that, not the tier label, is what §6's theory should have centered. (Operational note, in the spirit of the rest of this paper: the smaller models were reachable only in the post-reset off-peak window — during peak hours every one returned upstream rate-limit errors to a single probe — a concrete instance of the churn §11.4 anticipates.)

7 · Beyond One Modality: Best-of-Breed, and Capabilities No Single Model Has

The experiments above, and almost all of the prior work we cite (§2), live inside a single modality — text — and coordinate models that are, broadly, generalists. A second axis of the same idea is, we argue, where it most clearly surpasses the monolithic alternative: modality and capability coverage. A single "multimodal" model must amortize one set of weights across text, vision, audio and every domain at once; it is precisely in the niche modes — specialist vision, audio, OCR, rare languages, narrow technical fields — that even the strongest monolithic models measurably soften, because breadth is bought at the cost of per-mode depth. An orchestrator removes that compromise. Because the coordinator only routes, each sub-task can be sent to the single strongest model for that mode — a dedicated vision model, a dedicated audio model, a code specialist — so the system fields the frontier of every modality simultaneously, which no one model can.

This yields two distinct effects, and they should be measured separately. (i) Within-modality specialist lift. A domain specialist routed onto its domain raises the composite's ceiling there; the gain is the familiar selection bound of §3, now realized with a model whose existence the generalist panel cannot match. (ii) Capability unlock. This is categorical, not marginal: a text-only ensemble scores at chance on a task whose inputs it cannot even read, while a coordinator that can call a model supporting that modality turns zero into working. Formally, the composite's reachable capability set is the union of its members' — strictly larger than any single member's. That union property is the precise, defensible statement of the intuition that orchestration "opens a door" a bigger single model cannot.

How we propose to prove it (free-tier, reproducible). Two clean tests, both runnable on the same zero-cost rail as our text experiments:

• Within-modality (code). On a code-heavy held-out slice, compare the best single generalist (e.g. gpt-oss-120b) against a panel that routes code items to a dedicated free code specialist (qwen/qwen3-coder, with cohere/north-mini-code / poolside/laguna as alternates). The prediction is asymmetric: the specialist lifts accuracy on code while not dominating off-domain, so routing-to-specialist beats any fixed single model across the mixed set. This isolates effect (i) with the modality held constant.
• Cross-modality (vision). On a small held-out visual-reasoning set, the text-only members receive no usable signal and score near chance; the coordinator routes image items to a free vision model (nvidia/nemotron-nano-12b-v2-vl, the omni nemotron-3-nano-omni, or vision-capable gemma-4) and composite accuracy steps from chance to the specialist's level. The reported metric is the capability delta itself — the cleanest demonstration that the architecture exceeds what any single text model, of any size, can do. This isolates effect (ii).

Honest framing. The free specialists available to us are not themselves state-of-the-art; what the two tests establish is the advantage of the architecture — that it can always field the best specialist that exists for a mode, and that its capability scales with whatever component is plugged in — not that our particular free parts are frontier-grade. The cross-modality result is, in one sense, definitional (a text model "cannot" see); but that is exactly the load-bearing point, because no amount of scaling a text-only model substitutes for routing to a model that has the capability — and the within-modality test shows the effect survives even when every member can nominally do the task, isolating the real, measurable niche-mode weakness of generalists. Together they extend the selection/synthesis taxonomy of §3 with a third axis — coverage — along which the composite is, by construction, never smaller than its strongest member and routinely larger.

8 · Reliability, Graceful Degradation, and the Never-Worse Floor

A system assembled from free models inherits a property the monolithic baseline does not have: its members are unreliable in two distinct senses, and conflating them corrupts both the measurement and the design. The first is reachability — whether a call returns at all (rate limits, throttling, provider 5xx). The second is answer-usability — whether a returned response can be parsed into a committed choice, which a small reasoner can fail by looping without ever concluding, exhausting its token budget before it commits. These are different axes, and a single "accuracy" number silently fuses them: a model that answers 88% of items at 78% accuracy-when-it-answers is mis-scored as ≈69% if its 12% of non-answers are counted as wrong. We therefore report a reliability/intelligence split — coverage (the fraction of items the model commits an answer to) reported separately from accuracy-among-answered (its competence when it does commit). The split is not bookkeeping: the two failures demand opposite responses — a member that has gone unreachable should be dropped, while one that is reachable but merely mis-formatting should be re-asked — and only a coordinator that distinguishes them can react correctly.

The degradation ladder, and the floor it installs. Reliability is a cascade, not a switch, and a well-built coordinator treats it as one. When a member fails to return a parseable answer the system first re-asks it under a stricter, format-forcing prompt (recovering the merely-mis-formatted case deterministically); if it still cannot conclude, that member abstains and the aggregate is taken among whoever answered (§4 already shows the vote degrades this way when an arbitrary member times out); and if coordination collapses entirely — too few members, or no usable consensus — the system falls back to querying the single strongest member directly. This last rung is the load-bearing one, because it installs a floor: since the terminal fallback is the best member, a coordinator that triggers it on detected failure can never, on those items, score below the best member alone. Properly built, graceful degradation is not merely "robustness" — it is a guarantee that the architecture's downside is bounded by its strongest component. (How far accuracy falls before that floor is reached — the dropout curve as members vanish, and the worst case of losing the dominant member — is the quantitative question we flag for measurement in §11.2.)

The confidently-wrong override — the real hazard. The floor protects against detected failure. It does nothing about the subtler and more dangerous case: the orchestrator coordinates successfully — every member answers, a consensus forms, the synthesis returns a confident result — and that result is worse than the strongest member would have produced alone. This is not hypothetical; it is the mechanism behind our negative selection results (§3, §10): a plain vote lets weaker members outvote a correct strong one, and handing a strong model its weaker peers' drafts can drag its answer down rather than up. Nothing looks broken, so no fallback fires. And it exposes a hard truth about the whole enterprise: one cannot guarantee "never worse than the best member" while ever overriding it — overriding the best member is precisely the act that risks being wrong, and every point of the +8.8 selection-ceiling upside (§10) is bought by sometimes overruling the soloist, a fraction of which overrules will be mistakes.

Never-worse in expectation, via a calibrated override gate. The achievable guarantee is therefore not never-worse on every item — that is incompatible with capturing any upside — but never-worse in expectation over many items. The instrument is a selective override gate: the coordinator deviates from the strongest member's answer only when a confidence signal on the coordination justifies it (strong cross-member agreement against the soloist, low internal disagreement in the synthesis, or a trust score calibrated against held-out outcomes), and defers to the soloist otherwise. The floor removes the detected-failure downside; the gate manages the confidently-wrong downside; together they make the composite ≥ best-single in expectation while still reaching toward the ceiling. A natural source for such a signal already exists in any coordination method that can measure its own internal disagreement — the degree to which members are mutually inconsistent or deadlocked is a ready proxy for "do not trust this coordination; defer to the best member." Designing and calibrating that gate is, we argue, not a detail of the architecture but the central piece of engineering that separates a coordinator that can beat the best member from one that reliably does; locating where the gate should sit is a direct objective of the frontier-panel experiments (§11.3).

First measurement (this revision) — the mechanical gate, built and honestly falsified. We built the simplest version of that gate and pre-registered its design and evaluation before looking at any per-item outcome (design and code committed first: OVERRIDE_GATE_DESIGN.md, prototype/override_gate.py; every number recomputed from the committed answer matrices, zero new model calls). The gate defers to the best-on-calibration-fold member and overrides only where the other two members agree against it and the calibration fold's measured odds favor the override (2-fold cross-validation, stratified by discipline, seeded). The pre-registered hypothesis — gate ≥ best-single on both panels — failed: on the tight trio the gate reached 83.7% (+1.5, retaining most of the vote's +2.0), but on the loose panel it scored 79.6% (−2.0, CI [−5.1, +1.0]) — below the best member it exists to protect, and below the plain vote it was meant to discipline. The two failure mechanisms are precisely localizable, and they are the actual contribution. First: the floor guarantee has an unpriced prerequisite — knowing who the best member is. On both panels the two 98-item calibration folds disagree about which member is best (the leader flips fold-to-fold on each panel): members 1–2 points apart are statistically unidentifiable from ~10² calibration items — resolving a ~1.5-point separation takes calibration data of order 10³. The condition that makes a panel coordinate well — near-equal members — is exactly the condition that makes its best member unidentifiable, so "defer to the best member" is easy to say and expensive to ground. Second: the disagreement cell is difficulty-selected. Where the empirical cell rule had data (≥8 calibration items) it decided correctly — refusing a cell running 1 gain to 4 losses; where the cell was too small, the analytic fallback (global member accuracies with a measured wrong-collision correction, κ_w ≈ 0.5–0.7) overfired, because conditioning on "the panel disagrees" selects disproportionately hard items on which every member's global accuracy is a large overestimate of its cell-conditional accuracy. A calibrated gate needs cell-conditional estimates, not global ones — and those need even more data. What survives is the prize, now measured: an oracle gate — perfect decisions on only the mechanically-reachable cells (2-vs-1 disagreements, splits, abstentions; the unanimous and best-in-majority blocks stay locked) — scores 87.2% on both panels (+5.1 and +5.6 over the respective best members, bootstrap CIs excluding zero — significant even at n=196, unlike every realized lift in this paper). The decision problem is worth solving and its value is quantified; what this experiment establishes is the price of solving it mechanically: roughly an order of magnitude more calibration data than a mini-bench provides, or per-item confidence signals (logprobs, verbalized calibration) that free-tier members do not expose. Until one of those is paid, the honest deployment default for a small-calibration panel is the §11.5(c) cascade — escalate disagreements to a stronger model — rather than a self-adjudicating gate.

9 · Limitations & Variables

The honest account of what this draft does not settle — the variables a sceptical reader should weigh before trusting the numbers. Each is a real caveat; none, we argue, overturns the central finding, but several bound how far it can yet be pushed.

Sample size and significance. Every result is on a frozen, stratified 196-question sample (14 disciplines × 14), not the full ~12,000-item MMLU-Pro — running the whole benchmark across every member and every coordination round on rate-limited free models is impractical (weeks of wall-clock, exhausted daily caps). We do not rescale a single number: we grade each model's actual answer on every sampled item and report bootstrap confidence intervals, so this is a measured estimate with error bars (the established efficient-benchmarking practice — tinyBenchmarks ^[12] recovers full-MMLU accuracy from ~100 curated items). The headline vote lift (+2.0) carries a 95% CI of [−0.5, +5.1] — it includes zero, so the win is directional, not yet statistically settled; a larger run is the obvious next step.

Free-tier coverage gaps. One panel member (glm-4.5) is intermittently rate-limited and answered ~92% of items; unreached calls are scored as wrong (graceful degradation, §8), which widens the intervals rather than biasing the point estimate. "Accuracy on the items answered" and "full-bench accuracy" therefore diverge for the throttled models — we treat the full-bench (every-question) figure as the deployable one, which is why gemma-4-26b (82.1%, 100% reached) is the best single member even though a throttled peer scores higher on the subset it manages to answer.

Lineage vs. member-strength entanglement. The selection ceiling rises with lineage diversity, but in our pool lineage and strength are confounded (the broader panels also carry the weakest member). We cannot cleanly attribute the ceiling gap to lineage rather than strength; isolating it needs a strength-matched panel — flagged as future work, not claimed.

Correlated errors on the hardest items. Controlling for base rates, every model pair's errors are positively correlated (≈3× what independence predicts) — members fail together on the genuinely hard questions. The engine is therefore complementary coverage on the medium-hard items, not statistically independent errors; this also bounds agreement-gated methods (§11.5), since a too-hard question can surface as a confident shared wrong answer that no agreement signal will flag.

Single benchmark, and a coarse metric. MMLU-Pro is a public benchmark (contamination risk, though held-out here), and single-shot multiple-choice is a coarse proxy that almost certainly under-states the dividend: on a fixed-option item the most synthesis can do is select an answer some member already holds, whereas on open-ended generation (essays, code, proofs) it can combine partial answers into a result better than any draft. Read our numbers as a conservative lower bound; we expect the effect to be larger on generative tasks.

The panel is hand-composed. The winning trio was chosen post-hoc for complementarity (§4.4 → §4.5). Whether that composition transfers to a fresh benchmark — rather than overfitting this sample — is not established here, and a held-out re-selection is exactly the test that would settle it.

Deliberately out of scope. Operational dynamics — member dropout, provider churn, live reliability over months, adversarial rate-limit environments — are a distinct program deferred to a successor paper (§11.2–§11.4). This draft measures whether the lever exists, not whether the dividend survives long-run production. The further synthesis and orchestration claims of §5/§11 that are not yet run at scale remain hypotheses, reported as they resolve with the same candor as the negative results.

10 · Conclusion

Composition of free, unreliable models is a real lever, with a clean theoretical structure: selection is bounded by the best member, and synthesis can exceed it. Our experiments make the structure concrete — and, on clean measurement, they relocate the lever. On a panel with a wide spread in member strength, routing — calibration-bound — lost, and so did every other selection rule: plain vote (−1.9), accuracy-weighted vote (−1.9), even an optimally-weighted vote (−0.4) all sat at or below the best single member, exactly as the selection bound predicts. But that verdict is conditional, not universal: re-run on an evenly-matched, complementary trio (§4.5), the very same vote rules clear the best member (+2.0 — small, and not yet statistically significant), because mechanical selection wins precisely when no member dominates and the errors decorrelate. The decisive, robust finding above either is the selection ceiling: the winning answer is present in the panel +8.8 points above the best single — above a frontier reference, so the diversity needed to win is genuinely there. The lever that reaches it is synthesis: a training-free critique-fusion judge is the first method to clear the best member (+0.6 over best, +2.6 over the vote) — crossing where selection structurally cannot, though the margin is small and not yet statistically significant. Whether the orchestra beats the soloist by voting thus turns on how the panel is composed — a loose panel needs one member handed the editor's pen; an even-matched one is carried past the bar by the vote alone, with a strong editor then adding nothing over it (§5.1). Tracing the full ladder (§5.2) sharpens why the simple methods fail: a judge that re-solves the question reverts to its own ability, and one that merely selects among the drafts performs at chance — so it is the panel's agreement, not the judge's adjudication, that carries the lift, and the unclaimed headroom sits specifically in the wrong-majority items a calibrated override must reach. Capturing the remaining headroom up to the +8.8 ceiling — richer synthesis, multi-agent debate, a true orchestrator — is the open frontier. The destination is unchanged: a coordinated system whose capability is set by how its members are orchestrated, not by how large any one of them is.

11 · Future Work

This paper establishes a proof of concept with a clean, mixed result: selection coordination of free, unreliable models does not beat the strongest member, but the panel's selection ceiling sits +8.8 above it (above a frontier reference), and a first synthesis step — critique fusion — is the only method to clear the best member, establishing synthesis (not selection) as the lever. It does not address what happens when such a system is operated — over months, under churn, against an adversarial environment of rate limits and provider outages. We separate the remaining work into two tracks. The first — ascending the orchestration ladder — is a direct continuation of this paper's method and is sketched in §5–§7. The second — the robustness and dynamics of a live coordinated panel — is a distinct research program that we believe is best carried by a dedicated successor paper, both to keep the present contribution focused and because its central question is operational rather than existential: not does coordination help? (answered here) but does the dividend survive contact with a changing, unreliable real world?

11.1 · Ascending the orchestration ladder

Each rung of the orchestration stack remains to be measured at full size — the dashed bars of the roadmap. In increasing order of ambition: per-discipline weighted voting (each member's vote scaled by its leave-one-out competence in the question's discipline — the rung currently under measurement); localized-critique fusion and full Mixture-of-Agents synthesis (§5.1), the first regimes that can in principle exceed the selection ceiling; an LLM acting as the orchestrator itself — a general model that reads the query, decides which members to consult, and composes their outputs in-context rather than by a fixed aggregation rule; and, at the top, a model trained for orchestration alone, whose only competence is routing-and-composition over an arbitrary panel. Each becomes a solid bar the day its real, interval-bearing result lands; until then we report only the floor. This revision (§5.2) lands three of these rungs at mini-bench scale — judge-selection, re-solve synthesis, and the agreement-gated tiebreak — each confirming that the panel's consensus block, not the judge's adjudication, carries the lift; and §5.3 has since landed the remaining ones — mixture-of-agents, ungated critique-fusion, reliability-weighted voting, and the LLM-as-orchestrator itself (in two conductor strengths) — every one at or below the plain vote, so the rungs that remain genuinely open narrow to the trained orchestrator and the calibrated override gate (§8) that the wrong-majority headroom demands.

Stacking, not a single rung. These methods are not mutually exclusive, and the largest lift likely comes from composing them rather than picking one: agreement-gating to keep the consensus block untouched, synthesis applied only to the contested remainder, and a calibrated override reserved for the wrong-majority items where the real headroom sits. Two forms of the stack are worth distinguishing — a hand-built pipeline with a fixed order of techniques, and an LLM orchestrator that chooses which techniques to apply per query, the more ambitious self-composing form. Measuring stacked pipelines against the single-method baselines reported here is the immediate next experiment this work points to. First stacking measurements (this revision). We computed the full family offline from the committed answer matrices and the two cached judge runs (zero new calls; prototype/coordinator_zoo.py), on the tight §4.5 trio, best member 82.1, plain vote 84.2: unanimity-else-judge and judge-as-4th-voter (strong panel-member judge, vote fallback on judge-abstain) both reach 85.2 (+3.1, CI [−0.5, +6.6]); majority-else-judge 84.7; judge-confirmed override 84.2 (exactly the vote — on a panel whose 2-vs-1 cell already runs 3:1 in the vote’s favor, a confirmation signal has nothing to veto); and the per-discipline LOO-weighted vote — the rung previously flagged as under measurement — lands at 83.7, below the plain vote: at 14 items per discipline the specialization signal is noise, an honest null for that rung. The same stacks under the weak external judge add nothing (82.1–84.2) — the judge-quality law of §5.1 again. Every figure here is exploratory in the strict sense: a 13-system sweep on one frozen n=196 bench, so the best number is inflated by selection across the sweep and no CI clears zero; the pre-registered confirmation set is the n≈980 extension of the same stratified sample, on which the winning stack rides as the single confirmatory hypothesis.

11.2 · Robustness to member dropout — and to losing the strongest member

The free-and-unreliable regime guarantees that members vanish mid-session: rate limits exhaust, providers throttle, endpoints 500. §4.1 already showed the vote degrades gracefully when an arbitrary member times out — the aggregate is simply taken among whoever answered. The sharper, unquantified case is the loss of the strongest member, which is also the most consequential one in practice. A production panel that has come to rely on its best model must answer: how far does composite accuracy fall when that model is temporarily unavailable, and does the surviving panel still clear the best remaining member? We propose to characterize accuracy as a function of which and how many members are offline — a dropout-curve — with explicit attention to the worst case in which the dominant member disappears for the duration of a query batch. A coordinator whose value collapses the moment its best member blinks offline is not robust; one whose degradation is shallow and bounded is the real deliverable of this line.

11.3 · Frontier models as the coordinated members

Every experiment here coordinates free, mid-tier models. A natural and important extension is to run the identical protocol with frontier models as the panel members. The theory of §6 predicts the dividend should compress as members approach the ceiling — their residual errors correlate on the genuinely-hard items and headroom shrinks — but this is a prediction, not a measurement. Whether a balanced panel of frontier models yields any synthesis lift at all, and how it compares to the mid-tier result reported here, is the cleanest test of whether coordination is a scale-invariant lever or a phenomenon of the imperfect middle of the capability ladder.

11.4 · Dynamic panel composition over time

A real deployment is never a fixed roster. New, stronger models ship and should be adopted; providers revoke or rate-limit access and members must be dropped; a model is deprecated and replaced by its successor. Each of these is a perturbation to the panel, and the operational question is one of stability under churn: when a member is added, removed, or swapped, does the composite's accuracy move smoothly and predictably, or can a single substitution destabilize a panel that was previously winning (as §4.2's correlated-lineage crossover warns it might)? We propose to measure the coordinator under a stream of membership changes that mirror real life — periodic upgrades, sudden provider blocks, graceful replacements — and to report not just steady-state accuracy but the transient behaviour around each change. This is the question that decides whether the architecture is something one can run in production for a year, not merely demonstrate once.

Why a separate paper. §11.1 completes the present argument; §11.2–§11.4 begin a different one. The robustness-and-dynamics program treats the coordinator as a living system embedded in an adversarial, time-varying environment — a framing whose natural experiments (dropout curves, frontier panels, churn streams) are numerous enough, and distinct enough in method, that folding them into this paper would dilute its single, clean claim. We therefore flag them here as the explicit successor work, and intend to report them as a companion paper that takes this proof-of-concept as its premise.

11.5 · The judge-design space — three routes past the vote

A continuation of the synthesis thread (§5.1, §11.1), distinct from the successor program above. §5.1 fixed the open problem precisely: for a tight, diverse free panel a reconciling judge does not beat the plain vote — a judge weaker than the panel's best is lifted by the drafts yet capped below it, and a judge at the panel's strength reverts to its own answer (self-preference bias ^[25][26]). Three routes could cross that bound; we record all three here as the synthesis-tier roadmap — none yet run at scale — so the directions are not lost.

(a) A judge stronger than the panel's best. The capability lever — a judge strong enough to identify the correct draft, or supply the answer when every draft misses, not merely re-rank them. Our off-chart frontier-judge probe (§5.1) bounds the upside (reasoning over the drafts beat counting them by +19 points on the contested items) but carries the structural caveat that keeps it off the headline: at the genuine frontier the model able to adjudicate the panel does not yet exist — a superior judge is available only when the panel is deliberately a tier below frontier. The honest free-tier form is a judge one clear tier above a sub-frontier panel.

(b) A purpose-trained judge / critic model. Rather than a general model pressed into judging, a model fine-tuned for critique or evaluation — a generative reward or critic model ^[21][22] — may beat a same-strength general model at the reconciliation task specifically. The open question is whether a reliable critic-tuned model of at-least-panel strength is freely available; identifying and screening one is direct future work.

(c) Agreement-gated escalation — the cost-efficient route. The cheapest and likely most practical design: do not judge every question. Keep the free panel's answer wherever it is unanimous — that consensus is already a strong correctness signal (§5.2) — and escalate only the contested minority to a stronger or trained judge. A caveat on the gate's core assumption: unanimity is a reliable correctness signal only when the question is within the panel's competence. On items beyond all members' ability, errors can correlate — the panel agrees on the same wrong answer — and a raw agreement gate passes that confident-but-wrong consensus through unescalated (precisely the wrong-majority headroom §5.1/§8 flag). The gate's strength is therefore bounded by how decorrelated the panel's errors are; two mitigations follow — gate on calibrated confidence/uncertainty rather than bare agreement, and maximize lineage diversity so a hard question is more likely to surface as disagreement (and thus escalate) than as shared error. Our §5.1 cascade realized exactly this: it matched a strong model's full-bench accuracy while invoking that expensive model on only ~27% of items and spending zero tokens on draft-reading. This reframes the synthesis tier as an inference-economics result as much as an accuracy one — near-strong-model quality at a fraction of the calls, a deployment in which free models handle the easy majority and a costly model is spent only where the panel is genuinely uncertain. Screening method (used here): because a judge only matters where the panel disagrees, candidate judges are evaluated on the contested subset alone (~28% of items) — a fast, discriminating test that screens many judges at a fraction of a full run. That harness is built; candidates across all three routes are queued. First free-tier realization (this revision) — the cascade, composed end-to-end and measured. Route (c) now exists with no paid model anywhere in the loop. The escalation target is the free near-ceiling single the §5.2 coda surfaced — nemotron-3-ultra-550b (86.8 among answered, via a free NVIDIA endpoint) — escalated solo, per the §5.1 finding that drafts subtract value from a judge at or above the panel’s strength; its residual unreachability degrades to the §5.1 draft-reading judge, then the plain vote, then the best member — §8’s degradation ladder, literally implemented (prototype/cascade_coordinator.py). Composed offline from the committed answer banks (prototype/best_coordinator.py), the full ladder scores 86.7% on the mini-bench — +4.6 over the best member (paired-lift CI [+0.5, +8.7]), numerically level with the paid frontier cascade above (86.7) and with the frontier reference answering everything itself (86.7, 170/196) — at zero marginal cost, escalating 26% of items. Two honest brakes, stated before any celebration: (i) it is the winner of an exploratory sweep over coordinator designs on the same frozen 196 — selection across a sweep inflates the best number, so the nominally-positive interval does not survive multiple-comparison correction; it therefore rides as the single pre-registered confirmatory hypothesis on the held-out n≈980 extension of the same stratified sample, and is reported here as the design to beat, not a settled claim; (ii) the free escalation endpoint is slow and serial (~31 s per call) — the 26% escalation fraction is precisely what makes it deployable. The rung economics are the deployment story: roughly three-quarters of questions are answered by consensus of three cheap parallel models, and the one slow strong model is spent only where they disagree. Live-path validation (same day): replaying the mini-bench through the runnable pipeline itself — real API degradation included: one panel member was only partially reachable (its direct-API key absent), so escalation jumped from 26% to 42% of items — the ladder returned 86.2%, within half a point of the offline composite and still level with the frontier reference. The design absorbs a degraded member by spending more escalation, which is precisely the behaviour it exists to guarantee (prototype/cascade_bench_out.json).

References

1. Wang, J. et al. Mixture-of-Agents Enhances Large Language Model Capabilities. arXiv:2406.04692, 2024.
2. Akiba, T. et al. Evolutionary Optimization of Model Merging Recipes. arXiv:2403.13187, 2024. (Sakana AI)
3. Hu, Q. et al. RouterBench: A Benchmark for Multi-LLM Routing Systems. arXiv:2403.12031, 2024.
4. Chen, L. et al. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. arXiv:2305.05176, 2023.
5. Jiang, D. et al. LLM-Blender: Ensembling LLMs with Pairwise Ranking and Generative Fusion. arXiv:2306.02561, 2023.
6. Du, Y. et al. Improving Factuality and Reasoning in Language Models through Multiagent Debate. arXiv:2305.14325, 2023.
7. Wang, X. et al. Self-Consistency Improves Chain-of-Thought Reasoning in Language Models. arXiv:2203.11171, 2022.
8. Li, J. et al. More Agents Is All You Need. arXiv:2402.05120, 2024.
9. Wang, Y. et al. MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark. arXiv:2406.01574, 2024.
10. Wu, Q. et al. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. arXiv:2308.08155, 2023. (Microsoft)
11. Park, J. S. et al. Generative Agents: Interactive Simulacra of Human Behavior. arXiv:2304.03442, 2023. (Stanford)
12. Polo, F. M. et al. tinyBenchmarks: Evaluating LLMs with Fewer Examples. arXiv:2402.14992, 2024.
13. Lightman, H. et al. Let's Verify Step by Step. arXiv:2305.20050, 2023. (OpenAI — process supervision)
14. Shinn, N. et al. Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366, 2023.
15. Madaan, A. et al. Self-Refine: Iterative Refinement with Self-Feedback. arXiv:2303.17651, 2023.
16. Chen, X. et al. SLM-MUX: Orchestrating Small Language Models for Reasoning. arXiv:2510.05077, 2025. (discussion-based composition fails on small models)
17. Author(s) et al. Best-of-∞: Asymptotic Performance of Test-Time LLM Ensembling. arXiv:2509.21091, 2025. (optimal ensemble weights as a MILP)
18. Xi, Z. et al. Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision. arXiv:2411.16579, 2024.
19. Mousavi, S. M. et al. N-Critics: Self-Refinement of LLMs with Ensemble of Critics. arXiv:2310.18679, 2023.
20. Yan, Y. et al. Dancing with Critiques: Enhancing LLM Reasoning with Stepwise Natural Language Self-Critique. arXiv:2503.17363, 2025.
21. Smit, A. P. et al. Should We Be Going MAD? A Look at Multi-Agent Debate Strategies for LLMs. ICML 2024 (arXiv:2311.17371). (MAD does not reliably beat self-consistency; hyper-parameter sensitive)
22. Li, D. et al. Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge. arXiv:2410.02736, 2024. (position, verbosity, self-preference + 12 judge biases)
23. Wu, K. et al. Self-Preference Bias in LLM-as-a-Judge. arXiv:2410.21819, 2024. (judges favour their own family's outputs)
24. Wang, J. et al. Mixture-of-Agents Enhances Large Language Model Capabilities. arXiv:2406.04692, 2024. (proposer diversity, not raw strength, drives aggregation gains)
25. Li, W. et al. Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial? arXiv:2502.00674, 2025. (Self-MoA — aggregating the single best model beats mixed-panel MoA; quality dominates diversity)
26. Author(s) et al. Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models? arXiv:2508.17536, 2025. (NeurIPS 2025 — majority voting accounts for most of debate's gains; debate adds no expected correctness)
27. Ahmadi, S. et al. DFPE: A Diverse Fingerprint Ensemble for Improving LLM Performance. arXiv:2501.17479, 2025. (per-domain reliability-weighted voting + weak-member pruning; 70.8 → 73.5 on a spread open-model panel)