BroadNet’s second Uno preprint applies self-drafting to a mixture-of-experts model for the first time. Published on Zenodo on 18 September 2026 under DOI 10.5281/zenodo.22820511, it serves Gemma 4 26B A4B at 4-bit and trains an Uno adapter for it. The target is the same base model as BroadNet Gemma 4 VL 26B on BroadNet Inference.
It follows the Uno in vLLM serving study, which tested a dense 8B model. The numbers below are BroadNet’s; we have not reproduced them.
What was the question?
Speculative decoding drafts several tokens and verifies them in one pass. On a mixture-of-experts model there is an extra worry: draft rows can route to different experts, so drafting four tokens might read many more expert weights than generating one. The paper asks whether that “expert fan-out” makes self-drafting too expensive, and when self-drafting beats a separate drafter.
How did BroadNet test it?
From the paper’s abstract and BroadNet’s methods line:
- Model: Gemma 4 26B A4B, about 4B parameters active per token, with 4-bit weights.
- Hardware: 24 GB-class GPUs, one sequence at a time.
- Workload: 72 prompts from BroadNet’s production traffic; each speed-up is divided by a plain-decoding run in the same session.
- Long context: a separate set of five prompts at about 10,500 tokens.
- Adapter: rank 16, trained on 965 rows through the same 4-bit checkpoint it is served on, on a single 24 GB GPU, with no full-precision weights in the loop.
What did it find?
| How Gemma 4 26B A4B was decoded | Speed-up over plain decoding |
|---|---|
| Plain decoding | 1.000 |
| Uno pilot adapter, width 4 | 1.166 |
| Multi-token-prediction drafter, width 2 | 1.474 |
| Block drafter, width 8 | 1.553 |
| Uno at about 10,500 tokens of context | 1.089 (MTP drafter: 1.011) |
Other results BroadNet reports:
- Training helped. Accepted share rose from 42.4% to 48.9% over 1,500 updates, and a larger 2,000-row corpus, trained on the same recipe, accepted 0.49 more draft tokens per verify step than the matched anchor run.
- Expert fan-out is real but cheap. Four draft rows touched 22 distinct experts against one row’s 8, yet the whole row-related cost was bounded by 0.53 steps in a cycle of about 2.4. BroadNet concludes that routing cost is “worth under a fifth of a cycle” on this class of hardware.
- Lossless. BroadNet built a check for a profile whose plain output does not reproduce itself across sessions, and reports Uno “as close to plain as plain is to itself”.
- Five languages. The paper gives what it calls the first measurement of this adapter’s acceptance across five languages.
What does it mean in practice?
The paper’s own rule is simple: if a good small drafter exists for your model, use it; self-drafting earns its place at long context and on models that have no drafter. That matches what BroadNet says it runs: its catalogue lists Gemma 4 26B in production with a multi-token-prediction drafter (k=2), with “90%+ acceptance on structured prompts”.
How much does a drafter add in production? BroadNet’s catalogue reports that multi-token prediction added 48.8% on an RTX 3090, and its Gemma 4 VL 26B page gives about 136 tokens per second on a single stream with speculative decoding on. Both are BroadNet’s own measurements, on the serving configuration its model page describes, so your speed through the API also depends on load and on the length of your prompt and answer.
For buyers of an API, the useful part is the method, not the adapter. BroadNet says every model it serves goes through this arithmetic before it reaches a customer: measure the cost of one draft cycle, then pick the drafter that wins on the actual workload. The same discipline led it to rule out EAGLE3 for saturated serving.
What are the limits?
- Single-stream only. One sequence at a time, one run per arm. Batch serving behaves differently, as the dense study showed.
- One model. Results are for Gemma 4 26B A4B at 4-bit on 24 GB-class cards.
- A preprint. Version 2.3 on Zenodo, not peer-reviewed, with a companion evidence archive attached.
Release v0.3.0 of vllm-uno ships the serving profile, a container image and the validation record, so the numbers can be checked. To try the served model itself, BroadNet’s browser playground runs Gemma 4 with no account.
