---
title: "Uno in vLLM: BroadNet serving study, DOI and results"
description: "BroadNet built the Uno self-drafting adapter into vLLM and measured it on an H100: 2.58x plain throughput at one stream, 1.74x at 32. The method, DOI and limits."
url: https://broadrouter.com/blog/uno-in-vllm-serving-study/
lang: en
updated: 2026-11-15
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Uno in vLLM: what BroadNet's serving study measured

By the BroadNet Inference team · 4 min read · Published 2026-11-15

Uno in vLLM is a September 2026 preprint by BroadNet (DOI 10.5281/zenodo.22652610). It adds Uno, an adapter that lets a model draft its own tokens, to vLLM. On an H100 with Qwen3-8B, BroadNet measured 2.58 times plain-vLLM throughput at one stream, 2.20 times at eight and 1.74 times at 32, within 3% of the reference implementation.

## Key takeaways

-   The paper is a preprint on Zenodo, DOI 10.5281/zenodo.22652610, published 8 September 2026 under CC BY 4.0. It is not peer-reviewed.
-   Uno is IFM's method; BroadNet's contribution is the native vLLM integration and the serving measurements.
-   On a controlled H100 and Qwen3-8B math workload: 2.58x plain vLLM at 1 stream, 2.20x at 8, 1.74x at 32.
-   With production settings, 7,657 tokens per second at 32 streams, against 7,505 for the SGLang reference.
-   The gains are for one model, one workload and specific settings. Measure on your own traffic before you rely on them.

**On this page**

1.  What problem does Uno address?
2.  What did BroadNet build?
3.  How was it measured?
4.  What did the study find?
5.  What are the limits?
6.  Why does it matter if you just call an API?

Uno in vLLM is a preprint from BroadNet’s research arm, published on Zenodo on 8 September 2026 under DOI [10.5281/zenodo.22652610](https://doi.org/10.5281/zenodo.22652610). It reports a native vLLM implementation of Uno, a speculative-decoding method developed by IFM, and measures how much faster it serves than plain vLLM. Its Zenodo record lists Mohammad Al Zubaidi as the author.

This page summarises what the paper says, how it measured it, and what it does and does not tell you. The numbers are BroadNet’s; we have not reproduced them.

## What problem does Uno address?

A language model normally generates one token per forward pass. Speculative decoding speeds this up by guessing several tokens ahead and checking them in one pass of the full model. Most methods use a separate, smaller draft model. Uno instead attaches a trained adapter to the model itself, so the model drafts its own tokens and then verifies them with its original weights. See [speculative decoding](https://broadrouter.com/glossary/speculative-decoding/) for the general idea.

The paper’s question is practical: does that speed-up survive inside a production serving engine, with batching, caching and many requests at once?

## What did BroadNet build?

BroadNet’s blog post is clear about credit: “Uno, developed by IFM and collaborators” is the method, and BroadNet’s contribution is “the native vLLM integration, the engineering that makes its execution paths work together, and the serving evaluation”. The abstract lists the execution pieces: conditional adapter routing, complete draft key/value state, asynchronous scheduling, CUDA graph replay, overlapping adapter computation and seed-row reuse.

The code is public in BroadNet’s [vllm-uno repository](https://github.com/brntech/vllm-uno), and BroadNet has proposed it to vLLM upstream as pull request 55947.

## How was it measured?

The main test is a controlled one:

-   **Hardware and model:** one NVIDIA H100, Qwen3-8B in bfloat16, with the published Qwen3-8B Uno adapter.
-   **Workload:** eight math prompts, cycled to fill larger batches, each generating up to 512 tokens, with prefix caching off.
-   **Baseline:** ordinary autoregressive vLLM (“plain vLLM”) on the same setup.
-   **Concurrency:** 1, 8 and 32 simultaneous streams.

A second, separate comparison turns prefix caching on and uses BroadNet’s chosen serving settings, to compare against the SGLang reference implementation.

## What did the study find?

| Measurement | Result reported by BroadNet |
| --- | --- |
| Throughput vs plain vLLM, 1 stream | 2.58x |
| Throughput vs plain vLLM, 8 streams | 2.20x |
| Throughput vs plain vLLM, 32 streams | 1.74x |
| Gap to the SGLang reference implementation | within 3% at 1, 8 and 32 streams |
| Production settings, 32 streams | 7,657 tokens/s, against 7,505 for SGLang (2.0% ahead) |
| Warm first-token latency, within-vLLM ablation | 25% lower with two API processes instead of one |

The headline for capacity planning is the last row of the first block: 74% more output per second at 32 concurrent streams, from the same GPU. That is what BroadNet’s blog calls “the central payoff”, because a speed-up that disappears under load does little for a shared API.

A later release (v0.2.0, on vLLM’s Model Runner V2) is described on BroadNet’s blog with 2.1 to 2.4 times single-stream output on Blackwell, 1.7 to 1.9 times on Ampere and 2.6 times on Hopper. Those are different hardware and versions from the paper’s main table, which is why the figures differ.

## What are the limits?

-   **One model, one workload.** The controlled result is Qwen3-8B on math prompts. Code, chat or long documents can accept fewer drafted tokens.
-   **Gains shrink with load.** The trend from 2.58x to 1.74x is the expected shape: when the GPU is busy, spare compute for drafting runs out.
-   **A preprint.** The paper is deposited on Zenodo, not peer-reviewed. Its companion archive holds the evidence inventory and figure data, which is what lets others check it.
-   **Self-reported.** BroadNet ran both arms. The 3% agreement with the reference implementation is the paper’s own check on that.

## Why does it matter if you just call an API?

Throughput per GPU sets what a provider can charge and how fast it answers under load. BroadNet’s catalogue reports that it measures drafting methods at real concurrency before choosing one, and that it ruled out EAGLE3 after it cost 69% of throughput at 32 concurrent requests. The [follow-up study on Gemma 4 26B](https://broadrouter.com/blog/uno-mixture-of-experts-gemma-4-study/) applies the same method to the mixture-of-experts model BroadNet sells as [Gemma 4 VL 26B](https://broadrouter.com/models/gemma-4-vl-26b/). For how this fits BroadNet’s wider serving choices, see [BroadNet’s GPU hardware and serving approach](https://broadrouter.com/blog/broadnet-gpu-fleet-and-serving/).

If you run your own vLLM servers, the practical lesson is to benchmark any drafting method at the concurrency you actually run, not at one stream.

Free tools for this

-   [OpenAI to BroadNet migration helper](https://broadrouter.com/tools/openai-to-broadnet-migration/) Paste OpenAI SDK code; get base URL, key and model edits, a diff and the checks that remain.
-   [max\_tokens calculator for reasoning models](https://broadrouter.com/tools/reasoning-token-budget/) Answer length + reasoning budget + margin = a max\_tokens that does not cut the answer off.
-   [JSON schema generator for LLM structured output](https://broadrouter.com/tools/json-schema-generator/) Sample JSON to strict schema, Pydantic, Zod and a ready request.
-   [LLM JSON output validator](https://broadrouter.com/tools/llm-json-output-validator/) Parse errors by line and column, safe repairs, cut-off detection and JSON Schema checks for model output.

[All free tools](https://broadrouter.com/tools/)

## Terms in this article

[vLLM](https://broadrouter.com/glossary/vllm/)

vLLM is an open-source library for LLM inference and serving, first developed at UC Berkeley's Sky Computing Lab. It manages KV cache memory with PagedAttention, batches incoming requests continuously, supports many quantization formats and exposes an OpenAI-compatible HTTP server.

[Token](https://broadrouter.com/glossary/token/)

A token is the unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence produced by the model's tokenizer. LLM APIs count, limit and usually bill usage in tokens.

[Throughput](https://broadrouter.com/glossary/throughput/)

Throughput, in LLM serving, is the total amount of work a system completes per unit of time across all concurrent requests, usually measured as output tokens per second or requests per second. It differs from per-stream speed, which is how fast a single request receives its tokens.

[Concurrency](https://broadrouter.com/glossary/concurrency/)

Concurrency is the number of requests a client has in flight at the same time. LLM APIs often cap it per key or plan, separately from requests per minute, because each open request holds GPU capacity until its response has finished.

[Speculative decoding](https://broadrouter.com/glossary/speculative-decoding/)

Speculative decoding is an inference technique in which a small, fast draft model proposes several next tokens and the large target model verifies them in a single forward pass, keeping the accepted ones. It speeds up generation without changing the distribution of the output.

[Latency](https://broadrouter.com/glossary/latency/)

Latency, for a language model API, is the time a request takes from being sent to the final output token being received. It is the sum of time to first token and generation time, and is usually reported as percentiles such as p50 and p95 rather than as an average.

[Mixture of experts (MoE)](https://broadrouter.com/glossary/mixture-of-experts/)

Mixture of experts (MoE) is a neural network design in which a layer holds many expert sub-networks and a learned router sends each token to only a few of them. An MoE language model has a large total parameter count but uses a small active subset for each token.

[Time to first token (TTFT)](https://broadrouter.com/glossary/time-to-first-token/)

Time to first token (TTFT) is the time from sending a request to a language model until the first output token arrives. It includes network time, queueing on the server and prefill, the step where the model reads the whole prompt, so it grows with prompt length and load.

## Questions

### What is Uno?

Uno is a speculative-decoding method developed by IFM and collaborators. A small trained adapter lets the model propose several tokens at once, and the model's own weights then verify them. There is no separate draft model.

### Is the speed-up lossless?

BroadNet says so for its release: whenever plain vLLM reproduces its own output, Uno matches its greedy tokens, and its sampled-distribution tests pass. Speculative decoding verifies every drafted token with the full model, so output quality should not change; check this on your own prompts.

### Does Uno help at high concurrency?

Less than at low concurrency. In the paper's controlled test the gain fell from 2.58x at one stream to 1.74x at 32. BroadNet's catalogue also reports that another method, EAGLE3, cost 69% of throughput at 32 concurrent requests, so drafting methods need testing at your real load.

### Can I run it myself?

Yes. BroadNet publishes the implementation in its vllm-uno repository, with a container image, pinned versions and a validation record for each release, and has proposed it to vLLM as pull request 55947.

### Does BroadNet Inference use Uno for its API models?

BroadNet does not say so. Its catalogue lists multi-token-prediction drafters (k=2) in production for Gemma 4 26B. Uno is a research result BroadNet publishes; ask BroadNet if you need it on a private endpoint.

## Keep reading

-   [Uno on Gemma 4 26B: BroadNet's mixture-of-experts study](https://broadrouter.com/blog/uno-mixture-of-experts-gemma-4-study/)
-   [BroadNet's GPU hardware and serving approach, as BroadNet describes it](https://broadrouter.com/blog/broadnet-gpu-fleet-and-serving/)
-   [LLM latency: TTFT vs tokens per second vs end-to-end time](https://broadrouter.com/blog/llm-latency-ttft-throughput/)
-   [Models BroadNet evaluated and ruled out, and why](https://broadrouter.com/blog/models-broadnet-ruled-out/)
-   [A private endpoint starts with a clear workload](https://broadrouter.com/blog/private-endpoint-planning/)
-   [Before fine-tuning: define the problem to solve](https://broadrouter.com/blog/fine-tuning-readiness/)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=blog-uno-in-vllm-serving-study&utm_content=page-cta) ·

**Sources**

-   [Al Zubaidi, Uno in vLLM: An Independent Implementation and Empirical Serving Study, Zenodo preprint v1.0, 2026-09-08 (record fetched 2026-11-15)](https://doi.org/10.5281/zenodo.22652610)
-   [BroadNet Research: published papers and key findings (fetched 2026-11-15)](https://www.broadnet.ai/research)
-   [BroadNet blog, Bringing Uno to vLLM (8 September 2026; fetched 2026-11-15)](https://www.broadnet.ai/blog/bringing-uno-to-vllm)
-   [vllm-uno repository, BroadNet (fetched 2026-11-15)](https://github.com/brntech/vllm-uno)
-   [IFM, Uno project repository (fetched 2026-11-15)](https://github.com/ifm-ai/uno)
