---
title: "What is chain-of-thought prompting? CoT vs reasoning models"
description: "Chain-of-thought prompting asks an LLM to write out steps before the answer. Where it came from, why it helps, and how reasoning models now do it themselves."
url: https://broadrouter.com/glossary/chain-of-thought/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Chain-of-thought (CoT)

Updated 2026-11-16

Chain-of-thought (CoT) is a prompting technique in which a language model is shown or asked to write intermediate reasoning steps before its final answer. It improves accuracy on multi-step problems such as arithmetic and logic, and reasoning models now generate it automatically.

Also called: CoT, chain-of-thought prompting, step-by-step prompting, CoT prompting

Chain-of-thought started as a prompt trick and became a model feature.

## How it works

Wei and colleagues published the method in 2022. Instead of few-shot examples that show only question and answer, they wrote examples that show the steps in between. With eight such examples, a 540-billion-parameter model reached state-of-the-art accuracy on GSM8K, a set of grade-school maths word problems, ahead of a fine-tuned GPT-3 with a verifier. They tested arithmetic, commonsense and symbolic reasoning.

A chain-of-thought example in a prompt:

```text
Q: A shop has 23 apples. It uses 20 and buys 6 more. How many are left?
A: It starts with 23. After using 20 it has 3. After buying 6 it has 9. The answer is 9.
```

Writing the steps gives the model more tokens of computation before it commits to an answer, and each step conditions the next. It is a form of [few-shot prompting](https://broadrouter.com/glossary/few-shot-prompting/) where the examples teach a method, not just a format.

## How reasoning models internalize it

Reasoning models are trained to produce this step-by-step text themselves. OpenAI’s guide says they “use internal reasoning tokens before producing a response”, and that those tokens are not visible through its API but still use context and are billed as output. Some open models return the reasoning in a separate field instead. See [reasoning tokens](https://broadrouter.com/glossary/reasoning-tokens/) for how they are counted.

## Why it matters

-   **Accuracy.** Multi-step tasks such as maths, planning and code fixes benefit most. Simple lookups gain little.
-   **Cost and latency.** Every step is an [output token](https://broadrouter.com/glossary/input-and-output-tokens/). A long chain can cost more than the answer.
-   **Not proof.** Written steps can look sound and still be wrong, so they do not prevent [hallucination](https://broadrouter.com/glossary/hallucination/).

## On BroadNet Inference

BroadNet’s docs say DeepSeek V4.1 Flash, GLM 5.3 and Qwen3.8 27B reason before they answer, and the reasoning counts toward [`max_tokens`](https://broadrouter.com/glossary/max-tokens/). Leave 1,000 tokens or more, or set `reasoning_effort` to `low`; with a small limit the answer can come back empty. When reported, `usage.completion_tokens_details.reasoning_tokens` shows how many were used. The docs do not list BroadNet Gemma 4 VL 26B as a reasoning model, so classic chain-of-thought prompting is the way to ask it for steps. The [empty-response guide](https://broadrouter.com/guides/reasoning-tokens-empty-response/) and the post on [reasoning token cost](https://broadrouter.com/blog/reasoning-tokens-cost/) cover budgeting.

## Questions

### Is chain-of-thought the same as reasoning tokens?

Related, not the same. Chain-of-thought is the technique of writing out steps. Reasoning tokens are what a reasoning model generates when it does this internally, and they are billed as output.

### Should I write "think step by step" for a reasoning model?

Usually it is unnecessary. Reasoning models already reason before answering. OpenAI's guide suggests giving them a clear goal and constraints rather than prescribing every step.

### Does chain-of-thought cost more?

Yes. The steps are output tokens, which are usually the most expensive tokens, and they add latency.

## See also

-   [Reasoning tokens](https://broadrouter.com/glossary/reasoning-tokens/) (term)
-   [Few-shot prompting](https://broadrouter.com/glossary/few-shot-prompting/) (term)
-   [max\_tokens](https://broadrouter.com/glossary/max-tokens/) (term)
-   [Hallucination](https://broadrouter.com/glossary/hallucination/) (term)
-   [Input and output tokens](https://broadrouter.com/glossary/input-and-output-tokens/) (term)
-   [Why a reasoning model returns an empty answer](https://broadrouter.com/guides/reasoning-tokens-empty-response/) (guide)
-   [Reasoning tokens: why you pay for more output than you see](https://broadrouter.com/blog/reasoning-tokens-cost/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-chain-of-thought&utm_content=page-cta) ·

**Sources**

-   [Wei et al. (2022), Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (checked 2026-11-16)](https://arxiv.org/abs/2201.11903)
-   [OpenAI, reasoning models guide (checked 2026-11-16)](https://developers.openai.com/api/docs/guides/reasoning)
-   [BroadNet Inference API reference, reasoning and cached tokens (checked 2026-11-16)](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-chain-of-thought&utm_content=sources)
