---
title: "What is exponential backoff? Retries with jitter explained"
description: "Exponential backoff retries a failed API call after waits that grow each attempt, plus random jitter. How it works, what to retry, and how to handle LLM 429s."
url: https://broadrouter.com/glossary/exponential-backoff/
lang: en
updated: 2026-11-16
publisher: BroadRouter (a BroadNet site); API by BroadNet Inference, https://inference.broadnet.ai
---

# Exponential backoff

Updated 2026-11-16

Exponential backoff is a retry strategy in which a client waits longer after each failed attempt, typically doubling the delay up to a cap, before trying again. Adding random jitter to each wait spreads out retries from many clients so they do not hit the server at the same moment.

Also called: backoff and jitter, retry with backoff, jittered backoff, capped exponential backoff, full jitter

When an API says “slow down”, retrying at once makes things worse. Exponential backoff waits longer each time.

## How it works

After a failure, wait a base delay, then double it on each new failure, up to a maximum. With a 1-second base and a 32-second cap, the waits run 1, 2, 4, 8, 16, 32, 32 seconds. Stop after a fixed number of attempts and surface the error.

Backoff alone is not enough. If 100 clients fail at the same moment, they all retry at the same moments too. Marc Brooker’s AWS post compares “Full Jitter” (a random wait between zero and the current cap), “Equal Jitter” and “Decorrelated Jitter”. Equal Jitter came out weakest, and Full Jitter “uses less work, but slightly more time.” His conclusion: jittered backoff “should be considered a standard approach for remote clients.”

```python
import random, time

def call_with_backoff(fn, attempts=6, base=1.0, cap=32.0):
    for n in range(attempts):
        try:
            return fn()
        except RetryableError:
            if n == attempts - 1:
                raise
            time.sleep(random.uniform(0, min(cap, base * 2 ** n)))  # full jitter
```

## Why it matters

-   **429s count too.** OpenAI’s rate-limit guide warns that “unsuccessful requests contribute to your per-minute limit, so continuously resending a request won’t work.” Hammering a [rate limit](https://broadrouter.com/glossary/rate-limit/) keeps you locked out.
-   **Retries multiply.** AWS’s Builders’ Library notes that retries are “selfish” and recommends retrying at a single point in the stack; retrying at every layer can multiply load many times over.
-   **Side effects.** Retrying a call that charges money is safe only with an [idempotency key](https://broadrouter.com/glossary/idempotency-key/).
-   **Prevention beats retry.** Cap requests in flight below your [concurrency](https://broadrouter.com/glossary/concurrency/) limit so 429s are rare in the first place.

## On BroadNet Inference

BroadNet’s error table says to answer a 429 with “response details and bounded backoff” and to retry 500, 502, 503 and 504 “with bounded exponential backoff.” A pay-as-you-go key allows 30 requests per minute and 4 in flight. For 400, 401, 403, 413 and 422, fix the cause first. Keep the `X-Request-ID` header for support. The [errors and retries guide](https://broadrouter.com/guides/errors-and-retries/) sets SDK retry counts, and [how to handle LLM API 429s](https://broadrouter.com/blog/handle-llm-api-429-rate-limits/) has limiter and queue code. The [rate limit calculator](https://broadrouter.com/tools/llm-rate-limit-calculator/) shows how much traffic fits under a limit.

## Questions

### Which errors should I retry with backoff?

Rate limits (429), server errors (5xx), timeouts and dropped connections. Do not retry 400, 401, 403, 413 or 422 unchanged; fix the request or the key first.

### Why add jitter?

Without it, clients that failed together retry together and overload the server again. Jitter randomises each wait so retries spread out.

### Do the OpenAI SDKs already back off?

Yes. The official OpenAI SDKs retry connection errors, 429 and 5xx responses with a short exponential backoff. Set max\_retries explicitly, higher for long batch jobs, and add your own limiter on top.

## See also

-   [Rate limit](https://broadrouter.com/glossary/rate-limit/) (term)
-   [Idempotency key](https://broadrouter.com/glossary/idempotency-key/) (term)
-   [Concurrency](https://broadrouter.com/glossary/concurrency/) (term)
-   [API key](https://broadrouter.com/glossary/api-key/) (term)
-   [Request timeout](https://broadrouter.com/glossary/request-timeout/) (term)
-   [Model fallback](https://broadrouter.com/glossary/model-fallback/) (term)
-   [BroadNet Inference error codes and when to retry](https://broadrouter.com/guides/errors-and-retries/) (guide)
-   [Use the OpenAI Python SDK with BroadNet Inference](https://broadrouter.com/guides/openai-python-sdk/) (guide)
-   [How to handle LLM API 429 errors: backoff, jitter and limiters](https://broadrouter.com/blog/handle-llm-api-429-rate-limits/) (article)
-   [Long-running agents: timeouts, retries and checkpoints that hold](https://broadrouter.com/blog/long-running-agents-timeouts-retries/) (article)

Try it on your real workload. New accounts start with $5 of credit for Gemma and transcription. DeepSeek, GLM and Qwen3.8 need a top-up.

[Get started](https://inference.broadnet.ai/account/?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-exponential-backoff&utm_content=page-cta) ·

**Sources**

-   [AWS Architecture Blog, Exponential Backoff And Jitter (Marc Brooker)](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/)
-   [AWS Builders' Library, Timeouts, retries and backoff with jitter](https://builder.aws.com/content/3EumjoZascWd1oZiEgL8ORlv3qE/timeouts-retries-and-backoff-with-jitter)
-   [OpenAI, rate limits guide (retrying with exponential backoff)](https://developers.openai.com/api/docs/guides/rate-limits)
-   [BroadNet Inference API reference, errors, request IDs and retries](https://inference.broadnet.ai/docs?utm_source=broadrouter&utm_medium=referral&utm_campaign=glossary-exponential-backoff&utm_content=sources#errors)
