An agent that runs for five minutes can ignore failure. One that runs for five hours will see a timeout, a 429, a dropped stream and probably a restart. Most guides on long-running agents sell a workflow engine. You may not need one. This post covers the five habits that keep a plain agent loop alive for hours, with code for the OpenAI Python SDK.
What breaks when an agent runs for hours?
| Failure | How it looks | Habit that handles it |
|---|---|---|
| Hung or slow call | No reply for minutes | Per-call timeout, streaming |
| Rate limit or overload | 429, 5xx | Bounded retry with backoff |
| Dropped stream | Connection ends before [DONE] |
Treat as incomplete, retry the call |
| Duplicate side effect | Tool runs twice after a retry | Idempotency keys |
| Process crash or deploy | Agent restarts at step 1 | Checkpoint per step |
1. How long should a call be allowed to take?
The OpenAI Python SDK’s README says requests time out after 10 minutes by default and certain errors are retried twice. For an agent, set both yourself, so the behavior doesn’t change under you:
client = OpenAI(base_url="https://inference.broadnet.ai/v1", api_key=os.environ["BROADNET_API_KEY"],
timeout=300, max_retries=4)
Pick the timeout from data. Log the duration of every call for a day; set the timeout well above the slowest successful call. Large prompts take longer to start, and reasoning models think before writing. At about 100 tokens per second, 8,000 output tokens take over a minute on their own. Add a separate wall-clock budget per task so a loop that keeps going can’t run all night.
2. Why stream in a long-running agent?
Streaming gives you a signal while the model works. Without it, a slow call and a dead connection look the same until the timeout fires. With it, you can see chunks arrive and also get token usage in the last chunk.
BroadNet’s streaming rules: read until the literal data: [DONE] line. If the connection closes before it, the answer is incomplete. Record the X-Request-ID, then decide whether to make a new request. Full streaming guide.
def call_streaming(messages, tools=None):
parts, done = [], False
stream = client.chat.completions.create(
model="broadnet/glm-5.3", messages=messages, tools=tools, stream=True,
stream_options={"include_usage": True}, max_tokens=8000)
for chunk in stream:
if chunk.choices and chunk.choices[0].finish_reason:
done = True
if chunk.choices and chunk.choices[0].delta.content:
parts.append(chunk.choices[0].delta.content)
if not done:
raise IncompleteStream() # retry the whole call
return "".join(parts)
This sketch collects text only. If you stream tool calls, accumulate delta.tool_calls by index as well.
3. What should you retry?
The SDK already retries connection errors, 408, 409, 429 and 5xx. Layer your own retry on top only for what it can’t see, such as an incomplete stream:
- Retry: connection errors, 429, 5xx, incomplete streams. Use exponential backoff with jitter and a cap, for example 4 attempts.
- Fix, don’t retry: 400, 401, 402, 403, 413, 422. These fail the same way every time. See error codes and when to retry.
- Budget retries per task, not only per call. Ten steps each retrying four times is forty extra calls.
- Fail over once retries are spent. If a run must survive a provider outage, move connection errors, 5xx and rate-limit 429s to a second provider. LLM API failover shows the client code and a circuit breaker.
On pay-as-you-go, every retry is a new inference, billed again. A pay-as-you-go key also allows 30 requests a minute and 4 in flight, so retry storms hit the rate limit quickly; the rate limit calculator shows how much headroom your agent has. The 429 handling post covers backoff in detail. On an unlimited plan, retries cost nothing extra, but they occupy the session’s single request slot while they run.
4. How do you make tool calls idempotent?
A model call has no side effects. A tool call can. If the agent crashes after send_invoice ran but before the step was saved, the restarted agent will send it again.
Give each side-effecting call an idempotency key derived from the task and step, check it before acting, and skip the call if it’s already recorded:
def run_side_effect(task_id, step, call, store):
key = f"{task_id}:{step}:{call.id}"
if store.done(key):
return store.result(key) # already ran: return the saved result
result = execute(call) # pass key to the API if it supports one
store.save(key, result)
return result
Many payment and messaging APIs accept an idempotency key header. Pass yours through when they do.
5. How do you checkpoint an agent?
Save after every completed step: the message list, the step number and the side-effect keys. On start, load the latest checkpoint if one exists.
def save_checkpoint(path, step, messages):
tmp = path + ".tmp"
with open(tmp, "w") as f:
json.dump({"step": step, "messages": messages}, f)
os.replace(tmp, path) # atomic on the same filesystem
Writing to a temp file and renaming means a crash mid-write never leaves a half checkpoint. For very long sessions, also checkpoint a summary: replace old turns with a short summary so a resumed agent doesn’t reload a huge history. That keeps you inside the context window, even at 1,048,576 tokens on DeepSeek V4.1 Flash.
A checklist before an overnight run
- Timeout and
max_retriesset explicitly on the client. - Streaming on, with
include_usage, and incomplete streams retried. - A retry budget per task and a wall-clock limit.
- Idempotency keys on every tool that writes, sends or pays.
- A checkpoint after every step, written atomically.
- Logs with step number, duration, tokens and
X-Request-ID.
If you run several agents side by side, read running parallel coding agents on a flat rate and what multi-agent setups cost.
Where BroadNet Inference fits
BroadNet Inference uses standard chat-completions streaming, returns an X-Request-ID header you can log, and documents which errors to retry, so the patterns above work as written. For agents that run all day, the unlimited plans remove the token meter on DeepSeek V4.1 Flash for $60 a month per session, which makes retries free. Keep in mind that a Meteor hour ends one hour after purchase and an Orbit week after seven days, so a long job must finish inside the period or move to a renewing Full Moon session. Plans are at inference.broadnet.ai/plans.
