API

Errors

Ten of them. Four are worth retrying, and the difference matters more here than on a cloud API.

The error shape

Errors come back with an HTTP status and a body of the same shape every time. The type is the field to branch on — the message is written for a human and may change.

json

{  "error": {    "type": "no_capacity",    "message": "No Core-class node currently meets max_latency_ms 120.",    "param": "vacuum.max_latency_ms",    "retry_after_ms": 2000  }}

retry_after_ms is present on transient errors and is the router’s own estimate of when capacity will exist. Prefer it over your own backoff when it is there.

Every error

Status	Type	What happened	Retry
400	invalid_request	Malformed body, or a parameter outside its range.	No
400	context_length_exceeded	Prompt plus max_tokens exceeds the model context window.	No
401	invalid_key	Missing, malformed or revoked bearer token.	No
402	insufficient_balance	The balance cannot cover the reservation for this request.	After topping up
404	model_unknown	No catalogue entry by that name.	No
409	model_unavailable	The pinned registry build is no longer served by any node.	No — repin
429	rate_limited	Too many concurrent requests on this key.	Yes, with backoff
503	no_capacity	No node matches the routing constraints right now.	Yes, or relax the constraints
504	node_timeout	The node accepted the work and did not deliver in time.	Yes
502	stream_interrupted	A stream ended without [DONE] and no replacement node was found.	Yes

A failed request is never billed. Reservations are released on every path out, including timeouts and interrupted streams, so a retry storm costs you latency but not balance.

Retrying well

no_capacity and node_timeout are ordinary on a network of consumer machines, not signs that something is broken. They are also the two errors most often caused by your own routing constraints: a tight max_latency_ms or a high min_reputation can empty the pool at peak hours. Before adding retries, check whether you narrowed it yourself.

typescript

async function send(body, attempt = 0) {  const res = await fetch(BASE + "/chat/completions", { ...init, body });  if (res.ok) return res.json();   const { error } = await res.json();   // Only these are worth trying again; everything else is your bug or your balance.  const transient = ["no_capacity", "node_timeout", "stream_interrupted", "rate_limited"];  if (!transient.includes(error.type) || attempt >= 3) throw new Error(error.message);   const wait = error.retry_after_ms ?? 250 * 2 ** attempt;  await new Promise((r) => setTimeout(r, wait + Math.random() * 200));  return send(body, attempt + 1);}

Retry no_capacity, node_timeout, stream_interrupted and rate_limited. Nothing else — the rest will fail identically forever.
Cap at three attempts, and jitter the wait. Everyone reading these docs is writing the same loop against the same pool.
For a stream, a retry restarts from scratch and produces a new receipt. Do not present partial output from the failed attempt as part of the new one.
If no_capacity persists for a model, check nodes_serving for it. A model with few nodes is a model you should not build a critical path on yet.