Synchronous calls have about two minutes
A normal (synchronous) call has about 115 s to return. Long-running endpoints such as the AI asks fit their retries into that window: if they can’t finish, you get an error response instead of a dropped connection. Failed calls are not billed.- Set your HTTP client timeout to 130 s for AI endpoints. A shorter timeout aborts answers that would have arrived.
- If your connection drops before the response arrives, the call is not billed.
Run slow calls asynchronously
AddPrefer: respond-async, wait=25 to any read request. Async calls are not bound by that window.
-
If the call finishes within
waitseconds (1–120), you get the normal response. -
Otherwise you get
202 Acceptedwith arequest_id:
GET /v1/requests/{request_id}. Polling is free. It returns 202 while the call runs, then the original status, headers and body. Results stay available for about 15 minutes.
The call is billed once, when it completes. A call that never completes is not billed. Don’t re-send the original request while it runs: that starts, and bills, a second call.
Prefer: respond-async and poll the request_ids.
Concurrency
There is no per-key concurrency limit beyond your plan’s rate limit. AI asks render a live answer, so latency rises when many run at once. Start around 10 in parallel per engine and scale up while you watch the error rate. A502 on an AI ask means the engine produced no usable answer on that attempt. It is not billed, and retrying it is safe.
