Limits

Rate limits per key, spend limits per key and per workspace, the prepaid credit hard stop, request size and time limits, and what happens when each is reached.

Rate limits

Each API key has a requests-per-minute limit.

The default is 60 requests per minute. You can set a key's own limit, from 1 to 10,000, when you create or edit it on API keys.
Requests are counted over a rolling 60-second window.
Past the limit, requests are refused with 429 rate_limited and a Retry-After header giving the seconds to wait. Refused requests are not counted.
The limit is applied by each gateway server process separately, so treat it as approximate: short bursts can go somewhat over the number you set. It is there to stop runaway loops. The dollar limits below are what cap your spend.

Runs in the Console Workbench are limited to 60 a minute for each person.

When a provider rate limits the gateway itself, the request is refused with 429 upstream_rate_limited and Retry-After: 5.

Spend limits

Before every request, the gateway checks three things against the most the request could cost, in this order, and refuses with 402 at the first that fails:

Check	Refused with	Who sets it
Prepaid credit. Your balance, minus what requests in flight have reserved, must cover this request. A balance at or below zero refuses every request.	`402 insufficient_credit`	Credit is added by Safeguard, or by card where card top-up is on.
The key's monthly limit. What the key spent this month, plus its requests in flight, plus this request, must stay within its limit.	`402 key_spend_limit_exceeded`	The key's owner, on API keys
The workspace's monthly limit. The same sum for the whole workspace must stay within its limit.	`402 account_spend_limit_exceeded`	A tenant admin or super admin, in Usage under Gateway settings

Months are calendar months in UTC. A new month starts every limit at zero spent.
A key or workspace with no monthly limit is limited only by credit.
Workbench runs spend the workspace's credit and count toward the workspace limit. They use no key, so no key limit applies.

Spend controls explains how to set and raise each limit.

How the most a request could cost is worked out

The gateway cannot know a request's cost before the provider answers, so it holds the most it could cost and charges the actual cost afterwards.

Input. It estimates the input tokens from your messages, generously, at about one token for every three characters plus a little for each message.
Output. It assumes the full output cap: your max_completion_tokens or max_tokens, or when you set none, 4,096 tokens or the model's maximum, whichever is lower.
Price. It prices both at the model's catalog prices. When fallbacks are allowed, it uses the most expensive of the models it might try.

That amount is held while the request runs and shows as Reserved in the Console. When the request finishes, the actual cost is charged and the rest of the hold is released. A hold that is never settled, for example because a request was cut off, is released after about 30 minutes.

If you are close to your credit or a limit, a request with a smaller max_tokens holds less and may fit where a larger one does not.

The actual cost can be higher than the hold only when the provider reports more input tokens than the estimate. The provider bills that, so it is charged, and the balance can go slightly below zero. Requests are then refused until credit is added.

Request size

Limit	Value
Request body	8 MiB. Larger bodies get `413 request_too_large`.
Messages	1 to 2,000
`max_completion_tokens` or `max_tokens`	1 to 1,000,000, then lowered to the model's maximum output
Output cap when you set none	4,096 tokens, or the model's maximum if lower
`stop`	Up to 4 sequences
`n`	1
Content	Text only

Time

Limit	Value
Waiting for a provider to respond	120 seconds, then `504 upstream_timeout`
A single stream	900 seconds, then an `upstream_timeout` error event