Request lifecycle

Every call to POST /gpt/v1/chat/completions goes through the same steps in the same order. The order tells you which error you get when more than one thing is wrong, and when you start paying. Until step 7 nothing has been sent to a provider, nothing is charged and nothing is written to the request log.

1. Check the key

• The gateway reads the key from the Authorization: Bearer header. A value that is not a well-formed sg-api- key is refused at once with 401 invalid_api_key.
• Otherwise it asks Safeguard's authentication service whose key it is. The same 401 comes back for an unknown, revoked or expired key, a key without the inference scope, and a key whose owner is no longer an active member. The response never says which.
• Once a key checks out, the gateway trusts it for up to 30 seconds before asking again. That is why revoking a key takes up to 30 seconds. An unknown key is not remembered, so a key you have just created works at once.
• If the authentication service cannot be reached, the answer is 503 auth_unavailable.

Sign-in tokens, admin sessions and other kinds of Safeguard key are not accepted on this API.

2. Check that the API is on

The gateway has to be switched on as a whole and for your workspace. If either is off, the answer is 403 gateway_disabled. The gateway reads your workspace's switch from Safeguard's authentication service at most once a minute. If it cannot read it, it treats the switch as off.

3. Check the request body

The body must be JSON of at most 8 MiB, with a model and a non-empty messages array of text messages. A body that breaks the rules gets 400 invalid_request or unsupported_parameter. A body over the size limit gets 413 request_too_large. The fields are listed in Chat completions.

4. Apply the rate limit

Each key may send a set number of requests per minute: 60, unless you chose another number for the key. Over the limit, the answer is 429 rate_limited with a Retry-After header.

5. Check the model

• The gateway looks up model in Safeguard's model catalog. A model that is not there gets 404 model_not_found. Models that are not published in the catalog are never served.
• The model also has to be available through the API right now. All of these must hold:
  • the catalog lists it as available, with both an input price and an output price;
  • Safeguard has credentials set up for its provider;
  • for a third-party provider, Safeguard has confirmed that the provider's terms allow resale, listed the provider on its Trust Center subprocessor list, and updated its DPA, privacy notice and AI disclosure;
  • a real end-to-end call to that model, through that provider, has succeeded.
• If your workspace or the request requires zero data retention, the model must offer it.
• A model that fails any of these gets 503 model_unavailable. With retries and fallbacks on, each listed fallback goes through the same checks.

If the gateway cannot read the provider checklist and verification records, it treats every model as unavailable. It reads them, and the catalog, at most once a minute. If the catalog itself cannot be read, the gateway keeps using its last copy for up to 10 minutes, and after that answers 503.

GET /gpt/v1/models lists the models available through the API now. See Models.

6. Hold credit

• The gateway works out the most the request could cost: a deliberately high estimate of the input tokens, plus every output token the request allows, at the price of the most expensive model it might use.
• max_tokens sets the output allowance. Without it, the gateway uses 4,096 tokens, or the model's own maximum output when that is lower.
• Safeguard's billing service holds that amount against your credit. It checks your balance, then the key's monthly limit, then the workspace's monthly limit. If the hold does not fit, the answer is a 402 that names the reason. See Spend controls.
• If the billing service cannot be reached, the answer is 503 billing_unavailable. The gateway does not serve a request it cannot meter.

7. Call the provider

• The gateway sends your messages and the supported parameters to the provider, using Safeguard's credentials for that provider. The user field is not passed on.
• It always sends a token cap: your max_tokens, lowered to the model's maximum output if it is above that, or the default from step 6.
• For models served through Anthropic's API, only temperature, tools and tool_choice are passed on from the optional parameters.
• On a zero data retention request to OpenAI, it also sends store: false.
• It makes one attempt, unless retries and fallbacks are on. If the provider rejects a parameter, the gateway adjusts it and sends the request again to the same provider; see Chat completions. Each attempt has the provider's timeout, 120 seconds by default. See Reliability and fallbacks.
• When a provider fails, the gateway writes its own error from the provider's HTTP status. The provider's error text is never passed through.

8. Settle the cost

• The cost is the provider's token counts times the model's list price, with no markup, worked out to 8 decimal places. Cached input tokens are priced at the catalog's cached-input price where it lists one, and at the input price otherwise.
• The billing service releases the hold and charges the cost. A request that failed before the provider produced anything is charged $0.
• If the provider reported no usage, the gateway estimates the tokens and adds "estimated": true to the response's usage.
• The response carries usage.cost in USD, and provider, which names the provider the gateway called. Its id is the request's id, which starts with gen-, and the X-Generation-Id header carries the same id, streamed or not. An error after this point, from the provider call, carries that header too.
• For a stream, the gateway reads the provider's answer to the end even if your client disconnects, and settles from the provider's own figures.
• If the settlement cannot be recorded, the gateway tries once more. A hold that is never settled is released after about 30 minutes.

9. Write the log row

Settlement writes one row to your workspace's request log. The row holds metadata only:

• the time (stored in UTC), the source (api or workbench), and the key with its name and masked form as they were at the time;
• the model, the provider and the model's region, plus the original model when a fallback answered;
• the status, the HTTP status, any error code and how many attempts were made;
• input, output and cached input tokens, and whether they were estimated;
• the cost, the latency and, for a stream, the time to the first token;
• whether the response was streamed and whether zero data retention applied.

The row holds no prompt and no output. Prompts and outputs are stored, separately, only if a tenant admin or super admin has turned on Store prompts and outputs. See Privacy and data. The Console shows these rows on Usage. See Usage and analytics.

Workbench runs

The Console's Workbench goes through the same steps, with a few differences:

• There is no key. You are signed in, and the run is charged to your workspace's credit.
• With no key, there is no key limit: only the balance and the workspace limit apply.
• The rate limit is 60 runs a minute for each person.
• Runs are logged with the source workbench.

Related

• Chat completions
• Reliability and fallbacks
• Spend controls
• Privacy and data