Routing

Send an ordered list of models, prefer or exclude providers, or let the gateway order them by price, latency or uptime for each request.

Early access, off by default

Routing is switched on per workspace by Safeguard, in addition to the API Platform itself. Where it is off, a request that uses any routing field is refused with 400 unsupported_parameter and the field named in param. The gateway never ignores a routing field silently.

Routing works on chat completions and completions. It does not apply to embeddings, because vectors from different models cannot be mixed, or to your own models (custom:<id>).

Fields

{
  "model": "anthropic/claude-sonnet-5-5",
  "models": ["openai/gpt-6-sol", "google/gemini-3.5-flash"],
  "provider": {
    "order": ["anthropic", "openai"],
    "only": ["anthropic", "openai", "google"],
    "ignore": ["groq"],
    "sort": "price",
    "allow_fallbacks": true,
    "zdr": false
  },
  "messages": [{ "role": "user", "content": "Hello" }]
}

Field	Type	Meaning
`models`	array of model ids	Candidates to try after `model`, in your order. Any provider or family. You may leave `model` out, and the first entry is used. By default up to 5 extra models; the platform limit is shown in the error if you send more.
`provider.order`	array of provider routes	Routes to prefer, in this order, among the candidates.
`provider.only`	array of provider routes	Keep only candidates on these routes.
`provider.ignore`	array of provider routes	Drop candidates on these routes.
`provider.sort`	`price`, `latency` or `uptime`	Sort the candidates automatically. See Sorting.
`provider.allow_fallbacks`	boolean	As before: whether the gateway may retry and move on when a provider fails. See Reliability and fallbacks.
`provider.zdr`	boolean	As before: only models offered with zero data retention.

A provider route is the API the gateway calls for a model, such as openai, anthropic, google, mistral, deepseek, xai, cohere, together, fireworks or groq. The route is not always the model's publisher: an open-weight model can be hosted on several routes.

How the candidates are built

model, then each entry of models, in your order.
After each of them, the same model on other routes (the same weights, another host), when you set sort, order or only, or when fallbacks are allowed.
When fallbacks are allowed, the model's listed fallbacks of the same family, as before.

Then only and ignore filter by route, and every candidate must be one the API Platform can serve for your workspace (and offer zero data retention when you asked for it). If nothing is left, the answer is 503 model_unavailable. A model id in models that does not exist is 404 model_not_found with param: "models".

Sorting

The candidates are ordered by sort (or the platform default, which is none unless Safeguard sets one), then by order, then in the order above. Ties keep their order.

`sort`	First	Measured how
`price`	the smallest estimated cost for this request	your input tokens, estimated, times the input price, plus your `max_tokens` (or the model's output limit) times the output price, at each provider's list price with no markup
`latency`	the fastest	the median end-to-end latency of successful requests through the gateway over the last 24 hours, across all workspaces
`uptime`	the most reliable	the share of requests over the last 24 hours that did not fail at the provider (provider errors, timeouts, refused credentials and provider rate limits)

A model with no price, or with too few recent requests to measure, sorts last. Latency and uptime are measured only from real traffic through the gateway; nothing is estimated or copied from elsewhere, so a new model has no figure until it has been used.

What happens when a provider fails

The gateway calls the first candidate. If the provider fails before anything is streamed (a timeout, a connection failure, a provider error or a provider rate limit), it moves to the next candidate when:

fallbacks are allowed (provider.allow_fallbacks, or the workspace setting), or
you listed models yourself and did not set allow_fallbacks: false.

Retrying the same model once happens only when fallbacks are allowed. Once text has been streamed, the gateway never switches models in the middle of an answer.

Which model answered

The response's model field is the model that actually answered, and the X-Route response header is the provider route that served it. Each request is billed at the list price of the model and route that served it, with no markup. The request log records the route and the sort that chose it.

Example

curl https://api.safeguard.sh/gpt/v1/chat/completions \
  -H "Authorization: Bearer $SAFEGUARD_INFERENCE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "models": ["openai/gpt-6-sol", "anthropic/claude-sonnet-5-5"],
    "provider": { "sort": "uptime" },
    "messages": [{ "role": "user", "content": "Summarise SBOMs in one sentence." }]
  }' -i

The Console Workbench builds these requests for you: pick fallback models and a sort, and copy the code it shows.