3.6 Model routing and automatic fallback

Model routing decides which model a request is actually sent, and what it steps down to when that one cannot answer. It lives at Model routing in the sidebar.

The reason it exists: an agent picks its own model names. Claude Code asks for claude-sonnet-4-5 for your turn and claude-haiku-4-5 for its background summaries, whatever it is configured with; Codex Desktop remembers the model chosen in its own state and ignores the config file. Point either at a provider that never heard of those names and the model in the request is not a name that provider serves.

What happens with no rules at allCopy Anchor Link

Nothing to configure — this is the default, and for most setups it is enough.

• A model a provider does serve is asked for as written.

• A model it does not serve is answered with the closest model of the same size class that provider offers, read off the name: haiku, mini, flash, lite, air and friends are small; opus, pro, max, reasoner, thinking are large; everything else is in between. A request for claude-haiku-4-5 on a DeepSeek provider lands on deepseek-chat, not on deepseek-reasoner.

  That is the automatic part: background traffic stays on the cheap model without anyone writing a rule.

• If several providers serve the model, they are tried in priority order, and one inside its failure cooldown goes last.

What a rule addsCopy Anchor Link

Write a rule when you want to say where a model goes rather than let the name decide.

Each rung names a model, a provider, or both:

• A model only — any enabled provider that serves it may answer.
• A provider only — that provider answers, asked for the model the client sent.
• Both — that provider, asked for that model.

The ladder is the fallbackCopy Anchor Link

The first rung is where the request goes. Every rung below it is what the request steps down to when the rung above could not answer: a rate limit, an exhausted quota, a rejected key, a 5xx, a timeout, or a model that provider does not have. Up to eight rungs.

A rule that reads

1. glm-4.5-air @ Zhipu
2. deepseek-chat

sends the covered requests to Zhipu's small model, and when Zhipu is rate-limited sends the same request to whichever enabled provider serves deepseek-chat.

Below the last rung, Gateway still falls back to what it would have done with no rule at all, so a rule whose providers have all been deleted or disabled leaves the request working rather than dead.

The client is answered in the model name it asked for, whichever model actually served the turn.

Which rule winsCopy Anchor Link

Rules are read in order: the Sort number first, then the more specific pattern (an exact model name beats claude-*, which beats *), then a rule tied to an agent before a general one. The first rule whose pattern matches decides; only one rule applies to a request.

Routing is not permissionCopy Anchor Link

A rule says where a request goes. It never refuses one — what an agent may ask for is set on the Permissions page, and the ladder is filtered through that agent's permissions before it is walked. A downgrade can never reach a model or a provider the agent is not allowed.

Seeing where a request would goCopy Anchor Link

Where would this go? on the same page answers with the whole plan for a model name, without sending a request: every step in the order it would be tried, which rule chose it, and which providers are in a cooldown right now. The rungs below the first are exactly the part nobody sees until an upstream is already failing.

After the fact, LLM Records shows the same thing for a request that really happened: Routed to is the model that actually went on the wire when it differed from the one asked for, Routed by names the rule, and the attempts the request failed over from are listed with the model each of them asked for. The cost is priced against the model that was really billed.

Next: 4.1 Permissions.