Models and Reasoning
On Desktop, each service tier runs on an ordered chain of providers. When a provider can't respond, the step moves on to the next provider in the chain without you having to do anything.
How is the chain ordered?
- A model is first served directly by its maker.
- If the maker can't respond, the next provider in the chain is tried.
- Providers with shared capacity are only the last fallback.
When a provider is full or unreachable
If a provider is full, busy, down, or unreachable and no part of the response has been sent yet, the step automatically moves to the next provider.
Provider is full
The first provider is full, so the request moves to the next provider before any part of the response has been sent.
- Model makerFull
- Other providerDone
- 3Shared capacityLast fallback
Answered by Other provider
I summarized the three decisions in the meeting notes: the budget was approved, the SMB pilot is starting, and the launch moved to November.
The 30-second rule
If a response does not start within 30 seconds, the provider is considered unreachable and the next provider is tried.
No response in 30 seconds
The first provider does not start responding within 30 seconds, so it is considered unreachable and the next provider is tried.
- Model makerNo response
- Other providerDone
- 3Shared capacityLast fallback
Answered by Other provider
The report's main finding is clear: most new users come back a second time within the first week, and retention depends most on finishing the first task.
Long thinking
Once part of a response has been sent, the provider is not changed. After a response has started, models that think for a long time get a 3-minute silence allowance.
Long thinking
Once a response has started, the provider does not change; a model that thinks for a long time gets a 3-minute silence allowance.
- Model makerDone
- 2Other provider2nd in line
- 3Shared capacityLast fallback
Answered by Model maker
Let's approach the plan in three stages. First we clean the existing data and gather it in one table, then we model it by segment, and finally we track the results with a weekly report.
At the end of the chain
PROVIDER_BUSY: All providers in the chain are currently full; try again shortly.PROVIDER_RESPONSE_TIMEOUT: The model sent no data for 3 minutes (3 attempts were made); try again shortly.- For other error codes, see Error references.