Fallback Policies

Automatic failover between models for maximum reliability

Fallback Policies automatically retry your requests with different models if one fails, ensuring your application stays reliable even when individual providers have issues.

Set up fallback policies in the Requesty Console.

How It Works

1. Request sent to primary model

   Your request goes to the primary model first.

2. Automatic failover on failure

   If it fails (timeout, rate limit, error, etc.), the router immediately tries the next model in the chain.

3. Transparent response

   Your application receives the successful response without knowing about the failures.

Benefits

Higher Success Rates

No more failed requests due to provider issues.

Zero Downtime

Automatic failover without code changes.

Cost Optimization

Start with cheaper models, fall back to premium ones only when needed.

No Stalled Workflows

Your users never see “model unavailable” errors.

Creating a Fallback Policy

1. Create the Policy

   Go to Routing Policies, click Create Policy, and select Fallback Chain as the policy type.

   
2. Configure Your Fallback Chain

   Set up your models in priority order. For example:

   Priority	Model	Retries
   1st	anthropic/claude-sonnet-4-5	1 retry
   2nd	bedrock/claude-sonnet-4-5@eu-central-1	1 retry

   The router will try each model in order, retrying the configured number of times before moving to the next.

3. Use the Policy in Your Code

   Change your model parameter to reference your policy:

   Python

   from openai import OpenAI
   
   client = OpenAI(
       base_url="https://router.requesty.ai/v1",
       api_key="your-requesty-api-key"
   )
   
   response = client.chat.completions.create(
       model="policy/sonnet",
       messages=[{"role": "user", "content": "Hello!"}]
   )
   

   import OpenAI from 'openai';
   
   const client = new OpenAI({
     baseURL: 'https://router.requesty.ai/v1',
     apiKey: 'your-requesty-api-key'
   });
   
   const response = await client.chat.completions.create({
     model: 'policy/sonnet',
     messages: [{ role: 'user', content: 'Hello!' }]
   });
   

   curl https://router.requesty.ai/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer your-requesty-api-key" \
     -d '{
       "model": "policy/sonnet",
       "messages": [{"role": "user", "content": "Hello!"}]
     }'
   

To find your policy reference, go to Routing Policies, click the copy button next to your policy name, and paste it directly into your model parameter.

Use Cases

Cost-Effective GPT Chain

Start with cheaper models, only use expensive ones if needed:

Priority	Model	Retries
1st	openai/gpt-4o-mini	2 retries
2nd	openai/gpt-4o	1 retry
3rd	openai/gpt-5.2	1 retry

Multi-Provider Reliability

Distribute across providers for maximum uptime:

Priority	Model	Retries
1st	openai/gpt-5.2	1 retry
2nd	anthropic/claude-sonnet-4-5	1 retry
3rd	google/gemini-2.5-pro	1 retry

Regional Failover

Try regional endpoints before falling back to global:

Priority	Model	Retries
1st	bedrock/claude-sonnet-4-5@eu-central-1	2 retries
2nd	anthropic/claude-sonnet-4-5	2 retries

How Retries Work

Each model in the chain can have 0 to 10 retries. The router uses:

Mechanism	Description
Exponential backoff	Wait time increases between retries (500ms → 1s → 2s → 4s)
Jitter	Random variation (±10%) to prevent thundering herd
Immediate failover	On non-retryable errors (invalid request, auth failure)

Make sure all models in your fallback chain support your request parameters (context length, streaming, tool calling, etc.). If a model cannot handle the request, the policy will skip to the next model.

Key Selection (BYOK)

For each model, you can choose which API key to use:

Option	Description
Requesty provided key	Use Requesty’s managed keys (default)
My own key	Use your BYOK credentials
Requesty first, then mine	Fallback to BYOK if Requesty key fails
Mine first, then Requesty	Prefer BYOK, fallback to Requesty

Monitoring and Debugging

1. Open Analytics

   Go to Analytics.

2. Filter by policy

   Filter by your policy name to see which models succeeded, failed, and how often fallback occurred.

FAQ

What happens if all models in the chain fail?

The request returns an error with details about the last model attempted. You will see all the failures in your request logs.

Can I nest policies?

Yes. A fallback policy can reference another policy as one of its fallback options. For example, your second priority could be policy/multi-provider-backup instead of a single model.

Do I get charged for failed attempts?

No. You only pay for successful requests that return tokens. Failed attempts do not incur costs.

How do I update a policy?

Click the edit icon next to your policy in the Routing Policies page. Changes take effect immediately, no code deployment needed.