Auto Routing

Send every conversation to the right model automatically. A decision model reads the first message and picks the route.

Auto Routing picks the right model for each conversation for you. You describe the kinds of requests you get in plain words, attach a model to each one, and a decision model reads the first message of every new conversation and picks the matching route. Easy requests go to a fast, cheap model. Hard ones go to your strongest model.

Up to 43% lower cost

On coding tasks in early internal Requesty benchmarks, compared with sending everything to the strongest model.

Up to 30% lower latency

Simple requests are answered by faster models, so most conversations finish sooner.

It works with the decision model of your choice and with any kind of decision: size of a code change, support triage, reasoning depth, language, topic, risk, or anything else you can describe in a sentence.

Create an Auto policy in the Requesty Console. Auto Routing is in beta.

An Auto policy: one classifier, one question, a model per route and a fallback.

How It Works

1. Your app calls the policy

   Set model to policy/<your-policy-name>, exactly like any other routing policy. Nothing else in your code changes.

2. The classifier reads the first user message

   On the first request of a conversation, Requesty sends only the first user message to your classifier, the decision model you picked. Earlier system prompts, later turns and tool results are not sent.

3. It scores every route

   The classifier gets one multiple-choice question: your question, with each route label as an option. It returns a probability for every label, so you see how sure it is, not just one answer.

4. The best route wins, or the fallback does

   The label with the highest probability wins if it reaches your confidence threshold, and the request goes to that route’s model. If no label is sure enough, or the classifier returns an error, the request goes to the fallback model instead. Your request is never blocked by the classifier.

5. The choice is pinned for the session

   The chosen model is remembered for the rest of the conversation, so later turns skip the classifier and the model never switches halfway through a task. Prompt caches keep working because every turn hits the same model.

The classifier runs once per conversation, not once per request, so it adds a single short call (typically well under a second) at the start of a session and nothing after that.

Benefits

Pay for hard work only

Routine requests stop paying top-tier prices. Only the requests that need your best model get it.

Any decision you can describe

Routes are plain-language labels. No training data, no embeddings, no code.

Your choice of classifier

Pick the decision model that fits your quality, speed and region needs, and compare them side by side before you ship.

Safe by default

Low confidence or a classifier error always lands on your fallback model.

Choosing a Classifier

The classifier is any decision model your organization can use. They are built to return calibrated probabilities over a fixed set of options, which is exactly what routing needs.

Classifier	Notes
typesafe/jev-1.14.0	Jev by Typesafe. Our strongest router on coding prompts in internal tests
typesafe/jev-1.13.0	Previous Jev release
cloudflare/clef	Clef by Cloudflare. Fast, also reads images
cloudflare/clef-flash	Smaller, faster Clef
sference/clef	Clef served by Sference
azure/decision-1	Microsoft Decision-1 on Azure, available in EU regions

The classifier picker only lists models that are approved for your organization and allowed by your compliance rules. If a classifier you want is missing, approve it in the Model Library. Live availability is in GET /v1/models/decision.

Not sure which one to pick? Use Try it (below) to run the same messages through several classifiers at once and see where they agree.

Creating an Auto Policy

1. Open Auto policies

   Go to Routing Policies > Auto and click New. Only organization admins can create or edit policies.

2. Start from a template, or from scratch

   Templates give you a working question and routes to edit:

   Template	What it does
   Coding: size of the change	Small, local edits go to a fast model; changes that must stay consistent across many places go to the strongest one
   Support triage	Quick account questions get a cheap model; bugs and escalations get a stronger one
   Reasoning depth	Chit-chat and lookups stay fast; multi-step reasoning, math and planning escalate

3. Pick the classifier and write the question

   The question tells the classifier what to look at. Give it context about your users or codebase and end with the actual question, for example “What kind of change is this?”.

4. Add routes

   Each route is a label and a target. The label is the description the classifier matches against, so write it as a short, specific category. The target can be any model, or one of your fallback, load balancing or latency policies, so the chosen route still gets failover and provider selection. Auto policies cannot target other Auto policies.

5. Set the fallback and confidence threshold

   The fallback model handles anything the classifier is not sure about. A threshold of 50% is a good start: if the best label is below it, the fallback is used. Raise it to send more borderline requests to the fallback.

6. Save and use it

   Save the policy and set model to policy/<name> in your requests.

Try It Before You Ship

Every Auto policy has a Try it panel. Type a first message, pick one or more classifiers, and run it. Only the classifiers are called, never the target models, so testing costs a fraction of a cent.

Six classifiers compared on the same message. All six agree on the refactor route, with the spread of each classifier's confidence on every label.

• Consensus shows which route most classifiers picked, or the fallback if most of them would fall back.
• Spread plots every classifier’s probability on every route, with your threshold as a dashed line, so you can see how close the call was.
• Region lets you run the test on the Global router or on your organization’s regional router (for example EU), the same way your traffic is served.

Use the Policy in Your Code

Python

from openai import OpenAI

client = OpenAI(
    base_url="https://router.requesty.ai/v1",
    api_key="your-requesty-api-key"
)

response = client.chat.completions.create(
    model="policy/claude-auto-coder",
    messages=[{"role": "user", "content": "Add a created_at field to the Invoice struct"}],
    extra_body={"requesty": {"trace_id": "session-8f2c"}}
)
print(response.model)  # the model the route picked

Auto policies work on the Chat Completions, Messages and Responses APIs, with streaming, tools and every other feature of the target model.

Keeping a conversation on one model

The choice is pinned for 60 minutes per conversation. Requesty recognises a conversation by its trace_id if you send one, otherwise by the conversation itself. Send a stable trace_id per session (as above) to make sure every turn of the same task stays on the same model. A new trace_id starts a new decision.

Example: Coding Agent Router

This is the policy in the screenshot above, used internally for coding agents working on a Go backend.

Route label	Model
additive: new field, constant, helper, adapter or endpoint following an existing pattern	anthropic/claude-haiku-5-5
changes existing behaviour in one place	anthropic/claude-sonnet-5-5
changes existing behaviour in several places (handlers, directions, streaming paths) that must all match	anthropic/claude-opus-5-5
rename, move or refactor with no behaviour change	anthropic/claude-opus-5-5
Fallback (below 50%)	anthropic/claude-opus-5-5

The question describes the codebase in two sentences, asks the classifier to infer what the agent will have to touch, and ends with a tie-break rule: “If the request lists several existing places that must all change, prefer the last label.”

Writing Good Routes

Describe the request, not the model

Labels should describe the kind of request (“simple account or billing question with a known answer”), never the model. The classifier never sees model names; routes attach them afterwards.

Keep labels distinct

Two labels that overlap split the probability between them, which pushes requests to the fallback. If two routes go to the same model anyway, consider merging them.

Put your safest model on the fallback

Anything the classifier is unsure about lands there, so it should be the model you trust on the hardest requests.

Give context, then ask

A sentence or two about your product or codebase helps. Very long instructions or worked examples tend to overfit and route worse on real traffic.

Test with real first messages

Paste real first messages from your logs into Try it, including any system reminders your client adds, since that is exactly what the classifier sees.

Good to Know

• First message only. The decision is made on the first user message of the conversation. If your client prepends long context to that message (some coding agents add project instructions), that text is part of what the classifier reads.
• Text only for most classifiers. Images in the first message are only read by cloudflare/clef and cloudflare/clef-flash.
• Billing. Each decision is billed as a normal request to the classifier, once per conversation, and shows up in your logs and analytics next to the routed request.
• Approved models. Every target model must be approved for your organization, like in any routing policy.

FAQ

How is this different from latency routing or load balancing?

Those policies choose between models that are interchangeable. Auto Routing chooses between models of different capability and price, based on what the request needs. You can combine them: point a route at a latency or fallback policy.

What happens if the classifier is down?

The request goes to the fallback model. The classifier never blocks a request.

Can a conversation switch models later?

No. The model is fixed for the session so answers stay consistent and prompt caching keeps working. Start a new session (or a new trace_id) to get a new decision.

Where do the savings numbers come from?

Early internal Requesty benchmarks on real coding tasks, comparing an Auto policy against sending every task to the strongest model. Your results depend on your traffic and routes; use Try it and your analytics to measure them.