Model cascades save money only below a hard escalation limit

The reason is simple: DeepSeek’s output tokens cost far more, so a discarded first answer can erase the saving.

What is model cascade routing?

Model cascade routing means trying one model first, then calling another only when the first result fails a check. A 2026 survey of multi-model systems draws a useful boundary: routing chooses one model before generation, while cascading makes its choice after seeing an initial answer. That second step can protect quality, but it means every escalated request pays for two calls. (arxiv.org)

Tokens are the chunks of text a model reads and bills by. Input tokens are your prompt, instructions and attached context. Output tokens are the answer. This distinction drives the result here: the lower-priced first model has higher input pricing than DeepSeek V4.1 Flash 1017, but much lower output pricing.

A cascade is not proof that one model is better. The LLM Routing with Dueling Feedback paper frames the actual task as choosing models while balancing user satisfaction, expertise and inference cost. A cost table can tell you when a cascade is affordable. Only an evaluation on your own tasks can tell you whether its accepted answers are good enough. (arxiv.org)

Which list prices decide this cascade?

OpenRouter’s gpt-oss-120b page lists $0.014 per million input tokens and $0.052 per million output tokens, with a 131K-token context window. Its DeepSeek V4.1 Flash 1017 page lists $0.006 input and $0.50 output per million, with a 1.0M-token context window. These are list prices shown on 8 November 2026, not a promise that either price will hold. (openrouter.ai)

That output-price gap matters more than the input-price gap for a normal answer. DeepSeek V4.1 Flash 1017 costs less to read a prompt, but its output rate is roughly 9.6 times gpt-oss-120b’s listed output rate. A cascade that generates a full first answer, then throws it away, pays the first model’s output bill every time it escalates.

When does a first model actually save money?

The break-even point is not a vague “most requests must be easy” rule. It is a number: divide the first-call cost by the second-model-only cost, then subtract that result from one.

Longer prompts make the threshold harsher.

The first-attempt and DeepSeek-only figures use the published token rates above. The escalation limit is the point where the two-call average equals a DeepSeek-only request.

Which option costs least at 50% escalation?

At a 50% escalation rate, the cascade costs less for all three profiles, but barely so for retrieval and agent work. By our calculation, short chat saves 34.4%, while the RAG and agent profiles save 3.7% and 5.6%. That is the practical answer: do not treat a low model rate as a universal reason to cascade. (openrouter.ai)

“Verified vendor list price” means OpenRouter’s displayed rate on 8 November 2026. “Official prices + FuturPulse calculation” means arithmetic using those rates and the stated token assumptions. It is not a benchmark result or a quality claim.

Why do output tokens change the answer?

Short prompts give the cascade room to work because the first model’s bill is only $0.040 per 1,000 requests in this example. A longer RAG prompt raises that first bill to $0.138. The direct DeepSeek price rises much less because its input-token price is lower. (openrouter.ai)

For teams building coding tools, judge the cascade on working results rather than polished prose. The HumanEval research paper made functional correctness the central measure for generated code. A first model that produces fluent but broken code should escalate, even if its own confidence score says otherwise.

Task specialisation also matters. The Qwen technical report describes separate code- and mathematics-focused models alongside general models. The buyer lesson is broader than any one vendor: price comparisons work only after the models have passed the task-specific bar you set.

What validator makes a cascade safe?

The validator is the real product decision. It decides which first answers reach users and which receive a second attempt. The survey says cascades usually rely on response-level signals such as confidence, token probabilities or a separate verifier, rather than the prompt alone. (arxiv.org)

Start with checks that match the job. For structured extraction, reject invalid JSON or missing fields. For code, run tests. For retrieval answers, check that required citations appear and that the answer addresses the question. For support, use human-review samples and downstream outcomes such as repeat contacts.

• Use a hard rule first: invalid format, failed test or missing required field means escalate.
• Add uncertainty second: send low-confidence answers to the next model.
• Measure false accepts: an accepted weak answer is more damaging than an unnecessary escalation.
• Price the validator: a costly judge model belongs in the same cost spreadsheet.

A real-world example shows why calibration matters. Hightouch’s identity-resolution experiment found that a Jev-plus-Opus setup reached 98.8% accuracy at $0.88 per 1,000 cases, against 99.0% at $3.86 for Opus alone in its easier test. That result is specific to its data and thresholding, but it demonstrates the important mechanism: the cheap model’s uncertainty signal must identify cases worth escalating. (hightouch.com)

Should you route first or cascade first?

Choose predictive routing when the prompt itself reveals difficulty. A short classification, an extraction with a fixed schema, or a known low-risk FAQ can go directly to the appropriate model. That avoids paying for a discarded answer on hard requests.

Choose a cascade when the first output reveals whether the task is hard. A failed unit test, malformed response or missing evidence is concrete information that a prompt-only router lacks. The practical distinction is straightforward: routing is cheaper when difficulty is predictable before generation, while cascading is safer when the answer itself supplies the signal. (khimananda.com)

Most production systems should combine the two. Route obviously difficult work straight to the model that has met your quality target. Cascade the uncertain middle. Send routine, validated work to the lowest-cost model. The research survey describes such compositions as normal rather than exceptional. (arxiv.org)

What should you actually do?

Start with the chat profile only if your product has short prompts and a cheap pass/fail check. At the published prices, the gpt-oss-120b-first cascade has a wide 84.4% escalation ceiling there. Do not copy that rule into a RAG or agent pipeline, where the safe ceiling is close to the mid-50% range.

1. Sample 500 to 1,000 real requests and label the answer quality your users need.
2. Run both models on that fixed sample, then build a simple validator around actual failures.
3. Record the escalation rate separately for chat, RAG and agent workloads.
4. Keep the cascade only where its observed escalation rate stays below the table’s break-even limit.
5. Recalculate whenever either vendor changes token rates or your prompt length changes.

Do not buy a cascade because a headline promises a multiple of savings. Buy it when your own validator accepts enough first answers and your quality review shows that those accepted answers remain safe. The cost advantage here is arithmetic. The quality advantage must be earned on your workload.

What we could not verify?

No public evidence establishes that gpt-oss-120b is consistently weaker than DeepSeek V4.1 Flash 1017 for the three request profiles used here. Neither vendor page supplies a matched quality comparison for short chat, RAG answers or multi-step agents.

No public price page settles provider availability, routing fees, retry rates, cache treatment or latency for a particular account. OpenRouter and the underlying model providers could settle commercial terms, while a buyer’s own test set can settle quality, escalation and user-impact questions.

Sources

• OpenRouter: OpenAI gpt-oss-120b
• OpenRouter: DeepSeek V4.1 Flash 1017
• Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey
• LLM Routing with Dueling Feedback
• Hightouch: LLM Judges for Identity Resolution at 4x Lower Cost
• LLM Model Routing and Cascading Explained
• Evaluating Large Language Models Trained on Code
• Qwen Technical Report