Your LLM Is Answering Multiple-Choice Questions at Essay Prices: The Decision Model Shift of Late 2026

What Is a Decision Model, and Why Did Five Vendors Ship One in Two Months?

Look at the AI calls in almost any production system and you will find that most of them are not really asking for prose. Is this support ticket a refund request? Which of six queues should this email go to? Does this agent trace contain an error? Should the agent call a tool or answer directly? Teams send these questions to a general-purpose LLM, ask it to reply in JSON, parse the result and hope the format holds. That works, but it is the equivalent of hiring an essayist to tick boxes.

Decision models are built for the box-ticking. On 15 September 2026 Typesafe founder Diogo Almeida introduced Jev, which the company describes as the first "System One Model": "Think of Jev as a frontier-intelligence function call: unstructured state in, typed probabilistic decisions out." Jev gives up string generation entirely. It samples answers to a fixed schema in parallel instead of token by token, and returns calibrated probabilities and confidence scores rather than text.

The category filled up fast. On 1 October Cloudflare released Clef and Clef-flash as open-source decision models under Apache 2.0, hosted on Workers AI. OpenAI's Decisions API is now generally available on gpt-6-luna. Perplexity offers a Decisions API with 27B and 4B "decider" models, and Liquid AI publishes open-weight d1 decision models as small as 587M parameters. O'Reilly's Radar Trends to Watch for November 2026 lists OpenAI's and Cloudflare's launches among the month's notable AI models.

The interfaces have converged on a shared shape. You send shared evidence, called input or state, plus a list of named questions. Each question is a yes/no predicate (some vendors call it a noul), a choice among fixed options, or a score on an ordered scale. You get back a probability per option, a confidence value and, for scores, a probability-weighted result. No reply is written, no JSON has to be repaired, and output tokens are not billed.

How Much Faster and Cheaper Are They, Really?

The vendor numbers are dramatic, and each one needs reading alongside its caveats. OpenAI says its Decisions API "returns typed answers about 10x faster than the Responses API", with uncached input at $0.10 per million tokens and no charge for output. Perplexity lists $0.02 per million input tokens for its 27B decider and $0.01 for the Flash variant. Typesafe quotes $0.042 per million input tokens and end-to-end responses of 70 to 500 milliseconds.

Typesafe is unusually candid about the limits of its own figures. The launch post admits that its speed evals are "generally run from our laptops on the West Coast", that its LLM comparison numbers come from OpenRouter with "almost certainly" some bias, and that its type-safety figure "is not empirical". Cloudflare's own measurements put Jev's median latency at 524.1 ms, against 209.3 ms for Clef and 38.8 ms for Clef-flash. On a real internal task, classifying domains for Cloudflare's threat intelligence team, Clef took 2.2 seconds to fetch, render and classify a domain compared with 4.7 seconds for gpt-oss-120b.

The most useful independent comparison so far comes from Randal Olson, who analysed the Artificial Analysis Decision Index of 11 decision models and 15 LLM configurations. On the tasks where the two classes tied, decision models answered in 0.06 to 0.54 seconds while LLMs took 1.1 to 17 seconds, and the median decision model was five to six times cheaper than the median LLM. The cost ranges overlap, so a cheap LLM can still beat an expensive decision model on price.

Latency is often the bigger prize. A 200 ms decision can sit inside a request path: routing a chat message, gating an agent's tool call, deciding which UI component to render. A two-second LLM call usually has to be pushed into a background job or hidden behind a spinner. When a decision becomes cheap enough to run on every request, products get designed differently.

Where Decision Models Win, and Where They Clearly Lose

Olson started with the suspicion that decision models are cheap because they are worse, and found that was "half right". On safety classification and tool-call routing, the median decision model came within two points of the median LLM, the best decision model beat every LLM on safety classification, and on tool-call routing only Claude Opus 5.5 beat the best decision models, at 50 or more times the cost per task.

The other half is the warning. On knowledge questions the median LLM scored about 30 points higher, and on hard reasoning about 19 points higher. Those two tasks account for nearly all of the gap in the blended score. His rule is simple: "Use an LLM when the right answer depends on facts the input doesn't contain or on multi-step reasoning." Liquid AI's documentation is just as explicit, advising teams to "use a language model instead for free-form text generation, creative writing, multi-turn conversation, complex multi-step reasoning", open-ended Q&A, summarisation and code generation.

The pattern makes sense once you see how these models work. Cloudflare explains that Clef runs a prefill-only pass and then scores the valid schema choices in parallel, and that "the decision step is non-autoregressive, so there's no intermediate text to generate token by token". No intermediate text also means no room to reason. If the evidence for the answer is in the input, reading it once is enough. If the answer needs world knowledge or a chain of inference, the model has nowhere to do that work.

One vendor claim needs particular care. Typesafe writes that Jev "can't hallucinate" because it is limited to structured outputs. The narrow version is true: it cannot invent a label outside your schema or return malformed JSON. It can still pick the wrong label with high confidence, and Olson reports that decision models were overconfident on the knowledge and reasoning tasks. A well-formed wrong answer is still a wrong answer. It is just harder to spot.

The Hidden Multiple-Choice Questions in Your AI Stack

The practical opportunity is not to replace your LLM. It is to find the many small decisions around it that never needed one. In a typical AI product built over the last two years, those include intent detection on incoming messages, routing between models or agents, moderation and guardrail checks, deciding whether an agent should call a tool, priority scoring of tickets, extraction confidence checks, and the LLM-as-judge evaluations that grade every other output.

Liquid AI's documentation lists almost exactly this set as the fit for decision models: classification, routing, scoring, triage, moderation, guardrails, LLM-as-judge replacement, tool-call approval and model cascades. Olson's shortlist is routing, triage, moderation, intent detection and tool-or-no-tool calls, "especially when the input holds everything you need to decide."

These calls are where AI budgets quietly leak. Each one is small, but they run on every request, often several times per request in agentic workflows, and they are usually sent to whatever general model the team set up first. We have written before about how API costs became an engineering line item and about moving narrow tasks to small language models. Decision models extend the same logic. For a fixed-choice question, you no longer even pay for output tokens.

The first step is an inventory, not a migration. List every model call in the system, write down what question it is actually answering, and sort them into three groups: open-ended generation, which stays on an LLM; fixed-choice decisions where the evidence is in the input, which are candidates; and fixed-choice decisions that depend on outside knowledge, which stay on an LLM or need retrieval first. Teams doing this for the first time are often surprised at how large the middle group is.

The Probabilities Are the Feature. They Are Also the Trap.

Probability outputs are what make decision models more useful than a JSON-mode LLM call. You can set thresholds, send low-confidence cases to a human or a bigger model, and trade precision against recall on purpose. That only works if the probabilities mean what they say, and every vendor stops short of guaranteeing it.

OpenAI calls its predicate value "the model's estimate that the condition is true" and recommends setting thresholds from labelled examples. Liquid AI says to "calibrate any decision threshold on data from your own use". Perplexity notes that identical requests occasionally differ in the second decimal place and advises setting thresholds with some margin. Olson's summary is shorter: "Check the probabilities before you trust them."

The clearest evidence of why comes from Hamel Husain and Antaripa Saha, who tested six decision models on a real eval task: flagging failed traces from an apartment-leasing AI agent, using 98 scored traces of which 61 were known failures. OpenAI's Decisions API separated the classes best, with an F1 of 0.87 at 3.72 cents per 100 eval runs, while Perplexity's decider reached 0.79 at 0.82 cents. Calibration varied widely. In one Clef Flash probability bucket the model predicted about an 8% failure rate, yet 60% of the traces in it were failures.

They also flagged the most common mistake: "We've also noticed people using the default decision threshold of 0.50 without checking whether it works well for their task." A threshold is a product decision about the cost of false positives versus false negatives, and it cannot be chosen without labelled data from your own traffic. Teams that skip this step get a fast, cheap classifier with an unknown error rate.

Designing the Cascade: Decision Model in Front, LLM Behind

Olson does not recommend choosing one or the other. He recommends a decision model in front for fixed-choice calls and an LLM behind it for knowledge or reasoning. In practice that is a cascade. The decision model handles the high-volume, high-confidence cases in a few hundred milliseconds. Anything below a confidence threshold, or anything outside the question types it handles well, escalates to an LLM or to a person.

The details matter more than the diagram. Perplexity's documentation warns that "grouping can hurt when the questions are independent candidates competing for the same judgement", and that answering questions together lowered accuracy on its 4B model, so the Flash variant answers each question separately and bills the shared input once per question. The advice: "If you're unsure, send one candidate per request, since a wrong grouping costs more than a missed one." Latency also grows with input size. Perplexity measured about 5 seconds at roughly 90,000 tokens and 14 seconds at about 190,000, so a decision model fed an entire document is not automatically fast.

Question design becomes a new engineering skill. A choice question needs mutually exclusive, exhaustive options, usually with an explicit "other" or "insufficient information" escape hatch. A score question needs levels whose meaning is stable enough that two humans would agree. This is close to the discipline we described in our piece on the eval gap, and it is no coincidence. A good decision schema and a good eval rubric are the same artefact.

Finally, keep the decision layer swappable. The API shapes are similar but not identical: OpenAI uses predicate, choice and score, Perplexity and Cloudflare use noul, choice and score, and Liquid's open-weight models add typed answers such as numbers, dates and bounding boxes. A thin internal interface, with your own question schema mapped to each vendor, costs little and protects you in a category that did not exist three months ago.

A 30-Day Adoption Plan for Engineering Teams

Week one is the inventory described above. Log every model call for a representative period, tag it by the question it answers, and estimate its volume and cost. Rank the fixed-choice candidates by volume multiplied by current cost, and pick one high-volume decision that a wrong answer would not turn into an incident. Ticket routing and eval grading are common first choices.

Week two is data. Pull a few hundred real examples for that decision, label them with whoever owns the outcome, and keep a held-out set. Hamel Husain's study used 98 traces with five-fold cross-validation, which shows that a useful first benchmark does not need thousands of rows. It does need honest labels from production rather than synthetic examples.

Week three is the bake-off. Run the current LLM call and two or three decision models against the labelled set. Measure F1 or the metric that matches your cost of error, Brier score or a reliability plot for calibration, p50 and p95 latency from your own region, and cost per thousand decisions. Choose thresholds on the training folds and confirm them on the held-out set. Make sure the decision model is not quietly handling cases that need outside knowledge. Those are the ones the benchmarks say it will get confidently wrong.

Week four is a shadow deployment. Run the decision model alongside the existing call in production, compare outcomes, route disagreements to review, and only then switch traffic with an escalation path for low-confidence answers. Add the decision to your monitoring like any other model, with drift alerts on the probability distribution. Inputs change, and a threshold tuned in November will not necessarily hold in March.

Why This Is a Team Capability Question, Not a Model Question

The decision model shift rewards a specific mix of skills: someone who can read a production system and find its hidden decisions, someone who can design schemas and labelling guidelines, and someone who can run a calibrated evaluation and turn it into thresholds and alerts. That is applied ML and evaluation engineering, not prompt engineering, and many product teams do not have it in-house.

It is also exactly the kind of bounded, measurable work that suits a dedicated development team working alongside your own engineers. At Stepto, our engineers in Serbia work within European and US business hours, so a decision inventory, labelling sprint and bake-off can run in the same working day as your product owners, not across a twelve-hour handoff. Our AI developers build the evaluation harness first, because a cheap model with an unmeasured error rate is a liability, not a saving.

The commercial case is easy to model. If a system makes millions of fixed-choice calls a month on a general LLM, moving the confident majority to a decision model can cut both the bill and the latency that users notice, while the LLM keeps the work it is actually good at. The savings depend on doing the inventory, labelling and threshold work properly. That work is where nearshore development earns its keep: senior engineers close enough in time and culture to sit with the people who know what the right answer is.