The Router Can Now Run on the Model

Most agent stacks still treat routing as a small function around the real model. A classifier chooses a provider, a policy checks a tool call, and the expensive language model does the work.

Decision 3.0 points at a different arrangement.

The vLLM Semantic Router team has released an open family of multimodal decision models ranging from 0.6B to 27B parameters. The models are designed to answer typed decisions, including choices, yes or no questions, and scores, rather than generate a long response. The collection is available on Hugging Face, and vLLM announced the weights as open sourced in this post.

The release is not just another open model drop. It makes the control plane itself replaceable.

An agent does not need a language model for every decision

A router often has to answer questions such as:

• Is this request safe to send to the browser worker?
• Does this pull request belong to the security or infrastructure queue?
• Is the screenshot showing the state needed for the next action?
• Should this input stay local, or can it go to a remote provider?
• Which specialist should receive the task?

Those are decisions, not essays. Generating a paragraph and extracting a label from it is an awkward way to solve them. It consumes more tokens, creates more room for formatting failures, and makes the verifier parse natural language before it can enforce a policy.

The Decision 3.0 model card describes a different interface. The model reads text, JSON, and, for the larger multimodal variants, images and video. It returns a typed choice with probabilities. That output is much closer to what an agent runtime actually needs:

observation
    ↓
local decision model
    ↓
choice + probability
    ↓
policy gate or specialist route
    ↓
action

The important boundary is between the choice and the action. A probability is not permission. The runtime still needs thresholds, fallback behavior, approval rules, and an audit record. But a typed decision is easier to constrain than a free form completion.

The collection is a routing ladder

The release is a family, not one monolithic checkpoint. The public collection lists six sizes:

Model	Approximate size	Likely operator role
d3-edge	0.6B	Cheap, narrow local filters
d3-lite	0.9B	High volume text decisions
d3-nano	2B	Local routing with more capacity
d3-mini	5B	Multimodal decisions on constrained hardware
d3-flash	8B	Higher quality general routing
d3	26B	Complex multimodal control decisions

The size ladder matters because routing is called more often than the main task. A worker model may run once. A gate can run before every tool call, every retry, every handoff, and every transition from untrusted content into durable state.

That changes the economics. The best router is not necessarily the most capable model. It is the model that is cheap enough to sit in the loop, reliable enough to reject the dangerous edge cases, and observable enough that an operator can understand why it chose a route.

Local decisions change the trust boundary

The vLLM Semantic Router team has also described Core ML conversions for the three smaller models. Its post reports local text, image, and video decisions on Apple Silicon, with no per call network round trip. It reports a median JevBench latency of 53 milliseconds for d3-lite, 69 milliseconds for d3-nano, and 169 milliseconds for d3-mini in its single machine measurements.

Those are the team’s measurements, not an independent benchmark. They still expose the systems consequence clearly.

A local router can inspect a pull request, screenshot, or document without sending the raw input to the same remote service that executes the next action. The decision can happen beside the data. The larger worker can receive only the selected task, or a redacted representation of it.

That is useful for privacy, but privacy is not the only gain. Local routing also reduces the number of network transitions in the control loop. Fewer transitions mean fewer places for credentials, prompt context, and untrusted content to cross a boundary.

The uncomfortable truth is that many agent systems have one giant trust domain because routing was treated as plumbing. Once routing becomes an explicit model, its placement becomes an architectural decision.

What the release does not solve

A decision model is still a model. It can be manipulated by the input it is asked to classify. A malicious instruction in a document may cause the router to label a dangerous action as routine. A screenshot may contain an adversarial visual cue. A high probability can be confidently wrong.

The model also does not know the full policy unless the runtime supplies it. A classifier that answers “send” cannot see whether the destination is approved, whether the token is scoped, or whether the action is reversible. Those facts belong in an out of band policy layer.

The safe design is therefore not:

model probability = authorization

It is:

model probability
    + authenticated task identity
    + capability policy
    + resource scope
    + approval requirement
    + audit event
    = executable decision

The smaller model can make the first classification. The runtime must own the final authority.

That distinction also changes evaluation. A router should not be measured only by aggregate classification accuracy. Operators need to know:

• How often does it abstain on unfamiliar inputs?
• Does it fail closed when the policy context is missing?
• Can an attacker move a request across a capability boundary through an indirect prompt?
• Are probability thresholds calibrated on the deployment distribution?
• Does the same decision remain stable when irrelevant content is added?
• Can every action be traced back to the observation and policy state that produced it?

A fast wrong router is still a fast way to make the wrong tool call.

The control plane is becoming a model tier

Open weights make the router auditable and deployable in a way a hosted classifier is not. Operators can pin a version, inspect the serving path, run it in the same network domain as sensitive data, and route only selected tasks to a larger model.

That gives agent builders a new composition pattern:

1. Keep a small decision model close to the event stream.
2. Use typed outputs for routing, triage, filtering, and approval recommendations.
3. Treat the output as a proposal, never as authority.
4. Enforce capabilities in infrastructure outside the model context.
5. Escalate ambiguous or high impact cases to a stronger model or a human.
6. Log the observation, decision, threshold, policy version, and action together.

This is model routing in the literal sense, but it is also policy engineering. The model no longer sits only behind the tool boundary. It helps decide which boundaries the rest of the system will approach.

Decision 3.0 is interesting because it makes that layer visible and open. The future agent stack may not have one model that thinks and a pile of hard coded rules around it. It may have a set of local decision models that continuously classify state, choose workers, and determine when the loop must stop.

The router is not plumbing anymore. It is part of the runtime, and it deserves the same isolation, testing, and auditability as the model doing the work.

Sources

• Decision 3.0 collection, vLLM Semantic Router on Hugging Face, accessed October 12, 2026. Lists the six open model sizes and the collection description.
• Decision 3.0 on Apple Silicon, vLLM Semantic Router, October 2026. Source for the Core ML deployment description, latency measurements, and stated evaluation limits.
• vLLM announcement of the Decision 3.0 release, October 11, 2026. Confirms the release announcement and open weights claim.
• Decision 3.0 announcement by Xunzhuo, October 11, 2026. Release post linked by the vLLM announcement and source for the team’s state of the art claim.