cev

Calibrated Enum Verdicts: typed, calibrated decisions from a small local model in one forward pass. You ask is this true, which of these or which level about any state, a ticket, a tool call, a screenshot or a voicemail, and get back an answer with its probabilities in about a tenth of a second. The model never generates text, so an answer outside your options can't happen. One Rust engine, in-process from Rust, Node and Python.

01 Three languages, one answer 02 Debiasing fixes the first-option habit 03 A screenshot, decided in 98 ms 04 21 labels, and it learned

The idea

Most of what a program asks a language model is not "write something". It is a decision: which team gets this ticket, is this tool call safe, how urgent is this voicemail. Generating text for that and parsing it back is slow and can fail in ways a match statement can't. cev asks the decision directly.

A request is a state plus named questions of three types:

• noul asks "is this true?" and returns the probability of yes.
• choice asks "which of these?" and returns an option and a distribution over all of them, up to 255.
• score asks "which level?" and returns the expected level and its distribution.

The answer comes from the model's own next-token logits, restricted to the one-token codes of the options (A, B, C…), so zero-shot works on any Qwen3 or Qwen3.5 checkpoint on day one. The wire format is compatible with POST /v1/systemone, the API of TypeSafe's Jev and System One models, so existing Jev SDKs work against it.

One forward pass

The state is read once and its prefix cached across requests. Every question, and every debiasing variant of it, is packed into the same pass under a block mask, with positions restarting per question: each one sees the state and itself, which is exactly equivalent to asking it alone. On an M4 Max, three questions about a short ticket take about 118 ms on Qwen3-1.7B, warm, through any of the three packages.

In f32, cev matches Hugging Face transformers to within 5×10⁻⁵ in any probability, on the CPU and on Metal, and each decision model below matches a PyTorch reference built from its published parts to about 10⁻⁵.

Calibrated

Small instruct models have a strong habit of picking whichever option is listed first. Zero-shot Qwen3-1.7B picks "A" on 28 of 30 severity tickets. cev corrects for it at inference time, in the same pass:

• calibrate subtracts what the model answers to the same question over content-free evidence. Those logits don't depend on the state, so they are cached and free after the first request.
• permute averages over rotations of the option order.
• full, both, is the default for cev's own readout. On the repository's small zero-shot tasks it lifts severity accuracy from 0.40 to 0.80 and "is the user blocked?" from 0.70 to 0.97.

Rust, TypeScript and Python

The engine is Rust, on candle, with Metal on a Mac, Accelerate or all the cores on a CPU, and CUDA if you build for it. Each package runs it inside your process: no server, no HTTP, no PyTorch.

• Rust: cargo add cev-rs. #[derive(cev::Choice)] on an enum turns a decision into a match; doc comments become the options' descriptions. Talks to the engine in-process or to a cev server over HTTP.
• Node: npm install cev-node, a native addon prebuilt for Apple silicon and Linux. Answers are typed from the questions you pass, so answers.team.choice is 'billing' | 'tech'. It never blocks the event loop.
• Python: pip install cev-py, a native module prebuilt for Apple silicon and Linux, one wheel for every Python from 3.11. Its shapes are TypedDicts, and it releases the GIL while it works.
• Server: cargo install cev-server --features metal gives the cev binary, with REST, GraphQL and a browser page.

The same ticket asked through all three returns the same probabilities to the sixth decimal place, because it is the same engine.

The decision models

Beyond its own zero-shot readout, cev runs the open decision models natively, with their own prompts and readout heads, weights pulled from Hugging Face: Bespoke Nimble, Jared Palmer's Kev, the vLLM Semantic Router's d3, and Liquid's d1 and d1-omni. Their image and audio inputs work too: d1-omni, at 600M parameters, reads a screenshot in about 100 ms and a nine-second voicemail in under a second.

Learns while it runs

Every answer is logged with a decision_id, its state, the exact prompt and its logits. Send the right label back by id and a small per-task adapter refits on the spot, learning a temperature, label priors and a linear probe on the backbone's own representation of the question. It is served only once it beats the base model prequentially, on labels it had not yet seen, so a handful of noisy labels can't make it worse. Namespaces keep one customer's labels from changing another's answers.

For bigger shifts, /v1/export hands every labelled decision, with the prompt exactly as served, to a PyTorch LoRA trainer, and cev serves the result.

In the browser

rust-cev ships a page that runs the same pipeline compiled to WebAssembly in a web worker, so nothing leaves the tab, or on the local cev server that serves it. It decides about live microphone, camera and screen streams too.

Stated honestly

• The repository's accuracy numbers come from small built-in task sets, tens of tickets each: tests of the method, not benchmarks.
• Long states are slow on the hybrid models: Qwen3.5's Gated DeltaNet prefill is plain candle ops for now. A fused Metal kernel, quantized weights and Qwen3-MoE are next.
• There is no "insufficient evidence" option yet; a model forced to choose will choose.

About these pictures

Everything here was measured on one M4 Max. The tour's three programs are in this site's repository, built against cev-rs from crates.io, cev-node from npm and cev-py from PyPI, and the build stops if their answers differ. The screenshots are rust-cev's own browser page on the cev server, running d1-omni on the examples it ships with.