opensourcellmrouter

An open-source router for language-model traffic, written in Rust.

The pipeline

Every request arrives on an OpenAI-compatible /v1/chat/completions or an Anthropic-compatible /v1/messages endpoint, then runs through classifiers, plugins, the router, the provider, plugins again, and finally response classifiers.

Routing

• Classifiers tag a request — vision, code, nsfw — from its content.
• Rules pick a provider by tag, cost, latency, throughput, model-name prefix, discovered Ollama model, or at random. They are read in order, and the first match wins.
• Plugins may rewrite the request or the response, or force a particular provider.

Tags without touching the body

Response classifiers tag what comes back — a refusal, say — and request and response tags are each surfaced in their own header. The OpenAI or Anthropic response body is never modified, so an existing client cannot tell the difference.

Watching it work

Every exchange is written as JSONL and broadcast live to a dashboard, and a TUI is built in. cargo install opensourcellmrouter, then point any OpenAI-compatible client at the local port.