Routing heat 1 · Quiet★ 0heat incomplete (no X)

Jev Inference Router

Observable decision layer for local GPU inference: Jev chooses the handler (local Qwen, document retrieval, or clarification), Python applies the policy, and local Qwen writes the answer.

What it is

Decision layer in front of a local LLM: TypeSafe Jev chooses where each request goes, Python applies the policy, and a local Qwen model generates the answer.

What it does: Routes to local Qwen, project-document retrieval, or clarification; a deterministic allowlist answers configuration questions without model calls; the browser chat shows probabilities, policy decisions, sources, timing and cost. The confidence policy can abstain, which it did on two documentation questions at 0.59 and 0.64.

Why it belongs in the showcase: Handler-split routing with a visible abstain policy and published raw results, next to a local-model baseline.

Caveats: Small synthetic suite; author-reported; live mode needs your own GPU setup.

Evidence

Public MIT README with CI; Choice, Noul and Score batched in one call; author-measured on 24 labeled synthetic questions (6 October 2026): Jev plus policy 22/24 at 387 ms p50, local Qwen JSON classification 21/24, keyword rules 22/24.

Caveats

  • 24 synthetic questions only; README says it does not establish robust calibration or superiority over rules
  • Qwen generates a JSON label while Jev returns distributions, so the latency comparison is not a universal speed claim
  • Live mode needs a TypeSafe key plus a local Qwen engine (Apple Silicon via vllm-metal); the offline demo uses fixtures

Unofficial case page. Stars / list position observed for heat; do not treat heat as a quality SLA. Review the upstream README before you depend on it.