Routing heat 1 · Quiet★ 0heat incomplete (no X)

jev-agent-router

Benchmark of three support-message routers behind one typed interface: keywords, Claude Haiku, and Jev. Author reports 98% vs 99% accuracy on 111 hard messages, with Jev about 5x faster at the median.

What it is

Routes customer messages to billing, bug, account or sales three ways (a keyword baseline, Claude Haiku, and Jev) on the same labeled set, all returning the same typed Route.

What it does: Reports accuracy by slice (easy, no-keyword, misleading-keyword, multi-intent, terse, noisy, ambiguous) plus p50 / p95 latency. Author reports keywords 42%, Jev 98% at 116 ms median, and Haiku 99% at 607 ms median on the 111 non-ambiguous messages.

Why it belongs in the showcase: An honest head-to-head for ticket routing that writes down what it does not show (cost, real traffic, concurrency).

Caveats: Author-labeled synthetic messages; numbers are author-reported; cost not measured.

Evidence

Public MIT README with methodology, per-slice results and confusion matrices; 120-message set (9 ambiguous messages reported separately), 3 runs on 2026-10-05, Pydantic AI with the same instruction and the same typed Route output.

Caveats

  • Author-written messages and labels (one annotator), not real ticket traffic; the 3 runs repeat the same 111 messages
  • Cost not measured; one machine, sequential calls, latency includes the network round trip
  • Using Jev confidence to escalate unsure messages to the LLM is not tested in the repo

Unofficial case page. Stars / list position observed for heat; do not treat heat as a quality SLA. Review the upstream README before you depend on it.