Can Jev Replace Hand-Written Routing Rules? (Mo Al Mamun)
External experiment by Mo Al Mamun on DEV (2026-10-17): rules, Jev, an LLM and a rules+Jev hybrid route 3,080 Banking77 messages to 77 backend actions, with accuracy, latency, cost and referral trade-offs.
What this guide points to
Mo Al Mamun published Can a Decision Model Replace Hand-Written Routing Rules? A 3,080-Message Experiment on DEV (2026-10-17). He built switchless, a fictional bank backend with 77 actions, and swapped four dispatchers in front of the same handlers: ordered keyword rules, jev-1.13.0, an LLM with structured output, and a hybrid that accepts rules+Jev agreement or Jev at confidence 0.9 or more.
Reported results on the Banking77 test set: exact-action accuracy 65.5% for rules, 80.9% for Jev and 81.5% for the LLM; median decision time 111 ms for Jev and 975 ms for the LLM. The hybrid answered 81.7% of messages at 90.7% accuracy and referred the rest. The author notes the LLM was cheaper in this workload and that correct selection is not the same as safe execution.
This page is an unofficial field manual pointer — verify claims against the article, the switchless repo and current TypeSafe docs. The article is marked AI-assisted by its author.
Takeaways
- Give Jev the full action list with one-line descriptions and measure exact-action accuracy.
- Use agreement plus a confidence floor to trade coverage for fewer wrong automatic picks.
- Add separate checks before any state-changing handler runs.