What it is
Multi-vendor decision-model benchmark documentation.
What it does: Publishes reproducible suites and analysis that compare how decision models apply supplied rules, how their probabilities behave, and what the decisions cost across hosted and local configurations.
Why it belongs in the showcase: One of the few public, multi-wave head-to-heads that includes TypeSafe Jev beside Microsoft, OpenAI Decisions and open/local alternatives, with explicit methodology limits.
Caveats: No license file; author-run; some local costs unmeasured.
Evidence
Public repo with documented protocols and published report tables. The latest Decision 3.0 flagship/Edge comparison reports flagship at 167/168 compact (99.4%) and 501/504 across all v2 conditions, versus Edge at 102/168 compact (60.7%) and 287/504 overall. Earlier waves cover hosted versus local setups, Perplexity 27B, and paired English/PT-BR expansions. The README states it is not an official vendor benchmark or a claim that one model is universally best.
Caveats
- No license file in the repo at intake
- Author-run study; not an independent third-party lab evaluation
- Local operating cost for some arms is unmeasured; hosted cost comparisons are report-specific
Unofficial case page. Stars / list position observed for heat; do not treat heat as a quality SLA. Review the upstream README before you depend on it.