Clairic and Jev on 100 Return Emails (UnlockJapan)
External write-up by Takuma Yoshida at UnlockJapan (2026-10-26): four layouts that combine Cloudflare Clairic, TypeSafe Jev 1.14 and Claude Haiku 5.5 on 100 return/refund emails, with accuracy, cost and confidence-split results.
What this guide points to
Takuma Yoshida published Clairic and Jev on 100 return emails: four experiments on UnlockJapan (2026-10-26). The task is sorting return and refund emails under a shop policy. Before the policy can run, someone must read what happened and what the customer wants. The experiments combine Cloudflare’s Clairic extraction model, TypeSafe Jev (jev-1.14.0), and Claude Haiku 5.5 in different layouts.
Reported highlights (author’s numbers on this 100-email set):
- A Jev + Python six-step layout (no Clairic) reached about 78% accuracy at roughly 0.3 s per case and 1.5 yen per 100 cases, versus Haiku at about 91%, 3.0 s, and 8.1 yen.
- In that layout, Jev’s decisions matched the policy code on all 100 cases; the misses came from reading the email, not from the typed decision step.
- Splitting by confidence helped: when Jev’s confidence was ≥ 0.8 on a blog-style layout, accuracy on those cases was about 95% (39 of 100). Leaving only those high-confidence cases to Jev and the rest to a person raised overall accuracy from 73% (person only) to about 80% and cut person time from 30.0 to 20.8 minutes (threshold chosen after seeing results).
- Clairic-assisted layouts read at about 68–77%; swapping Clairic reading for Jev+Python kept accuracy similar (77% → 78%) at about a sixth of the cost (8.8 → 1.5 yen per 100).
This page is an unofficial handbook entry — verify claims against the original post and TypeSafe docs. Results are task- and dataset-specific.
Takeaways
- Separate reading messy email text from deciding under a fixed policy; measure each stage.
- Prefer a confidence split: auto-handle only high-confidence Jev cases, escalate the rest.
- A typed decision model can be faster and cheaper than a general LLM on the decision slice even when overall accuracy trails.