Jev 101
Start Guides
Search
中文 EN
Retrieval heat 2 · Cool★ 0heat incomplete (no X)

Jev RAG Chinese Benchmark

A reproducible Chinese reranking benchmark for Jev on C-MTEB/T2Reranking with hard negatives: pinned model ids, cost caps, full per-call logs, and a 2026-10-19 spot check of Jev 1.14 against 1.13.

Repo: vicfei/jev-rag-zh-benchmark ↗

What it is

Rare, reproducible numbers for Jev on Chinese retrieval.

What it does: Measures Jev as a reranker after BM25, with logs and cost caps.

Why it belongs in the showcase: Shows the 1.13 to 1.14 change for non-English work without retuning guesswork.

Caveats: Small samples; single runs.

Evidence

Public MIT repo created 2026-10-12, updated 2026-10-19, with a Zenodo DOI (10.5281/zenodo.23322277) and HuggingFace/ModelScope mirrors. README reports S1 (n=100) Jev Score nDCG@5 0.546 vs BM25 0.413, and the 1.14 re-run: nDCG@5 0.5464 vs 0.5461 on 1.13 (paired delta +0.0004, 95% CI −0.037 to +0.038) with p50 latency 0.47 s → 0.66 s; 1,000 calls cost $0.037.

Caveats

  • Single run per variant on 100 queries
  • Latency change measured on one day and one channel
  • Independent community project; calls went through OpenRouter

Unofficial case page. Stars / list position observed for heat; do not treat heat as a quality SLA. Review the upstream README before you depend on it.