Retrieval heat 2 · Cool★ 0heat incomplete (no X)

Jev job-matching experiment

An independent experiment where Jev scored 400 real Kaggle job postings against 30 fictional résumés (12,000 evaluations), with a top-3 check, a "none" cutoff and a frank list of seniority mistakes.

What it is

A careful, reproducible look at ranking with a decision model.

What it does: Scores each posting against every candidate in one call and checks the ranking against an answer key.

Why it belongs in the showcase: Clear numbers, cheap to rerun, and honest failure analysis.

Caveats: Not for real hiring decisions; AI-labelled key.

Evidence

Public repo created 2026-10-19; code is MIT, data has a separate licence file. README reports 400 postings (0 failed), about 197 s, 3,988,299 input tokens, about US$0.17, top 3 vs answer key 96/100, top 1 with the 1.5 cutoff 45/100, and "none" detected 14/14. The answer key was labelled by Claude and spot-checked by the author. 16 of 26 top-1 misses were seniority mix-ups. `python avaliar.py` recomputes the numbers without API calls.

Caveats

  • Experiment only, not a hiring tool; keep people in the hiring decision
  • Answer key labelled by an AI model and only spot-checked
  • All candidates are fictional; file names and some notes are in Portuguese

Unofficial case page. Stars / list position observed for heat; do not treat heat as a quality SLA. Review the upstream README before you depend on it.