What it is
A CI failure triage bot for GitHub Actions.
What it does: On each failed workflow run it trims failed-job logs, strips secrets, asks Jev five typed questions in one call, and lets code policy decide whether to re-run once, comment on the PR, or label the run for a person. Optional weekly reports list tests that fail often without a related change.
Why it belongs in the showcase: A clear confidence-gated ops loop with a published head-to-head against regex rules and a small LLM on real CI logs.
Caveats: Author-labelled eval; re-runs are policy-gated.
Evidence
Public MIT repo. Author eval on 200 labelled CI failures (196 real failed jobs from public GitHub Actions of 61 MIT/Apache-2.0 repos, plus 4 invented excerpts): with the default policy Jev re-ran 1 real failure by mistake out of 18 re-runs versus 17 of 38 for regex rules; category accuracy 69.5% (macro-F1 0.692) versus 40.0% for regex; on a 74-failure subset shared with Gemini Flash-Lite, Jev category accuracy 83.8% versus 82.4% for the LLM and 44.6% for regex, at about 0.47 s median versus 1.73 s for the LLM. Full report in eval/results.md.
Caveats
- Labels and the evaluation set are author-maintained; flaky versus real test failure is hard from one log alone
- Automatic re-runs are opt-in and capped; uncertain cases stay labelled for a person
- Optional LLM fix suggestions are folded away and marked as suggestions
Unofficial case page. Stars / list position observed for heat; do not treat heat as a quality SLA. Review the upstream README before you depend on it.