trifero.ai
An AI research laboratory under Strife Technologies, focusing on code model training and benchmark evaluation.
Timeline (last 90 days)
A 360M-parameter code model trained by the trifero.ai lab under Strife Technologies initially reported 26.8% pass@1 on HumanEval, and after removing instruction data overlapping with the benchmark, the true number was 22.6%.
The full research report (including charts, confidence intervals, and McNemar tests) is at trifero.ai/research/elroy, and the code along with nine chapters and all benchmark completion and failure records is open-sourced under the Apache-2.0 license at github.com/strifero/elroy.
All Facts (2)
A 360M-parameter code model trained by the trifero.ai lab under Strife Technologies initially reported 26.8% pass@1 on HumanEval, and after removing instruction data overlapping with the benchmark, the true number was 22.6%.
50%UnverifiedThe full research report (including charts, confidence intervals, and McNemar tests) is at trifero.ai/research/elroy, and the code along with nine chapters and all benchmark completion and failure records is open-sourced under the Apache-2.0 license at github.com/strifero/elroy.
50%