Hamel Husain
An AI expert focused on LLM evaluation and machine learning engineering practice.
Timeline (last 90 days)
Hamel Husain's LLM judge guide recommends switching to binary pass/fail judgments accompanied by written comments, with the scoring criteria defined jointly by domain experts.
Hamel Husain recommends designating one lead domain expert as the single source of truth, making a binary pass/fail judgment on each output, and attaching written comments.
Hamel Husain recommends starting with about 30 examples, continuously expanding until no new failure modes appear, then accumulating about 100 labeled examples per failure mode to validate the evaluator.
All Facts (3)
Hamel Husain's LLM judge guide recommends switching to binary pass/fail judgments accompanied by written comments, with the scoring criteria defined jointly by domain experts.
50%UnverifiedHamel Husain recommends designating one lead domain expert as the single source of truth, making a binary pass/fail judgment on each output, and attaching written comments.
50%UnverifiedHamel Husain recommends starting with about 30 examples, continuously expanding until no new failure modes appear, then accumulating about 100 labeled examples per failure mode to validate the evaluator.
50%