Irene Solaiman
An AI safety and policy researcher at Hugging Face whose work focuses on research and advocacy concerning AI evaluation methods, standardization of safety testing, and external audit mechanisms.
Timeline (last 90 days)
Hugging Face chief policy officer Irene Solaiman pointed out that AI evaluation methods are quite complex and still maturing, and the industry still lacks consensus on "what counts as a good evaluation."
Solaiman says the AI evaluation field as a whole is currently at an early stage, with substantial room for subjective judgment in benchmark design, dataset construction and the choice of scoring criteria.
Hugging Face's Irene Solaiman notes that the industry is shifting from simple model-level benchmarks to agentic system evaluations, but testing protocols lack consensus, making results hard to reproduce.
Solaiman stresses that the direction of auditing needs to change: examining only the AI model itself is no longer enough, and evaluation is shifting toward the entire system, including the tools and software that support its operation.
Solaiman's position is that the industry needs more standardization, which is a necessary step for the evaluation landscape to move from fragmentation to interoperability.
All Facts (5)
Hugging Face chief policy officer Irene Solaiman pointed out that AI evaluation methods are quite complex and still maturing, and the industry still lacks consensus on "what counts as a good evaluation."
50%UnverifiedSolaiman says the AI evaluation field as a whole is currently at an early stage, with substantial room for subjective judgment in benchmark design, dataset construction and the choice of scoring criteria.
50%UnverifiedHugging Face's Irene Solaiman notes that the industry is shifting from simple model-level benchmarks to agentic system evaluations, but testing protocols lack consensus, making results hard to reproduce.
50%UnverifiedSolaiman stresses that the direction of auditing needs to change: examining only the AI model itself is no longer enough, and evaluation is shifting toward the entire system, including the tools and software that support its operation.
50%UnverifiedSolaiman's position is that the industry needs more standardization, which is a necessary step for the evaluation landscape to move from fragmentation to interoperability.
50%