DishwasherBench
A benchmark for evaluating the spatial reasoning ability of frontier AI models, requiring models to determine the correct arrangement of pots, pans, and dishes in a dishwasher. It involves an optimization problem under three-dimensional spatial constraints.
Timeline (last 90 days)
DishwasherBench is an AI benchmark that recently appeared on Hacker News, used to have frontier models judge how a dishwasher should be loaded.
DishwasherBench requires models to perform three-dimensional spatial reasoning through visual input and common sense, involving constraints such as pot placement, bowl orientation, cup tilt angle, cutlery basket zoning, and object occlusion.
If a benchmark can only determine 'whether it matches the reference solution,' then what it measures is not spatial reasoning ability but alignment.
The core question DishwasherBench needs to answer next is whether the scoring system can remain stable, that is, how to handle the situation where dishwasher loading has no single correct answer.
All Facts (4)
DishwasherBench requires models to perform three-dimensional spatial reasoning through visual input and common sense, involving constraints such as pot placement, bowl orientation, cup tilt angle, cutlery basket zoning, and object occlusion.
50%UnverifiedIf a benchmark can only determine 'whether it matches the reference solution,' then what it measures is not spatial reasoning ability but alignment.
50%UnverifiedDishwasherBench is an AI benchmark that recently appeared on Hacker News, used to have frontier models judge how a dishwasher should be loaded.
50%UnverifiedThe core question DishwasherBench needs to answer next is whether the scoring system can remain stable, that is, how to handle the situation where dishwasher loading has no single correct answer.
50%