Benchmark
HomeBench
An evaluation benchmark for smart home instruction understanding that assesses a model's ability to match household environment descriptions with user instructions. It can be extended to cover dimensions such as linguistic diversity, robustness, and multi-turn memory.
Timeline (last 90 days)
Nov 3
The Trail team extended HomeBench into three datasets: Trek-Base (testing language diversity), Trek-Robust (evaluating robustness to environment changes), and Trek-Memory (evaluating multi-turn dialogue memory capability).
Unverified50%