Trek-Robust
A smart home instruction evaluation dataset extended from HomeBench. It evaluates model robustness to environmental variation by perturbing room names, device aliases, and environment descriptions.
Timeline (last 90 days)
The Trail team extended HomeBench into three datasets: Trek-Base (testing language diversity), Trek-Robust (evaluating robustness to environment changes), and Trek-Memory (evaluating multi-turn dialogue memory capability).
On the Trek-Robust dataset, Trail achieves 85.3% accuracy, while the fine-tuned Qwen3.5-4B baseline is 69.8%.
All Facts (2)
On the Trek-Robust dataset, Trail achieves 85.3% accuracy, while the fine-tuned Qwen3.5-4B baseline is 69.8%.
50%UnverifiedThe Trail team extended HomeBench into three datasets: Trek-Base (testing language diversity), Trek-Robust (evaluating robustness to environment changes), and Trek-Memory (evaluating multi-turn dialogue memory capability).
50%