MASSIVE
A multilingual intent recognition benchmark dataset released by Amazon, covering 51 languages, 18 domains, and 60 intent categories. It is widely used to evaluate the performance of multilingual natural language understanding models.
Timeline (last 90 days)
Aplomb 1 在 JevBench Hard 上得分 77.5%，在 MASSIVE 数据集 51 种语言上平均 75% 零样本意图识别准确率
Laya achieves only 0.45 accuracy on the MASSIVE dataset covering 13 languages and only 0.425 on Banking77, while JAF reports 0.87 on the same tasks.
After fine-tuning, Laya's accuracy on the MASSIVE dataset can be improved from 0.45 to 0.85, and on another task from 0.362 to 0.766.
All Facts (3)
Aplomb 1 在 JevBench Hard 上得分 77.5%，在 MASSIVE 数据集 51 种语言上平均 75% 零样本意图识别准确率
50%UnverifiedLaya achieves only 0.45 accuracy on the MASSIVE dataset covering 13 languages and only 0.425 on Banking77, while JAF reports 0.87 on the same tasks.
50%UnverifiedAfter fine-tuning, Laya's accuracy on the MASSIVE dataset can be improved from 0.45 to 0.85, and on another task from 0.362 to 0.766.
50%