AfriNORM
A scenario-based multiple-choice question dataset benchmark for measuring and diagnosing the normative reasoning ability and cultural competence of large language models in African sociocultural contexts.
Timeline (last 90 days)
AfriNORM is a benchmark for measuring and diagnosing the limitations of LLM normative reasoning in African sociocultural contexts, taking the form of a scenario-based multiple-choice dataset.
The AfriNORM paper proposes two new metrics: the Universal Commonsense Bias Score (UCBS) and the Distractor Bias Score (DBS).
AfriNORM currently uses only Yoruba as a case study, and the linguistic and cultural diversity of the African continent far exceeds what a single case can cover.
All Facts (3)
AfriNORM is a benchmark for measuring and diagnosing the limitations of LLM normative reasoning in African sociocultural contexts, taking the form of a scenario-based multiple-choice dataset.
50%UnverifiedThe AfriNORM paper proposes two new metrics: the Universal Commonsense Bias Score (UCBS) and the Distractor Bias Score (DBS).
50%UnverifiedAfriNORM currently uses only Yoruba as a case study, and the linguistic and cultural diversity of the African continent far exceeds what a single case can cover.
50%