Module Coverage
A measure of the contextual diversity with which each module appears in the training set; it is another key evaluation property affecting compositional generalization performance.
Timeline (last 90 days)
Module coverage measures the diversity of contexts in which each module appears in the training set.
Although step-by-step models train intermediate outputs and provide denser supervision signals, they still cannot fully offset the negative effects of module coverage shift.
If a benchmark contains both module coverage violations and compositional equivalence, shortcut learning is exacerbated.
Compositional generalization benchmark design should avoid test sets that contain compositional equivalence, control module coverage shift, and consider both factors simultaneously.
Evaluating compositional generalization is not merely a training-test split problem; ignoring compositional equivalence and module coverage may lead to erroneous judgments about model capabilities.
All Facts (5)
Module coverage measures the diversity of contexts in which each module appears in the training set.
50%UnverifiedAlthough step-by-step models train intermediate outputs and provide denser supervision signals, they still cannot fully offset the negative effects of module coverage shift.
50%UnverifiedIf a benchmark contains both module coverage violations and compositional equivalence, shortcut learning is exacerbated.
50%UnverifiedCompositional generalization benchmark design should avoid test sets that contain compositional equivalence, control module coverage shift, and consider both factors simultaneously.
50%UnverifiedEvaluating compositional generalization is not merely a training-test split problem; ignoring compositional equivalence and module coverage may lead to erroneous judgments about model capabilities.
50%