Held-out Batch Design
An experimental protocol for evaluating model out-of-distribution generalization in which some experimental batches or data sources are entirely excluded during training and used only for testing on the held-out batches, in contrast to random train/test splits.
Timeline (last 90 days)
The core evaluation idea of BatchBench is to let participating models encounter a portion of experimental batches during training, and then test them on completely unseen batches and data sources (leave-batch-out design).
The leave-batch-out design is an experimental protocol for evaluating a model's out-of-distribution generalization ability: certain experimental batches or data sources are completely excluded from the training set, the model is trained only on the remaining batches, and then tested on the held-out batches.
All Facts (2)
The core evaluation idea of BatchBench is to let participating models encounter a portion of experimental batches during training, and then test them on completely unseen batches and data sources (leave-batch-out design).
50%UnverifiedThe leave-batch-out design is an experimental protocol for evaluating a model's out-of-distribution generalization ability: certain experimental batches or data sources are completely excluded from the training set, the model is trained only on the remaining batches, and then tested on the held-out batches.
50%