ModelRoBERTa base
RoBERTa-base
An improved pretrained language model over BERT with approximately 125 million parameters and 12 layers, commonly used as a benchmark for evaluating fine-tuning methods.
Timeline (last 90 days)
Nov 4
The paper evaluates on the GLUE benchmark with RoBERTa-base, following the La-LoRA/LoRA protocol.
Unverified50%
Nov 4
Current validation of LIRA is concentrated on medium-scale settings such as RoBERTa-base and GLUE, and its performance on larger models and more task types remains to be seen.
Unverified50%