XLM-RoBERTa
A multilingual pretrained language model released by Meta, based on the RoBERTa architecture and supporting 100 languages. Its tokenizer is often used as a multilingual baseline.
Timeline (last 90 days)
At the 64k vocabulary setting, the new tokenizer averaged 1.147 tokens per word, beating the best existing tokenizer's 1.208 and XLM-RoBERTa's 1.512.
XLM-RoBERTa is a multilingual pre-trained language model released by Facebook AI, with the large version having about 560 million parameters; BGE-M3 is a multilingual embedding model released by the Beijing Academy of Artificial Intelligence with about 570 million parameters; multilingual-e5-large-instruct is a multilingual instruction-tuned version of Microsoft's E5 series with about 560 million parameters.
All Facts (2)
At the 64k vocabulary setting, the new tokenizer averaged 1.147 tokens per word, beating the best existing tokenizer's 1.208 and XLM-RoBERTa's 1.512.
50%UnverifiedXLM-RoBERTa is a multilingual pre-trained language model released by Facebook AI, with the large version having about 560 million parameters; BGE-M3 is a multilingual embedding model released by the Beijing Academy of Artificial Intelligence with about 570 million parameters; multilingual-e5-large-instruct is a multilingual instruction-tuned version of Microsoft's E5 series with about 560 million parameters.
50%