Masked Language Modeling
A training objective used in pretrained language models in which some tokens in the input text are randomly masked and the model predicts the masked content from context. It is the core training method of models such as BERT.
Timeline (last 90 days)
The core technical path of current AI decipherment of ancient scripts is sequence completion and masked language modeling, derived from pretrained models such as BERT in natural language processing.
Pre-trained language models such as BERT learn to capture fine-grained contextual word meanings through masked language modeling tasks.
All Facts (2)
The core technical path of current AI decipherment of ancient scripts is sequence completion and masked language modeling, derived from pretrained models such as BERT in natural language processing.
50%UnverifiedPre-trained language models such as BERT learn to capture fine-grained contextual word meanings through masked language modeling tasks.
50%