MAE
Masked Autoencoders (MAE) is a self-supervised visual learning method based on masked reconstruction. Models are trained by masking portions of an image and learning to predict the concealed content.
Timeline (last 90 days)
MAE (Masked Autoencoders) replaces the bottleneck constraint with a high masking ratio to force the model to learn semantic representations, continuing the design philosophy of Context Encoders.
This plug-and-play framework does not alter the structure or loss function of the SSL backbone network, but only replaces the sampling module, and can be integrated into existing SSL pipelines such as MAE, SimCLR, and DINO.
When the quantile level τ=0.5, quantile loss degenerates to Mean Absolute Error (MAE).
In recommendation scenarios, models typically use Root Mean Square Error (RMSE) to measure the error between predicted ratings and actual ratings.
Compared to the historical mean baseline, entry-level and turn-level predictions reduced mean absolute error (MAE) by 4.03% and 3.92%, respectively.
Geospatial foundation models typically use the Transformer architecture as their core, employing self-supervised learning methods for pre-training on large volumes of remote sensing imagery, with typical methods including masked autoencoders (MAE) and contrastive learning.
The MAE proposed by He et al. in 2021 splits images into patches, randomly masks up to 75% of the patches, encodes the visible patches with a Vision Transformer, and reconstructs the masked portions.
The masked context augmentation strategy draws inspiration from masked language modeling in NLP (such as BERT) and masked autoencoders (MAE) in computer vision.
By constructing supervisory signals from the data itself, self-supervised learning has become the core training strategy for large-scale pre-trained models such as BERT, GPT, and MAE.
The choice of pretraining (ImageNet-1K vs ImageNet-21K, supervised vs self-supervised like MAE/DINO) significantly impacts downstream performance, potentially causing several percentage points of difference
All Facts (10)
The choice of pretraining (ImageNet-1K vs ImageNet-21K, supervised vs self-supervised like MAE/DINO) significantly impacts downstream performance, potentially causing several percentage points of difference
80%UnverifiedMAE (Masked Autoencoders) replaces the bottleneck constraint with a high masking ratio to force the model to learn semantic representations, continuing the design philosophy of Context Encoders.
50%UnverifiedThis plug-and-play framework does not alter the structure or loss function of the SSL backbone network, but only replaces the sampling module, and can be integrated into existing SSL pipelines such as MAE, SimCLR, and DINO.
50%UnverifiedWhen the quantile level τ=0.5, quantile loss degenerates to Mean Absolute Error (MAE).
50%UnverifiedIn recommendation scenarios, models typically use Root Mean Square Error (RMSE) to measure the error between predicted ratings and actual ratings.
50%UnverifiedCompared to the historical mean baseline, entry-level and turn-level predictions reduced mean absolute error (MAE) by 4.03% and 3.92%, respectively.
50%UnverifiedGeospatial foundation models typically use the Transformer architecture as their core, employing self-supervised learning methods for pre-training on large volumes of remote sensing imagery, with typical methods including masked autoencoders (MAE) and contrastive learning.
50%UnverifiedThe MAE proposed by He et al. in 2021 splits images into patches, randomly masks up to 75% of the patches, encodes the visible patches with a Vision Transformer, and reconstructs the masked portions.
50%UnverifiedThe masked context augmentation strategy draws inspiration from masked language modeling in NLP (such as BERT) and masked autoencoders (MAE) in computer vision.
50%UnverifiedBy constructing supervisory signals from the data itself, self-supervised learning has become the core training strategy for large-scale pre-trained models such as BERT, GPT, and MAE.
50%