Self-supervised Learning
A machine learning paradigm that requires no manually annotated labels, instead constructing supervisory signals from the inherent structure of the data itself. It is the training methodology employed by models such as DINOv2.
Core Facts
大语言模型的预训练阶段本质上属于自监督学习，是一种特殊的非监督学习，模型通过预测下一个词来从海量文本中自动学习语言规律
In-context learning was first observed in large language models, and its capability fundamentally stems from the diversity of pretraining data.
Foundation models learn by predicting the 'next word' on massive amounts of unlabeled text using a self-supervised learning paradigm, represented by models such as the GPT series and LLaMA
Timeline (last 90 days)
Common methods in academia and industry for addressing few-shot defect detection include anomaly detection (such as autoencoder reconstruction error methods), data augmentation and synthetic defect generation, transfer learning, and semi-supervised/self-supervised learning.
Using 'future trajectories' as labels is based on the assumption that the vehicle's current driving path is safely traversable, which holds in the scenario of following existing wheel tracks but may introduce noisy labels when breaking new ground.
The study adopts a self-supervised approach that automatically generates training labels for wheel tracks by back-projecting the vehicle's future trajectory onto the current image frame using GNSS data.
Unsupervised or self-supervised learning paradigms are particularly critical for studying non-human communication that lacks parallel translations.
The mainstream approaches to depth estimation fall into two main categories: geometry-based multi-view stereo (MVS) and deep-learning-based monocular depth estimation.
This approach is essentially a self-supervised learning application that uses the generation process itself to produce perfectly aligned supervision signals.
Self-supervised TTT work can be traced back to around 2020, enabling models to adapt to the current input by constructing auxiliary self-supervised tasks (such as rotation prediction) on test samples.
Current mainstream large language models and AI systems are based on a training approach that combines self-supervised learning on massive datasets with reinforcement learning
By constructing supervisory signals from the data itself, self-supervised learning has become the core training strategy for large-scale pre-trained models such as BERT, GPT, and MAE.
All Facts (12)
大语言模型的预训练阶段本质上属于自监督学习，是一种特殊的非监督学习，模型通过预测下一个词来从海量文本中自动学习语言规律
80%VerifiedIn-context learning was first observed in large language models, and its capability fundamentally stems from the diversity of pretraining data.
75%VerifiedFoundation models learn by predicting the 'next word' on massive amounts of unlabeled text using a self-supervised learning paradigm, represented by models such as the GPT series and LLaMA
65%UnverifiedCommon methods in academia and industry for addressing few-shot defect detection include anomaly detection (such as autoencoder reconstruction error methods), data augmentation and synthetic defect generation, transfer learning, and semi-supervised/self-supervised learning.
50%UnverifiedUsing 'future trajectories' as labels is based on the assumption that the vehicle's current driving path is safely traversable, which holds in the scenario of following existing wheel tracks but may introduce noisy labels when breaking new ground.
50%UnverifiedThe study adopts a self-supervised approach that automatically generates training labels for wheel tracks by back-projecting the vehicle's future trajectory onto the current image frame using GNSS data.
50%UnverifiedUnsupervised or self-supervised learning paradigms are particularly critical for studying non-human communication that lacks parallel translations.
50%UnverifiedThe mainstream approaches to depth estimation fall into two main categories: geometry-based multi-view stereo (MVS) and deep-learning-based monocular depth estimation.
50%UnverifiedThis approach is essentially a self-supervised learning application that uses the generation process itself to produce perfectly aligned supervision signals.
50%UnverifiedSelf-supervised TTT work can be traced back to around 2020, enabling models to adapt to the current input by constructing auxiliary self-supervised tasks (such as rotation prediction) on test samples.
50%UnverifiedCurrent mainstream large language models and AI systems are based on a training approach that combines self-supervised learning on massive datasets with reinforcement learning
50%UnverifiedBy constructing supervisory signals from the data itself, self-supervised learning has become the core training strategy for large-scale pre-trained models such as BERT, GPT, and MAE.
50%