wav2vec 2.0
A self-supervised speech representation learning model proposed by Meta (Facebook AI Research). It uses contrastive learning to pre-train on unlabeled audio data, substantially reducing speech recognition systems' dependence on labeled training data.
Timeline (last 90 days)
Dual-task vector merging is a parameter-efficient method that adapts a frozen wav2vec 2.0 backbone by injecting knowledge from sentiment and ASR tasks for Alzheimer's disease detection in low-resource settings.
The authors fine-tune wav2vec 2.0 separately on a sentiment dataset and an ASR dataset, compute the weight increment of each Transformer layer, and obtain two task vectors.
In the AD training stage, the wav2vec 2.0 backbone is frozen, the two task vectors are injected into each Transformer layer with per-layer learnable merging coefficients, and the entire training process optimizes only these merging coefficients and one linear classification head, with trainable parameters under 0.1% of full fine-tuning.
wav2vec 2.0 is a self-supervised speech representation model proposed by Meta (formerly Facebook) in 2020, consisting of a multi-layer convolutional feature encoder and 12 (base version) or 24 (large version) Transformer layers.
The output dimension of each Transformer layer in the base version of wav2vec 2.0 is 768.
Meta's MMS model is based on the wav2vec 2.0 architecture and achieves speech recognition coverage by using religious texts (primarily multilingual translations of the Bible) as weakly supervised data sources
All Facts (6)
Dual-task vector merging is a parameter-efficient method that adapts a frozen wav2vec 2.0 backbone by injecting knowledge from sentiment and ASR tasks for Alzheimer's disease detection in low-resource settings.
50%UnverifiedThe authors fine-tune wav2vec 2.0 separately on a sentiment dataset and an ASR dataset, compute the weight increment of each Transformer layer, and obtain two task vectors.
50%UnverifiedIn the AD training stage, the wav2vec 2.0 backbone is frozen, the two task vectors are injected into each Transformer layer with per-layer learnable merging coefficients, and the entire training process optimizes only these merging coefficients and one linear classification head, with trainable parameters under 0.1% of full fine-tuning.
50%Unverifiedwav2vec 2.0 is a self-supervised speech representation model proposed by Meta (formerly Facebook) in 2020, consisting of a multi-layer convolutional feature encoder and 12 (base version) or 24 (large version) Transformer layers.
50%UnverifiedThe output dimension of each Transformer layer in the base version of wav2vec 2.0 is 768.
50%UnverifiedMeta's MMS model is based on the wav2vec 2.0 architecture and achieves speech recognition coverage by using religious texts (primarily multilingual translations of the Bible) as weakly supervised data sources
50%