[KongchangAI]
Model

GSLM2

A multi-level, multi-stream speech language model proposed by Meta that narrows the gap in linguistic capability between speech and text models through high-quality discrete units and multi-stream modeling.

Timeline (last 90 days)

Nov 3

The Meta research team proposes GSLM2, which narrows the capability gap between speech language models and text models through multi-level, multi-stream representation learning.

Unverified50%
Nov 3

GSLM2's encoder is named SpidR 2.0, which uses self-supervised learning and introduces a differentiable prediction head to induce high-quality discrete units at multiple levels.

Unverified50%
Nov 3

GSLM2 uses a multi-stream SLM for joint modeling, comprising a shared temporal backbone network, an independent predictor for each stream, and an autoregressive head that captures cross-stream dependencies.

Unverified50%
Nov 3

The research team conducted GSLM2 experiments at up to 6 million hours of training data and a model scale of 6.5 billion parameters.

Unverified50%
Nov 3

GSLM2 consistently improves language ability across different data scales and model scales, and its efficiency increases with data scaling.

Unverified50%
Nov 3

On the three tasks sWuggy, sBLIMP, and tSC, GSLM2 narrows the gap between speech models and text models by about two orders of magnitude.

Unverified50%
Nov 3

GSLM2's training code and model checkpoints have been open-sourced on GitHub at https://github.com/facebookresearch/gslm2.

Unverified50%

All Facts (7)

Source Articles