[KongchangAI]
ModelLingBot-Touch 触觉预训练模型

LingBot-Touch

A self-supervised tactile encoder proposed by the Robbyant team. Drawing on the JEPA idea, it predicts the representations of masked tactile signals in latent space and is used for robotic dexterous manipulation tasks.

Timeline (last 90 days)

Nov 9

The Robbyant team published the paper LingBot-Touch, proposing that in tactile pretraining, predicting the latent representation of masked contacts is more effective than reconstructing raw sensor signals.

Unverified50%
Nov 9

LingBot-Touch's inference code and checkpoints have been released.

Unverified50%
Nov 9

On the tactile-to-video top-5 recall task, with 1K hours of data, JEPA reaches 32.1%, MAE is 5.6%, and VAE is 4.4%.

Unverified50%
Nov 9

After scaling the data to 4K hours, JEPA rises to 49.3% on the tactile-to-video top-5 recall task, MAE remains below 9%, and VAE degrades to 3.1%.

Unverified50%
Nov 9

The paper argues that since the vast majority of tactile segments are near-zero background, the reconstruction loss can keep decreasing without learning true contact features, so the reconstruction objective gives the wrong optimization direction on sparse tactile signals.

Unverified50%
Nov 9

During LingBot-Touch training, the masking ratio starts at 40% and gradually rises to 80% within 300 steps.

Unverified50%
Nov 9

Starting directly with a high masking ratio would cause the model to mask all taxels currently in contact, losing learnable signal and thereby triggering gradient explosion.

Unverified50%
Nov 9

LingBot-Touch feeds data from three types of sensors—magnetic, piezoresistive, and visuotactile—into the same shared Transformer via normalized fingertip coordinates and 5D RoPE.

Unverified50%
Nov 9

The paper evaluates on 26 real robot tasks and 4 policies, and frozen encoder features achieve the highest score in 89 of the 104 policy-task combinations.

Unverified50%
Nov 9

On a magnetic dexterous hand never seen during pretraining, LingBot-VLA's score improves from 57.21 to 75.04.

Unverified50%

2 more timeline events

All Facts (12)

Unverified

The Robbyant team published the paper LingBot-Touch, proposing that in tactile pretraining, predicting the latent representation of masked contacts is more effective than reconstructing raw sensor signals.

50%
Unverified

On a magnetic dexterous hand never seen during pretraining, LingBot-VLA's score improves from 57.21 to 75.04.

50%
Unverified

The Tiny version model's performance levels off after about 4K hours of data, while the Small version keeps improving.

50%
Unverified

On the tactile-to-video top-5 recall task, with 1K hours of data, JEPA reaches 32.1%, MAE is 5.6%, and VAE is 4.4%.

50%
Unverified

After scaling the data to 4K hours, JEPA rises to 49.3% on the tactile-to-video top-5 recall task, MAE remains below 9%, and VAE degrades to 3.1%.

50%
Unverified

The paper argues that since the vast majority of tactile segments are near-zero background, the reconstruction loss can keep decreasing without learning true contact features, so the reconstruction objective gives the wrong optimization direction on sparse tactile signals.

50%
Unverified

During LingBot-Touch training, the masking ratio starts at 40% and gradually rises to 80% within 300 steps.

50%
Unverified

Starting directly with a high masking ratio would cause the model to mask all taxels currently in contact, losing learnable signal and thereby triggering gradient explosion.

50%
Unverified

LingBot-Touch feeds data from three types of sensors—magnetic, piezoresistive, and visuotactile—into the same shared Transformer via normalized fingertip coordinates and 5D RoPE.

50%
Unverified

The paper evaluates on 26 real robot tasks and 4 policies, and frozen encoder features achieve the highest score in 89 of the 104 policy-task combinations.

50%
Unverified

LingBot-Touch's inference code and checkpoints have been released.

50%
Unverified

The authors judge that larger-scale models paired with more data are the natural next step for tactile pretraining, suggesting the field may still be in an early scaling stage of data and capacity.

50%

Source Articles