LingBot-Touch
A self-supervised tactile encoder proposed by the Robbyant team. Drawing on the JEPA idea, it predicts the representations of masked tactile signals in latent space and is used for robotic dexterous manipulation tasks.
Timeline (last 90 days)
The Robbyant team published the paper LingBot-Touch, proposing that in tactile pretraining, predicting the latent representation of masked contacts is more effective than reconstructing raw sensor signals.
LingBot-Touch's inference code and checkpoints have been released.
On the tactile-to-video top-5 recall task, with 1K hours of data, JEPA reaches 32.1%, MAE is 5.6%, and VAE is 4.4%.
After scaling the data to 4K hours, JEPA rises to 49.3% on the tactile-to-video top-5 recall task, MAE remains below 9%, and VAE degrades to 3.1%.
The paper argues that since the vast majority of tactile segments are near-zero background, the reconstruction loss can keep decreasing without learning true contact features, so the reconstruction objective gives the wrong optimization direction on sparse tactile signals.
During LingBot-Touch training, the masking ratio starts at 40% and gradually rises to 80% within 300 steps.
Starting directly with a high masking ratio would cause the model to mask all taxels currently in contact, losing learnable signal and thereby triggering gradient explosion.
LingBot-Touch feeds data from three types of sensors—magnetic, piezoresistive, and visuotactile—into the same shared Transformer via normalized fingertip coordinates and 5D RoPE.
The paper evaluates on 26 real robot tasks and 4 policies, and frozen encoder features achieve the highest score in 89 of the 104 policy-task combinations.
On a magnetic dexterous hand never seen during pretraining, LingBot-VLA's score improves from 57.21 to 75.04.
2 more timeline events
All Facts (12)
The Robbyant team published the paper LingBot-Touch, proposing that in tactile pretraining, predicting the latent representation of masked contacts is more effective than reconstructing raw sensor signals.
50%UnverifiedOn a magnetic dexterous hand never seen during pretraining, LingBot-VLA's score improves from 57.21 to 75.04.
50%UnverifiedThe Tiny version model's performance levels off after about 4K hours of data, while the Small version keeps improving.
50%UnverifiedOn the tactile-to-video top-5 recall task, with 1K hours of data, JEPA reaches 32.1%, MAE is 5.6%, and VAE is 4.4%.
50%UnverifiedAfter scaling the data to 4K hours, JEPA rises to 49.3% on the tactile-to-video top-5 recall task, MAE remains below 9%, and VAE degrades to 3.1%.
50%UnverifiedThe paper argues that since the vast majority of tactile segments are near-zero background, the reconstruction loss can keep decreasing without learning true contact features, so the reconstruction objective gives the wrong optimization direction on sparse tactile signals.
50%UnverifiedDuring LingBot-Touch training, the masking ratio starts at 40% and gradually rises to 80% within 300 steps.
50%UnverifiedStarting directly with a high masking ratio would cause the model to mask all taxels currently in contact, losing learnable signal and thereby triggering gradient explosion.
50%UnverifiedLingBot-Touch feeds data from three types of sensors—magnetic, piezoresistive, and visuotactile—into the same shared Transformer via normalized fingertip coordinates and 5D RoPE.
50%UnverifiedThe paper evaluates on 26 real robot tasks and 4 policies, and frozen encoder features achieve the highest score in 89 of the 104 policy-task combinations.
50%UnverifiedLingBot-Touch's inference code and checkpoints have been released.
50%UnverifiedThe authors judge that larger-scale models paired with more data are the natural next step for tactile pretraining, suggesting the field may still be in an early scaling stage of data and capacity.
50%