ModelDeiT III / DeiT-3

DeiT III

A series of data-efficient vision transformer models proposed by Meta that achieve strong performance on ImageNet through knowledge distillation and improved training strategies.

Timeline (last 90 days)

Oct 13

In the ImageNet-1k supervised training setting, reViT-B/16 trained from scratch reaches accuracy comparable to DeiT III while reducing stored parameters by about 70%.

Unverified50%

All Facts (1)

Unverified

In the ImageNet-1k supervised training setting, reViT-B/16 trained from scratch reaches accuracy comparable to DeiT III while reducing stored parameters by about 70%.

50%

Source Articles

New Breakthrough in Recurrent Vision Transformers: A Single Reused Block Approaches Full-Depth Encoder PerformanceOct 13