Llama-3.2-3B
A small-scale large language model released by Meta with 3 billion parameters, part of the Llama 3.2 series, used as the baseline model in data selection experiments.
Timeline (last 90 days)
On Llama-3.2-3B, fine-tuning a brand-new dense model with the 5% of data selected by CACTUS reached 49.44% accuracy on BIG-Bench Hard; selecting data directly with the dense model gave 49.54%; data selected by the ALPS method could only reach 47.22%.
Both the Qwen series models and Llama-3.2-3B can expose human-perceptible style axes through this method.
All Facts (2)
On Llama-3.2-3B, fine-tuning a brand-new dense model with the 5% of data selected by CACTUS reached 49.44% accuracy on BIG-Bench Hard; selecting data directly with the dense model gave 49.54%; data selected by the ALPS method could only reach 47.22%.
50%UnverifiedBoth the Qwen series models and Llama-3.2-3B can expose human-perceptible style axes through this method.
50%