Unverified50% confidenceFactExact time
The core idea of model distillation was proposed by Geoffrey Hinton et al. in 2015, and Google's Gemma 2 makes extensive use of distillation techniques to improve small model performance.
1
Sources
50%
Confidence
Long-term
Relevance
9/2/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedGoogle's Flash series models achieve high cost-effectiveness using techniques such as model distillation, Sparse Mixture of Experts architecture (SMoE), and inference-time computation optimization.64% similarUnverifiedGQA was first proposed by Google during the training of PaLM 2 and subsequently adopted by open-source models such as Llama 2/3 and Mistral.62% similarUnverified知识蒸馏最早由Hinton、Vinyals与Dean于2015年在Google提出62% similarUnverifiedMainstream open-source models such as Meta's LLaMA and Google's Gemma make extensive use of quantization and knowledge distillation techniques62% similarVerifiedGoogle Research released DreamFusion in 2022, demonstrating for the first time the possibility of generating 3D models from text prompts by using a pre-trained 2D diffusion model as a prior and optimizing a Neural Radiance Field (NeRF) representation via Score Distillation Sampling (SDS)62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/845848API
curl https://kongchang.com/api/v1/knowledge/claims/845848MCP
get_claim(id=845848)