Phi
A series of on-device small language models developed by Microsoft. The series makes extensive use of model distillation and quantization compression techniques to achieve efficient inference at low energy consumption.
Core Facts
Integrated Information Theory (IIT) was first systematically proposed by neuroscientist Giulio Tononi in 2004, using the quantitative measure Φ to gauge a system's capacity to integrate information.
Microsoft's Phi series models (Phi-1, Phi-1.5, Phi-2), with parameter counts ranging from 1.3B to 2.7B, are trained on high-quality synthetic data generated by GPT-4 and approach or surpass models with 10 times the parameters on multiple benchmarks.
Timeline (last 90 days)
Open-source small model families such as Meta's Llama, Google's Gemma, and Microsoft's Phi continue to push performance ceilings in the 3B–7B parameter range.
Some versions of the Qwen and Phi series have benefited from distillation techniques.
Mistral/Mixtral wraps user input with [INST] and [/INST]; the ChatML format uses <|im_start|> and <|im_end|> as markers and is adopted by multiple models including Qwen and Phi.
参数量从千亿级压缩到70亿乃至更小规模的大语言模型（如Llama、Phi、Gemma系列）已能在消费级显卡上流畅运行
Representative micro LLMs in the industry include small-parameter versions of Meta's Llama series, Microsoft's Phi series, and Google's Gemma Nano.
Representative SLM products include Microsoft's Phi series, Google's Gemma, and Meta's small-parameter versions of Llama.
Cost-effective models such as Mistral and the Phi series have heavily drawn on knowledge distillation approaches.
A clear trend toward smaller models has emerged in the AI field, ranging from the Phi series to various high-efficiency models with 1–3B parameters.
Rebiha currently supports mainstream open-source models including Qwen, Gemma, Phi, DeepSeek, Llama, and Mistral.
Model quantization techniques (such as 4-bit quantization in the GGUF format) and on-device small models (such as Apple's OpenELM, Microsoft's Phi series, and Meta's Llama 3.2 lightweight versions) are narrowing the gap between local inference and cloud inference.
8 more timeline events
All Facts (20)
Integrated Information Theory (IIT) was first systematically proposed by neuroscientist Giulio Tononi in 2004, using the quantitative measure Φ to gauge a system's capacity to integrate information.
75%VerifiedMicrosoft's Phi series models (Phi-1, Phi-1.5, Phi-2), with parameter counts ranging from 1.3B to 2.7B, are trained on high-quality synthetic data generated by GPT-4 and approach or surpass models with 10 times the parameters on multiple benchmarks.
70%Unverified小语言模型（SLM）通常指参数量在数亿到数十亿级别的模型，代表包括Microsoft的Phi系列和Google的Gemma系列
90%Unverified微软的Phi系列小模型和Google的Gemma都大量使用了合成数据进行训练
85%UnverifiedMicrosoft Phi系列证明了在小语言模型中数据质量比数量更重要
80%UnverifiedLlama、Mistral、Phi等小模型的崛起证明并非所有场景都需要GPT-4级别的能力
80%Unverified2024年以来，Mistral、Phi、Gemma等小型模型通过更精细的数据筛选、知识蒸馏和训练策略优化，在特定任务上逼近甚至超越了体量数倍于己的大模型
80%Unverified微软的Phi系列通过精心筛选高质量教科书级训练数据，证明了1-3B参数量的模型在特定任务上可以媲美甚至超越十倍以上参数量的模型
75%UnverifiedMicrosoft's Phi series models have demonstrated that textbook-quality curated data can enable small models to far surpass competitors of the same scale in reasoning ability.
60%UnverifiedMicrosoft的Phi系列和Google的Gemma系列是小语言模型的代表性工作
60%UnverifiedPhi、Gemma、Qwen等系列有小参数版本用于端侧部署和低成本推理场景
60%UnverifiedOpen-source small model families such as Meta's Llama, Google's Gemma, and Microsoft's Phi continue to push performance ceilings in the 3B–7B parameter range.
50%UnverifiedSome versions of the Qwen and Phi series have benefited from distillation techniques.
50%UnverifiedMistral/Mixtral wraps user input with [INST] and [/INST]; the ChatML format uses <|im_start|> and <|im_end|> as markers and is adopted by multiple models including Qwen and Phi.
50%Unverified参数量从千亿级压缩到70亿乃至更小规模的大语言模型（如Llama、Phi、Gemma系列）已能在消费级显卡上流畅运行
50%UnverifiedRepresentative micro LLMs in the industry include small-parameter versions of Meta's Llama series, Microsoft's Phi series, and Google's Gemma Nano.
50%UnverifiedRepresentative SLM products include Microsoft's Phi series, Google's Gemma, and Meta's small-parameter versions of Llama.
50%UnverifiedCost-effective models such as Mistral and the Phi series have heavily drawn on knowledge distillation approaches.
50%UnverifiedA clear trend toward smaller models has emerged in the AI field, ranging from the Phi series to various high-efficiency models with 1–3B parameters.
50%UnverifiedRebiha currently supports mainstream open-source models including Qwen, Gemma, Phi, DeepSeek, Llama, and Mistral.
50%