Phi
A series of on-device small language models developed by Microsoft. The series makes extensive use of model distillation and quantization compression techniques to achieve efficient inference at low energy consumption.
Core Facts
Integrated Information Theory (IIT) was first systematically proposed by neuroscientist Giulio Tononi in 2004, using the quantitative measure Φ to gauge a system's capacity to integrate information.
Microsoft's Phi series models (Phi-1, Phi-1.5, Phi-2), with parameter counts ranging from 1.3B to 2.7B, are trained on high-quality synthetic data generated by GPT-4 and approach or surpass models with 10 times the parameters on multiple benchmarks.
Timeline (last 90 days)
From the Phi series to various vertical fine-tuned models, major vendors are breaking down large-model capabilities into composable specialized components
Common local small-model options include Meta's small Llama variants, Microsoft's Phi series, Google's small Gemma models, and loading locally via runtimes such as llama.cpp and Ollama.
Open-source models represented by Meta's Llama series, Mistral, and Microsoft Phi are the technical foundation that makes zero-cloud solutions viable.
Open-source small model families such as Meta's Llama, Google's Gemma, and Microsoft's Phi continue to push performance ceilings in the 3B–7B parameter range.
Some versions of the Qwen and Phi series have benefited from distillation techniques.
Mistral/Mixtral wraps user input with [INST] and [/INST]; the ChatML format uses <|im_start|> and <|im_end|> as markers and is adopted by multiple models including Qwen and Phi.
参数量从千亿级压缩到70亿乃至更小规模的大语言模型（如Llama、Phi、Gemma系列）已能在消费级显卡上流畅运行
Representative micro LLMs in the industry include small-parameter versions of Meta's Llama series, Microsoft's Phi series, and Google's Gemma Nano.
Representative SLM products include Microsoft's Phi series, Google's Gemma, and Meta's small-parameter versions of Llama.
Cost-effective models such as Mistral and the Phi series have heavily drawn on knowledge distillation approaches.
10 more timeline events
All Facts (20)
Integrated Information Theory (IIT) was first systematically proposed by neuroscientist Giulio Tononi in 2004, using the quantitative measure Φ to gauge a system's capacity to integrate information.
75%VerifiedMicrosoft's Phi series models (Phi-1, Phi-1.5, Phi-2), with parameter counts ranging from 1.3B to 2.7B, are trained on high-quality synthetic data generated by GPT-4 and approach or surpass models with 10 times the parameters on multiple benchmarks.
70%Unverified小语言模型（SLM）通常指参数量在数亿到数十亿级别的模型，代表包括Microsoft的Phi系列和Google的Gemma系列
90%Unverified微软的Phi系列小模型和Google的Gemma都大量使用了合成数据进行训练
85%UnverifiedMicrosoft Phi系列证明了在小语言模型中数据质量比数量更重要
80%UnverifiedLlama、Mistral、Phi等小模型的崛起证明并非所有场景都需要GPT-4级别的能力
80%Unverified2024年以来，Mistral、Phi、Gemma等小型模型通过更精细的数据筛选、知识蒸馏和训练策略优化，在特定任务上逼近甚至超越了体量数倍于己的大模型
80%Unverified微软的Phi系列通过精心筛选高质量教科书级训练数据，证明了1-3B参数量的模型在特定任务上可以媲美甚至超越十倍以上参数量的模型
75%UnverifiedMicrosoft's model supply hierarchy includes the top-tier OpenAI frontier large models, the middle-tier MAI in-house models, and the bottom-tier Phi series small models (such as Phi-4).
70%UnverifiedMicrosoft's Phi series models have demonstrated that textbook-quality curated data can enable small models to far surpass competitors of the same scale in reasoning ability.
60%UnverifiedMicrosoft的Phi系列和Google的Gemma系列是小语言模型的代表性工作
60%UnverifiedPhi、Gemma、Qwen等系列有小参数版本用于端侧部署和低成本推理场景
60%UnverifiedFrom the Phi series to various vertical fine-tuned models, major vendors are breaking down large-model capabilities into composable specialized components
50%UnverifiedCommon local small-model options include Meta's small Llama variants, Microsoft's Phi series, Google's small Gemma models, and loading locally via runtimes such as llama.cpp and Ollama.
50%UnverifiedOpen-source models represented by Meta's Llama series, Mistral, and Microsoft Phi are the technical foundation that makes zero-cloud solutions viable.
50%UnverifiedOpen-source small model families such as Meta's Llama, Google's Gemma, and Microsoft's Phi continue to push performance ceilings in the 3B–7B parameter range.
50%UnverifiedSome versions of the Qwen and Phi series have benefited from distillation techniques.
50%UnverifiedMistral/Mixtral wraps user input with [INST] and [/INST]; the ChatML format uses <|im_start|> and <|im_end|> as markers and is adopted by multiple models including Qwen and Phi.
50%Unverified参数量从千亿级压缩到70亿乃至更小规模的大语言模型（如Llama、Phi、Gemma系列）已能在消费级显卡上流畅运行
50%UnverifiedRepresentative micro LLMs in the industry include small-parameter versions of Meta's Llama series, Microsoft's Phi series, and Google's Gemma Nano.
50%