Unverified50% confidenceFactExact time
在LLM领域，GELU（高斯误差线性单元）和SwiGLU正成为主流激活函数选择
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026
Sources
Related Claims
VerifiedLLaMA模型系列使用SwiGLU激活函数71% similarUnverifiedSwiGLU将Swish激活函数与门控线性单元（GLU）结合，Google在PaLM模型中首先大规模应用，实验表明在相同参数量下比ReLU、GeLU等激活函数带来约8%的性能提升64% similarVerified从贝叶斯视角理解，L1正则化对应参数的拉普拉斯先验，L2正则化对应高斯先验63% similarUnverified选择性量化思路与LLM领域的GPTQ、AWQ等量化方法一脉相承63% similarVerified大模型分为基础模型（Base LLM）和指令微调模型（Instruction-tuned LLM）两大类型62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/47626API
curl https://kongchang.com/api/v1/knowledge/claims/47626MCP
get_claim(id=47626)