Unverified50% confidenceFactExact time
LLaMA-2 70B 模型共有 80 个 Transformer 层，每层包含自注意力机制和前馈神经网络两大模块
1
Sources
50%
Confidence
Long-term
Relevance
7/19/2026
First Seen
Sources
Related Claims
UnverifiedLlama-3 70B采用80层Transformer、每层64个注意力头、每头128维配置，并采用GQA设计73% similarUnverifiedLLaMA-2 70B采用80层、64个注意力头的架构，并使用GQA（8个KV头共享64个Q头）69% similarUnverifiedMegatron-LM最早系统化提出了针对Transformer的张量并行方案，将注意力头和FFN层分别切分到不同设备67% similarUnverifiedDiT（Diffusion Transformer）架构的参数量已达数十亿级别，单次推理的中间激活值在高分辨率下可占满80GB A100的全部显存64% similarUnverifiedThe Transformer architecture uses a Self-Attention mechanism to achieve efficient parallel processing of sequential data, replacing RNN/LSTM architectures.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/563109API
curl https://kongchang.com/api/v1/knowledge/claims/563109MCP
get_claim(id=563109)