Unverified50% confidenceFactExact time
Megatron-LM最早系统化提出了针对Transformer的张量并行方案，将注意力头和FFN层分别切分到不同设备
1
Sources
50%
Confidence
Long-term
Relevance
7/10/2026
First Seen
Sources
TensorRT多设备推理：突破单GPU显存瓶颈的工程实践
rss6/25/2026
Related Claims
Unverified相比RNN/LSTM架构，Transformer支持大规模并行计算73% similarUnverifiedThe Transformer architecture uses a Self-Attention mechanism to achieve efficient parallel processing of sequential data, replacing RNN/LSTM architectures.71% similarUnverifiedTransformers库集成了GPTQ、AWQ、bitsandbytes等量化技术和Flash Attention加速方案70% similarUnverifiedSora采用了Diffusion Transformer（DiT）架构，将Transformer全局注意力机制引入扩散框架67% similarUnverifiedLlama-3 70B采用80层Transformer、每层64个注意力头、每头128维配置，并采用GQA设计67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/461709API
curl https://kongchang.com/api/v1/knowledge/claims/461709MCP
get_claim(id=461709)