Tri Dao
An AI researcher and primary author of the FlashAttention paper, whose work focuses on the intersection of efficient deep learning algorithms and GPU systems optimization.
Core Facts
Flash Attention was proposed by Stanford's Tri Dao et al. in 2022, reducing memory complexity from O(N²) to O(N) through tiled computation and recomputation.
Flash Attention is an attention computation optimization algorithm proposed by Stanford University that processes data in chunks, keeping intermediate results in the GPU's high-speed SRAM to reduce memory bandwidth pressure and lower VRAM usage.
FlashAttention was proposed by Tri Dao et al. and improves attention computation speed by 2–4× through an IO-aware tiled computation strategy.
Timeline (last 90 days)
flash-attention was initially proposed by researchers such as Tri Dao, and subsequently community contributors continuously added support for new GPU architectures, low-precision formats and fused operators.
Flash Attention was proposed by Tri Dao and achieves a 2–4× speedup in practice.
FlashAttention was proposed by Tri Dao and colleagues at Stanford University; by performing attention computation in tiles within the GPU's on-chip SRAM, it reduces memory access count by an order of magnitude and is a mathematically exact-equivalent implementation.
All Facts (7)
Flash Attention was proposed by Stanford's Tri Dao et al. in 2022, reducing memory complexity from O(N²) to O(N) through tiled computation and recomputation.
90%VerifiedFlash Attention is an attention computation optimization algorithm proposed by Stanford University that processes data in chunks, keeping intermediate results in the GPU's high-speed SRAM to reduce memory bandwidth pressure and lower VRAM usage.
90%VerifiedFlashAttention was proposed by Tri Dao et al. and improves attention computation speed by 2–4× through an IO-aware tiled computation strategy.
90%VerifiedFlashAttention was proposed by Tri Dao and colleagues at Stanford University; by performing attention computation in tiles within the GPU's on-chip SRAM, it reduces memory access count by an order of magnitude and is a mathematically exact-equivalent implementation.
85%Unverifiedflash-attention was initially proposed by researchers such as Tri Dao, and subsequently community contributors continuously added support for new GPU architectures, low-precision formats and fused operators.
50%UnverifiedFlash Attention was proposed by Tri Dao and achieves a 2–4× speedup in practice.
50%UnverifiedMamba架构由Gu和Dao于2023年提出，其选择性状态空间机制以线性复杂度处理长序列
50%