ConceptFP4 GEMM
NVFP4 GEMM
A general matrix multiplication implementation based on NVIDIA's NVFP4 format, which uses a 4-bit floating-point format with block-level scaling. It is a core computational primitive in deep learning inference and training, with native hardware support provided by NVIDIA in the Blackwell and Vera Rubin architectures.
Timeline (last 90 days)
Oct 4
ThunderKittens has been ported to the NVIDIA Vera Rubin NVL72 platform, with NVFP4 GEMM kernels reconstructed around the new hardware.
Unverified50%
Oct 4
ThunderKittens' NVFP4 GEMM performance has increased from previously only 42% of the roofline theoretical peak to over 22 PFLOPS.
Unverified50%