ApexW
ApexW is a PyTorch-based optimizer implementation that combines compact state representations with Triton kernels, aiming to reduce the GPU memory footprint of optimizer states and to improve update speed, particularly for large-GPU workloads.
Timeline (last 90 days)
In benchmarks targeting only the optimizer itself, the PyTorch optimizer ApexW built by the developer, using a Triton execution path plus a compact state representation, is about 1.5 times faster than PyTorch fused AdamW in scenarios with a small number of large tensors.
ApexW's optimizer state memory usage is roughly halved compared with PyTorch fused AdamW.
ApexW's automatic execution mode currently cannot yet stably reproduce the 1.5x speedup.
Under workloads with a large number of small tensors, ApexW is noticeably slower than the PyTorch fused optimizer.
ApexW's project code has been published on GitHub, with the repository address github.com/omen42515-prog/apexw.
ApexW is currently better seen as an experimental project worth tracking rather than a production-environment replacement for AdamW.
All Facts (6)
In benchmarks targeting only the optimizer itself, the PyTorch optimizer ApexW built by the developer, using a Triton execution path plus a compact state representation, is about 1.5 times faster than PyTorch fused AdamW in scenarios with a small number of large tensors.
50%UnverifiedApexW's optimizer state memory usage is roughly halved compared with PyTorch fused AdamW.
50%UnverifiedApexW's automatic execution mode currently cannot yet stably reproduce the 1.5x speedup.
50%UnverifiedUnder workloads with a large number of small tensors, ApexW is noticeably slower than the PyTorch fused optimizer.
50%UnverifiedApexW is currently better seen as an experimental project worth tracking rather than a production-environment replacement for AdamW.
50%UnverifiedApexW's project code has been published on GitHub, with the repository address github.com/omen42515-prog/apexw.
50%