UVA
A unified virtual addressing mechanism provided by NVIDIA CUDA that lets the GPU and CPU share a single virtual address space. GPU kernels can directly dereference host memory pointers, with data transferred over PCIe on demand.
Timeline (last 90 days)
Diffbot's approach moves 4,608 of the 15,360 routed experts, plus lookup tables (Engram tables int4 and embedding), entirely to host memory, with the GPU reading them over PCIe via UVA, reducing the footprint per card to 72.4 GiB.
UVA (Unified Virtual Addressing) is a mechanism provided by NVIDIA CUDA that lets the GPU and CPU share the same virtual address space, allowing GPU kernels to directly dereference pointers to host memory, with data transferred on demand over PCIe.
All Facts (2)
Diffbot's approach moves 4,608 of the 15,360 routed experts, plus lookup tables (Engram tables int4 and embedding), entirely to host memory, with the GPU reading them over PCIe via UVA, reducing the footprint per card to 72.4 GiB.
50%UnverifiedUVA (Unified Virtual Addressing) is a mechanism provided by NVIDIA CUDA that lets the GPU and CPU share the same virtual address space, allowing GPU kernels to directly dereference pointers to host memory, with data transferred on demand over PCIe.
50%