BLOKS
A scalable kernel method solver based on the conjugate gradient method. It constructs a novel preconditioner by combining block Jacobi approximation with sketching techniques, making preconditioner memory requirements independent of both dataset size and the number of parameters simultaneously, thereby addressing the memory bottleneck in kernel ridge regression.
Timeline (last 90 days)
BLOKS is a conjugate gradient-based solver for Nyström-sketched kernel ridge regression.
The core innovation of BLOKS is combining block Jacobi approximation with sketching techniques to form a novel preconditioner.
The memory overhead of the BLOKS preconditioner is simultaneously independent of dataset size and the number of parameters.
BLOKS applies sketch compression again to each diagonal block, limiting the effective size of the block to a fixed upper bound, so that the memory footprint of the entire preconditioner depends only on the block size hyperparameter.
The paper's authors provide a memory-efficient, GPU-accelerated implementation of BLOKS.
On standard large-scale datasets used for benchmarking kernel solvers, BLOKS achieves performance comparable to or better than current state-of-the-art methods while maintaining favorable time and memory complexity.
All Facts (6)
BLOKS is a conjugate gradient-based solver for Nyström-sketched kernel ridge regression.
50%UnverifiedThe core innovation of BLOKS is combining block Jacobi approximation with sketching techniques to form a novel preconditioner.
50%UnverifiedThe memory overhead of the BLOKS preconditioner is simultaneously independent of dataset size and the number of parameters.
50%UnverifiedBLOKS applies sketch compression again to each diagonal block, limiting the effective size of the block to a fixed upper bound, so that the memory footprint of the entire preconditioner depends only on the block size hyperparameter.
50%UnverifiedThe paper's authors provide a memory-efficient, GPU-accelerated implementation of BLOKS.
50%UnverifiedOn standard large-scale datasets used for benchmarking kernel solvers, BLOKS achieves performance comparable to or better than current state-of-the-art methods while maintaining favorable time and memory complexity.
50%