Key Routing
A mechanism in sparse attention or cache retrieval that first predicts at low cost which Key pages are most relevant to the current Query, then performs exact QK dot-product computation only on the selected pages, reducing long-context inference overhead.
Timeline (last 90 days)
BasisKV reuses the compressed Value cache to predict part of the information needed for Key routing, and only reads back the original Keys for the selected Key pages to compute exact QK scores.
The two-stage design of Key routing is "coarse routing + fine scoring", that is, first predicting at low cost which Key pages are most relevant to the current Query, and then performing exact QK dot-product computation only on those pages.
All Facts (2)
BasisKV reuses the compressed Value cache to predict part of the information needed for Key routing, and only reads back the original Keys for the selected Key pages to compute exact QK scores.
50%UnverifiedThe two-stage design of Key routing is "coarse routing + fine scoring", that is, first predicting at low cost which Key pages are most relevant to the current Query, and then performing exact QK dot-product computation only on those pages.
50%