ConceptGrouped GEMM
Grouped GEMM
A technique that batches multiple independent small matrix multiplications into a single kernel invocation. It is commonly used in MoE inference to avoid the overhead of launching a kernel for each expert.
A technique that batches multiple independent small matrix multiplications into a single kernel invocation. It is commonly used in MoE inference to avoid the overhead of launching a kernel for each expert.