Unverified50% confidenceFactExact time
Matrix multiplication is typically compute-bound in high-batch inference scenarios, while autoregressive decoding with a batch size of 1 is almost invariably bandwidth-bound.
1
Sources
50%
Confidence
Long-term
Relevance
9/29/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFlattening a matrix and feeding it to an MLP results in a non-fixed input dimension due to variable row counts, and breaks permutation invariance.76% similarUnverified应优先融合访存受限的算子，对算力受限的大型算子（如大矩阵乘法）谨慎融合76% similarUnverified自回归推理时batch size通常为1，每个token生成只做一次矩阵-向量乘法，算术强度极低，导致memory-bound问题，单用户场景下GPU利用率往往不到10%73% similarUnverifiedThe standard approach for handling non-square cost matrices is to pad them into square matrices using large dummy cost values.71% similarVerifiedCode generation tasks are generally less sensitive to quantization than mathematical reasoning and complex logic tasks.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/958828API
curl https://kongchang.com/api/v1/knowledge/claims/958828MCP
get_claim(id=958828)