SM
The basic compute unit in Nvidia GPU architecture, containing multiple sub-partitions, each with its own instruction scheduler and execution units; in the Ampere architecture each SM contains four sub-partitions.
Timeline (last 90 days)
With a batch size of 1, GPU parallel compute unit utilization is very low, because only one token participates in matrix operations at a time and a large number of SMs sit idle.
Each SM in the Ampere architecture contains four sub-partitions, and each sub-partition has 32 FP32 cores.
The instruction issue rate determines how many instructions an SM can dispatch to execution units per clock cycle.
On the measurement side, the gpu-unlock project provides a microbenchmark method for observing the issue behavior of specific SM sub-partitions under continuous FP16/BF16 instruction streams.
Instruction Issue Efficiency measures the proportion of clock cycles in which an SM successfully issues instructions.
The L1 cache is a high-speed cache private to each SM, while the L2 cache is shared by all SMs.
The process by which thread blocks are dispatched to SMs is transparent and non-intervenable at the application layer; once a large kernel occupies all SMs, subsequently submitted small kernels can only queue and wait even if they have higher priority.
The core idea of Green Contexts is to perform a soft partitioning of GPU compute resources, allowing developers to explicitly specify how many SM resources a given portion of the workload uses.
The Streaming Multiprocessor (SM) is the core compute unit of NVIDIA GPUs; each SM contains a number of CUDA cores, Tensor Cores, a register file, and shared memory, and is responsible for executing thread blocks in parallel.
A modern data center GPU (such as the H100) typically has tens to over a hundred SMs.
3 more timeline events
All Facts (13)
With a batch size of 1, GPU parallel compute unit utilization is very low, because only one token participates in matrix operations at a time and a large number of SMs sit idle.
50%UnverifiedEach SM in the Ampere architecture contains four sub-partitions, and each sub-partition has 32 FP32 cores.
50%UnverifiedThe instruction issue rate determines how many instructions an SM can dispatch to execution units per clock cycle.
50%UnverifiedOn the measurement side, the gpu-unlock project provides a microbenchmark method for observing the issue behavior of specific SM sub-partitions under continuous FP16/BF16 instruction streams.
50%UnverifiedInstruction Issue Efficiency measures the proportion of clock cycles in which an SM successfully issues instructions.
50%UnverifiedThe L1 cache is a high-speed cache private to each SM, while the L2 cache is shared by all SMs.
50%UnverifiedThe process by which thread blocks are dispatched to SMs is transparent and non-intervenable at the application layer; once a large kernel occupies all SMs, subsequently submitted small kernels can only queue and wait even if they have higher priority.
50%UnverifiedThe core idea of Green Contexts is to perform a soft partitioning of GPU compute resources, allowing developers to explicitly specify how many SM resources a given portion of the workload uses.
50%UnverifiedThe Streaming Multiprocessor (SM) is the core compute unit of NVIDIA GPUs; each SM contains a number of CUDA cores, Tensor Cores, a register file, and shared memory, and is responsible for executing thread blocks in parallel.
50%UnverifiedA modern data center GPU (such as the H100) typically has tens to over a hundred SMs.
50%UnverifiedThe compute cores (SMs) inside the GPU access already-decrypted local protected memory and do not need to repeatedly perform software-level encryption and decryption before and after each computation.
50%UnverifiedSouth Korea's three major entertainment companies SM, JYP, and HYBE have transformed CDs into blind-box-style consumer products, drawing on Japan's AKB48 handshake ticket business model.
50%Unverified占用率（occupancy）衡量实际活跃warp数与SM支持的最大warp数之比，受每个线程使用的寄存器数量、每个线程块的共享内存大小和线程块配置方式影响
50%