Chunked Processing
An engineering approach for addressing memory bottlenecks that divides model layers or data tensors into small chunks, computes them chunk by chunk, and then reassembles the results to lower peak memory usage.
Timeline (last 90 days)
Chunked processing (Tiling/Chunking) splits model layers or data tensors into small chunks, computes them one by one, and then reassembles them, pressing peak VRAM usage down to the level required for a single chunk, at the cost of a slight drop in computational efficiency due to extra scheduling and synchronization between chunks.
The number of tiles can be set to 2, 4, or 8; the more tiles there are, the lower the VRAM requirement, but the longer the processing time, which is essentially trading time for VRAM.
All Facts (2)
Chunked processing (Tiling/Chunking) splits model layers or data tensors into small chunks, computes them one by one, and then reassembles them, pressing peak VRAM usage down to the level required for a single chunk, at the cost of a slight drop in computational efficiency due to extra scheduling and synchronization between chunks.
50%UnverifiedThe number of tiles can be set to 2, 4, or 8; the more tiles there are, the lower the VRAM requirement, but the longer the processing time, which is essentially trading time for VRAM.
50%