Decode Stage

The second stage of large language model inference, during which output tokens are generated serially, one at a time. This stage places high demands on memory capacity and memory bandwidth, making it well suited for devices with large unified memory.