Streaming

A transmission method in which a large language model generates a response token by token and pushes it to the client in real time. It substantially reduces time to first token and improves the experience of chat-based interaction.