ConceptTTFT / Time to First Token
Time to First Token
An inference performance metric for large models that measures the time from when a user issues a request to when the model outputs its first token, reflecting the efficiency of the prefill stage.
Timeline (last 90 days)
Sep 16
Two common metrics for evaluating local model speed are time to first token (TTFT) and generation speed (tokens/s).
Unverified50%