Llmspeed
An open-source crowdsourced benchmarking project and tool used to display the inference speed (tokens per second) of local large language models on different hardware, helping users assess the feasibility of running local LLMs before purchasing hardware.
Timeline (last 90 days)
Llmspeed is an open-source project released through the Show HN section of Hacker News, used to showcase local large language model inference speed.
Llmspeed accumulates real speed-test data in a crowdsourced way, letting users refer to the tokens-per-second output of a certain model on a certain type of hardware before purchasing hardware.
When using speed-test data to make purchasing decisions, one should first determine the model and quantization level to work backward to VRAM requirements, focus on generation speed rather than peak compute, accept the trade-off between precision and speed, and treat the data as a reference rather than a precise conclusion.
The advantage of the crowdsourced speed-test model is broad coverage and closeness to real usage scenarios, but when hardware configurations, software versions, and test prompts are not uniform, the comparability of the data suffers.
All Facts (4)
Llmspeed is an open-source project released through the Show HN section of Hacker News, used to showcase local large language model inference speed.
50%UnverifiedLlmspeed accumulates real speed-test data in a crowdsourced way, letting users refer to the tokens-per-second output of a certain model on a certain type of hardware before purchasing hardware.
50%UnverifiedWhen using speed-test data to make purchasing decisions, one should first determine the model and quantization level to work backward to VRAM requirements, focus on generation speed rather than peak compute, accept the trade-off between precision and speed, and treat the data as a reference rather than a precise conclusion.
50%UnverifiedThe advantage of the crowdsourced speed-test model is broad coverage and closeness to real usage scenarios, but when hardware configurations, software versions, and test prompts are not uniform, the comparability of the data suffers.
50%