Qwen2.5-0.5B-Instruct
A lightweight instruction-tuned language model released by Alibaba's Qwen team, with 0.5B parameters. It belongs to the Qwen2.5 series and is intended for text generation tasks in resource-constrained settings.
Timeline (last 90 days)
The authors actually ran Qwen2.5-0.5B-Instruct with transformers on the CPU in Colab, generating the probability data used in the animation.
The default top-p of Qwen2.5-0.5B-Instruct is 0.8, which in this example would directly cut off 'lunar', thereby blocking the generation path of ' lunar South Equatorial region'.
In Qwen2.5-0.5B-Instruct, after the input prompt 'Write one sentence about the moon.', following 'The', the model scores 151,936 vocabulary tokens, among which 'moon' has a probability of 78.04%, 'full' 4.46%, 'vast' 2.24%, 'Moon' 1.51%, and 'lunar' 1.50%.
When generating structured support tickets in the browser with Qwen2.5-0.5B-Instruct, the biggest difficulty is getting the model to reliably output correctly formatted JSON, rather than understanding the complaint content.
The author reproduced it on a T4 with the same model and testing method: at 8 concurrency, a single request decodes at 122–124 tokens/s, and the whole server produces 701–708 tokens/s, likewise about a 5.7-fold difference.
Running vLLM 0.27.1 official image to infer Qwen2.5-0.5B-Instruct (fp16) on an H100 SXM 80GB on vast.ai ($2.689 per hour, 20 vCPU), the cost per million output tokens is about $1.26 under a single-request serial load.
Under the same H100 configuration, the cost per million output tokens is about $0.22–0.23 under an 8-request concurrent load.
The boundaries of the experimental conclusions include: only a 0.5B small model was used, a single card, and only three rounds per load type; the numbers would change with a 7B or 70B model, different servers, or longer prompts.
The measured timeline was: billing starts at 0 seconds, prefill CUDA graph capture completes at about 371 seconds (taking 287.7 seconds), /health returns 200 at 386.7 seconds, the first request completes at about 563 seconds (taking about 176 seconds), the second request completes at about 579 seconds, and billing stops at 639.6 seconds.
Running SGLang 0.5.21 on a single T4 in Colab to load the Qwen2.5-0.5B-Instruct fp16 model, in one complete cold-start measurement, 88% of the billed time was spent before the first user request completed.
2 more timeline events
All Facts (12)
The authors actually ran Qwen2.5-0.5B-Instruct with transformers on the CPU in Colab, generating the probability data used in the animation.
50%UnverifiedThe default top-p of Qwen2.5-0.5B-Instruct is 0.8, which in this example would directly cut off 'lunar', thereby blocking the generation path of ' lunar South Equatorial region'.
50%UnverifiedIn Qwen2.5-0.5B-Instruct, after the input prompt 'Write one sentence about the moon.', following 'The', the model scores 151,936 vocabulary tokens, among which 'moon' has a probability of 78.04%, 'full' 4.46%, 'vast' 2.24%, 'Moon' 1.51%, and 'lunar' 1.50%.
50%UnverifiedWhen generating structured support tickets in the browser with Qwen2.5-0.5B-Instruct, the biggest difficulty is getting the model to reliably output correctly formatted JSON, rather than understanding the complaint content.
50%UnverifiedThe author reproduced it on a T4 with the same model and testing method: at 8 concurrency, a single request decodes at 122–124 tokens/s, and the whole server produces 701–708 tokens/s, likewise about a 5.7-fold difference.
50%UnverifiedRunning vLLM 0.27.1 official image to infer Qwen2.5-0.5B-Instruct (fp16) on an H100 SXM 80GB on vast.ai ($2.689 per hour, 20 vCPU), the cost per million output tokens is about $1.26 under a single-request serial load.
50%UnverifiedUnder the same H100 configuration, the cost per million output tokens is about $0.22–0.23 under an 8-request concurrent load.
50%UnverifiedThe boundaries of the experimental conclusions include: only a 0.5B small model was used, a single card, and only three rounds per load type; the numbers would change with a 7B or 70B model, different servers, or longer prompts.
50%UnverifiedRunning SGLang 0.5.21 on a single T4 in Colab to load the Qwen2.5-0.5B-Instruct fp16 model, in one complete cold-start measurement, 88% of the billed time was spent before the first user request completed.
50%UnverifiedShivacon-Energy is a 0.5B-parameter energy-domain LoRA adapter based on Qwen2.5-0.5B-Instruct, whose core goal is calibration rather than knowledge expansion, emphasizing refusal to fabricate and outputting valid JSON.
50%UnverifiedG1's 0 wins/0 losses/6 ties result shows that the energy adapter does not know more energy facts than the base model; its advantage is purely behavioral-level honest calibration and JSON reliability.
50%UnverifiedThe measured timeline was: billing starts at 0 seconds, prefill CUDA graph capture completes at about 371 seconds (taking 287.7 seconds), /health returns 200 at 386.7 seconds, the first request completes at about 563 seconds (taking about 176 seconds), the second request completes at about 579 seconds, and billing stops at 639.6 seconds.
50%