[KongchangAI]
ModelQwen2.5-0.5B-Instruct

Qwen2.5-0.5B-Instruct

A lightweight instruction-tuned language model released by Alibaba's Qwen team, with 0.5B parameters. It belongs to the Qwen2.5 series and is intended for text generation tasks in resource-constrained settings.

Timeline (last 90 days)

Nov 5

The authors actually ran Qwen2.5-0.5B-Instruct with transformers on the CPU in Colab, generating the probability data used in the animation.

Unverified50%
Nov 5

The default top-p of Qwen2.5-0.5B-Instruct is 0.8, which in this example would directly cut off 'lunar', thereby blocking the generation path of ' lunar South Equatorial region'.

Unverified50%
Nov 5

In Qwen2.5-0.5B-Instruct, after the input prompt 'Write one sentence about the moon.', following 'The', the model scores 151,936 vocabulary tokens, among which 'moon' has a probability of 78.04%, 'full' 4.46%, 'vast' 2.24%, 'Moon' 1.51%, and 'lunar' 1.50%.

Unverified50%
Oct 14

When generating structured support tickets in the browser with Qwen2.5-0.5B-Instruct, the biggest difficulty is getting the model to reliably output correctly formatted JSON, rather than understanding the complaint content.

Unverified50%
Oct 13

The author reproduced it on a T4 with the same model and testing method: at 8 concurrency, a single request decodes at 122–124 tokens/s, and the whole server produces 701–708 tokens/s, likewise about a 5.7-fold difference.

Unverified50%
Oct 13

Running vLLM 0.27.1 official image to infer Qwen2.5-0.5B-Instruct (fp16) on an H100 SXM 80GB on vast.ai ($2.689 per hour, 20 vCPU), the cost per million output tokens is about $1.26 under a single-request serial load.

Unverified50%
Oct 13

Under the same H100 configuration, the cost per million output tokens is about $0.22–0.23 under an 8-request concurrent load.

Unverified50%
Oct 13

The boundaries of the experimental conclusions include: only a 0.5B small model was used, a single card, and only three rounds per load type; the numbers would change with a 7B or 70B model, different servers, or longer prompts.

Unverified50%
Oct 12

The measured timeline was: billing starts at 0 seconds, prefill CUDA graph capture completes at about 371 seconds (taking 287.7 seconds), /health returns 200 at 386.7 seconds, the first request completes at about 563 seconds (taking about 176 seconds), the second request completes at about 579 seconds, and billing stops at 639.6 seconds.

Unverified50%
Oct 12

Running SGLang 0.5.21 on a single T4 in Colab to load the Qwen2.5-0.5B-Instruct fp16 model, in one complete cold-start measurement, 88% of the billed time was spent before the first user request completed.

Unverified50%

2 more timeline events

All Facts (12)

Unverified

The authors actually ran Qwen2.5-0.5B-Instruct with transformers on the CPU in Colab, generating the probability data used in the animation.

50%
Unverified

The default top-p of Qwen2.5-0.5B-Instruct is 0.8, which in this example would directly cut off 'lunar', thereby blocking the generation path of ' lunar South Equatorial region'.

50%
Unverified

In Qwen2.5-0.5B-Instruct, after the input prompt 'Write one sentence about the moon.', following 'The', the model scores 151,936 vocabulary tokens, among which 'moon' has a probability of 78.04%, 'full' 4.46%, 'vast' 2.24%, 'Moon' 1.51%, and 'lunar' 1.50%.

50%
Unverified

When generating structured support tickets in the browser with Qwen2.5-0.5B-Instruct, the biggest difficulty is getting the model to reliably output correctly formatted JSON, rather than understanding the complaint content.

50%
Unverified

The author reproduced it on a T4 with the same model and testing method: at 8 concurrency, a single request decodes at 122–124 tokens/s, and the whole server produces 701–708 tokens/s, likewise about a 5.7-fold difference.

50%
Unverified

Running vLLM 0.27.1 official image to infer Qwen2.5-0.5B-Instruct (fp16) on an H100 SXM 80GB on vast.ai ($2.689 per hour, 20 vCPU), the cost per million output tokens is about $1.26 under a single-request serial load.

50%
Unverified

Under the same H100 configuration, the cost per million output tokens is about $0.22–0.23 under an 8-request concurrent load.

50%
Unverified

The boundaries of the experimental conclusions include: only a 0.5B small model was used, a single card, and only three rounds per load type; the numbers would change with a 7B or 70B model, different servers, or longer prompts.

50%
Unverified

Running SGLang 0.5.21 on a single T4 in Colab to load the Qwen2.5-0.5B-Instruct fp16 model, in one complete cold-start measurement, 88% of the billed time was spent before the first user request completed.

50%
Unverified

Shivacon-Energy is a 0.5B-parameter energy-domain LoRA adapter based on Qwen2.5-0.5B-Instruct, whose core goal is calibration rather than knowledge expansion, emphasizing refusal to fabricate and outputting valid JSON.

50%
Unverified

G1's 0 wins/0 losses/6 ties result shows that the energy adapter does not know more energy facts than the base model; its advantage is purely behavioral-level honest calibration and JSON reliability.

50%
Unverified

The measured timeline was: billing starts at 0 seconds, prefill CUDA graph capture completes at about 371 seconds (taking 287.7 seconds), /health returns 200 at 386.7 seconds, the first request completes at about 563 seconds (taking about 176 seconds), the second request completes at about 579 seconds, and billing stops at 639.6 seconds.

50%

Source Articles