Unverified50% confidenceEventExact time
In the test, model loading took only a few seconds, GPU usage dropped from 23GB to about 17GB, the first test prompt processing took 225ms, generation speed was stable at 33.5 tok/s, with a total output of 1500 tokens.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/15/2026
First Seen
Valid until: 10/29/2026
Sources
GLM 5.3 Flash Local Test: Fitting a 320B Model into 4 RTX 3090s with 1-bit Quantization
bilibili四月的尾巴_10/13/2026
Related Entities
Related Claims
UnverifiedWhen using a consumer-grade GPU with 16GB of VRAM, lower-quantization versions can fit entirely into VRAM, achieving excellent running speeds.79% similarUnverified测试者用双4090（48GB显存）工作站运行20.7B稠密模型，每秒生成27.6个Token，仅比矮星略快，但参数量小10倍以上77% similarUnverifiedTaking a 70B BF16 model as an example, generating one token requires reading approximately 140 GB of weights, and when fully utilizing the H100's 3.35 TB/s memory bandwidth, the per-token latency is at least 42 milliseconds, with a theoretical TPS upper limit of approximately 24 tokens/second.77% similarUnverifiedIn BF16 configuration (two 48GB GPUs), decoding speed ranges from 25.1 tok/s at 1K to 23.9 tok/s at 128K, with almost no degradation.76% similarUnverifiedCloud-based APIs can typically achieve output speeds of tens to hundreds of tokens per second, while consumer-grade GPUs running 27B models often only reach 10–30 tokens per second.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1036001API
curl https://kongchang.com/api/v1/knowledge/claims/1036001MCP
get_claim(id=1036001)