128G Mini PC Running Local LLMs: Expensive Isn't Always Fast—Pick the Model First, Then the Machine

Running Local LLMs on Mini PCs: The Expensive Machine Doesn't Win Every Category

Bilibili creator AIPC Yusengui did a rather unusual mini PC comparison: two machines, one with 128G of RAM and one with 64G, under strictly controlled conditions—the same batch of models, the same batch of questions, the same script, the same order. Material length was tested across nine tiers, from the shortest 400-character Weibo post to the longest 200,000-character novel, with both machines running each tier once. Only four things were compared: how long you wait for the first character, how fast it reads the material, how many characters it writes per second, and how long it takes to finish the whole response. Whichever machine had better data for a given piece of material won that tier; winning X out of nine tiers determined its score for that category.

The conclusion up front: the expensive machine doesn't win every category. And more importantly, swap the model on the same machine and the speed ranking flips. This means the question "which machine is stronger" is itself wrong—what you should ask is "does my machine run the model I'm actually using fast enough?"

The Main Model, 35B: The Extra Money Really Does Buy Speed

First, the main model: Qianwen 3.6 35B A3B—35B means 35 billion parameters, where B stands for 1 billion parameters. Parameters are like the pre-tuned knobs inside a model; the more there are, the more memory it consumes. The same model, with each machine running nine tiers.

First-character latency: wins five tiers, ties one, loses three—not a slow start. Material reading speed: wins six tiers, loses three. Full response completion: wins five tiers, loses four. The most striking is characters written per second: wins seven of nine tiers, ties one, loses only one—meaning it's faster in 78% of the tiers. So on this main model, the extra money does buy you something.

The judgment here is straightforward: if you run the main model every day and care about how fast it writes, the expensive machine is worth it.

A3B and MoE Architecture: The "35B A3B" mentioned in the article is a naming convention for a Mixture of Experts (MoE) model. The 35B in front refers to the model's total parameter count of roughly 35 billion, while the A3B behind it typically indicates that only about 3 billion parameters are activated per token during inference (Activated 3B). MoE splits the feed-forward layers into multiple "experts," with a gating network dynamically selecting a few experts per token to participate in computation. This way the total parameter count expands the model's capacity, but the computation per single forward pass is far less than a dense model with the same total parameter count—so VRAM usage is high but compute cost is relatively low. This precisely explains why the memory difference between the two machines (128G vs. 64G) becomes a key variable: the model must fit entirely into VRAM/RAM to run, and whether it fits, along with how the remaining space is allocated, directly affects inference speed.

Switching to 27B: The Same Machine, the Ranking Flips

Now look at 27B—271 parameters, one size smaller than the main model. Same machines, same script, yet the results flipped. First-character latency: wins only one tier, loses seven—the expensive machine that was fast at outputting characters just moments ago is slow to start on 27B. Material reading speed is the same: wins one, loses seven. Characters per second: four wins, four losses—a 50/50 split. Full response completion: also four wins, four losses.

Why does 27B make the expensive machine stumble? The creator says in the video that he's still verifying the data and plans to make a separate episode to explain it clearly. This point is crucial: the specific reason isn't settled yet, but the phenomenon itself is enough to make the point—the machine's advantage is tied to a specific model, and swap the model and the win-loss relationship reshuffles.

The MTP Version: Fast Start, at the Cost of Slow Writing

The third test is even stranger. Swapping the main model for a version with the MTP suffix—MTP can be understood as the model carrying a little "guess first, answer later" assistant: it guesses a few characters first, then verifies them together. On first-character latency, this version wins seven tiers and loses two; on material reading speed, wins seven and loses two—both look great. But on characters written per second, it wins three tiers and loses six—meaning it's actually slower in 67% of the tiers. Full response completion: wins six, loses three.

So the MTP version trades a fast start for slow character output. If you're in a hurry for the first sentence versus wanting it to write fast the whole time, the answer differs. The video also mentions that in the 27B row, statistics for eight tiers are available; one tier is still being verified and is being held back for now.

MTP (Multi-Token Prediction): Traditional autoregressive models predict only the next token each time, while MTP lets the model predict multiple future tokens at once, which are then verified or accepted in subsequent steps. The "guess first, answer later little assistant" described in the article is a plain-language way of describing this idea. Its benefits concentrate in first-token latency and the material-reading phase—because it can "pre-judge" subsequent content in parallel, it starts faster; but once it enters steady generation, wrong guesses require rollback and recomputation, and effective throughput may actually drop. This matches the observed phenomenon of "first character wins seven tiers, characters per second wins three loses six" in the tests. Note that MTP and "speculative decoding" are often conflated; the latter uses a small draft model to guess and a large model to verify, whereas MTP adds multi-token prediction heads inside the same model—different mechanisms, similar goals.

All Four Models Together: No Clean Sweep, Only Strength on a Specific Model

Looking at all four models together, one thing is clear: the expensive machine doesn't win every metric on any single model, and the cheap machine doesn't lose every metric on any single model. The creator initially guessed it was due to different VRAM allocation—VRAM being the portion of memory carved out for the GPU—but this hasn't been verified yet, so he doesn't dare present it as a conclusion. What's certain is that no machine sweeps every category; each is only strong on a specific model.

The fourth model, the small JMA model, was tested on only seven tiers: first-character latency three wins, one tie, three losses; character writing six wins, one loss. All four rows together come down to one sentence—pick the model first, then choose the machine.

VRAM Allocation and Unified Memory: Mini PCs typically lack a discrete GPU and instead rely on an integrated GPU sharing the same physical memory with the CPU (unified memory architecture). The so-called "VRAM allocation" refers to how much memory the BIOS or driver carves out for the GPU to use as VRAM, leaving the rest for the system. This ratio significantly affects LLM inference: allocate too little, and the model weights won't fit or are forced into tiered loading, causing a sharp drop in speed; allocate too much, and system memory becomes tight, which can also slow everything down. Different models have different VRAM demand curves (parameter count, quantization precision, and KV Cache size all differ), so the same allocation might be just right for 35B but not necessarily optimal for 27B—this is one possible explanation for "machine advantages being tied to a specific model," but the original article also stresses that this guess hasn't been verified.

How to Choose: Three Steps—Pick the Model First, Then Look at the Machine

The creator offers a very practical three-step method. Step one: first decide which model you run every day—the 35B main model, the 27B, or something smaller. Step two: look only at the category corresponding to your model—for writing long articles and wanting it to write fast consistently, look at characters per second; for chatting and waiting on the first sentence, look at first-character latency. Step three: then calculate the price difference between the two machines and see whether that money is spent on the category you care about.

His judgment is direct: if you run the main model every day and care about writing speed, the expensive machine is worth it; if you run the 27B tier every day, the extra money won't buy you speed. It's not about whether expensive is good or bad—it's about whether the expensive machine matches your model.

Go ahead and save this win-loss table to refer to before picking a machine. One final reminder: the above data was produced by the creator on his own two machines—a single-source empirical test with a sample of two machines, four models, and nine material tiers. The conclusions are worth referencing, but don't treat them as a universal law.