LLM City: What It's Like to Turn Model Weights Into a 3D City

When Model Weights Become a City

The parameter scale of large language models is often described with abstract numbers like "hundreds of billions" or "trillions," but for ordinary people—and even industry practitioners—these figures rarely convey any intuitive sense of scale. Recently, a project that sparked heated discussion on Hacker News, LLM City, attempts to answer a question with striking visual impact: if you turned every single weight of a large model into a physical object, just how enormous would it be?

The project renders all the weights of the Kimi K3 model as a 3D city—each weight represented as a 2.5-millimeter "tile," densely laid out to eventually form a seemingly endless urban landscape. This approach to large model visualization gives the abstract parameter count, for the first time, an intuitive spatial correspondence.

Why Use a "City" as the Metaphor

From Numbers to Space: Making Parameter Scale Tangible

Human perception of pure numbers is limited. When we hear "trillion parameters," it's hard to construct a real sense of scale in our minds. But when each parameter is given a fixed physical size (2.5mm) and arranged in three-dimensional space according to some rule, differences in magnitude are translated into visible differences in area and volume.

This is precisely the cleverness of LLM City. It borrows the imagery of a city—humanity's most familiar collection of large-scale structures: skyscrapers, blocks, buildings stretching to the horizon—to carry the vast quantity of a model's weights. When you "look down" on this city built from weights in the rendered scene, the sense of scale is something no number could ever convey.

The Design Considerations Behind Each Tile

The choice of 2.5 millimeters as the size of a single tile is no accident. This scale is small enough to accommodate an enormous number of parameters, yet large enough to be individually distinguishable when zoomed in. This design lets viewers freely switch between "looking down on the entire city from above" and "examining a single tile up close," thereby understanding the continuous scale relationship from an individual weight to the model as a whole.

The Value of Large Model Visualization in Understanding AI

Another Attempt to Break the Black Box

Large models have long been criticized as "black boxes"—we know they work, but it's hard to understand their internal mechanisms. While LLM City can't truly explain the semantic relationships between weights or the underlying computational logic, it at least offers a "materialized" perspective that lets people intuitively feel the model's physical volume.

The value of such data visualization projects lies in education and communication. For beginners, understanding the core concept of "parameter count" is often the first hurdle to entry. Transforming it into a 3D city you can "roam through" undoubtedly lowers the cognitive barrier and makes technical communication more engaging.

Echoing Other AI Visualization Projects

In recent years, the AI community has produced many attempts to visualize the internal structure of models—such as heatmaps of Transformer attention mechanisms and dynamic demonstrations of neural network activations. What distinguishes LLM City from these is that it doesn't focus on "how the model thinks," but returns to the most basic question of "how big the model is." This back-to-basics angle actually feels remarkably fresh in an era of information overload.

The Project's Limitations and Deeper Reflections

Shortcomings Beyond the Visual Impact

It must be acknowledged that LLM City is more a piece of "data art" than a rigorous analytical tool. Simply mapping weights to equally-sized tiles ignores the magnitude of the weights, their sign, sparsity, and the structural differences between layers. In other words, all the "buildings" in this city look identical, unable to reflect the true heterogeneity within the model.

If future versions could assign tiles different heights, colors, or densities based on a weight's numerical value, importance, or the layer it belongs to, then this city might evolve from a pure "scale display" into a true "structural map," providing more practical value for model interpretability research.

A Metaphor for the AI Scale Race

It's worth pondering that when we see an entire city representing merely the weights of a single model, we can't help but reflect on the current "scale race" in the AI field. Models are getting larger and larger, with parameters surging from tens of billions toward trillions—and LLM City reminds us precisely through physical volume: every leap in parameter count corresponds to staggering costs in storage, computation, and energy consumption. This city is both a showcase of technical achievement and a metaphor for resource consumption.

Conclusion

LLM City is a small but exquisite project. It doesn't propose new algorithms or offer groundbreaking technology, but it transforms cold parameter numbers into a perceptible spatial experience in a highly imaginative way. In an era where AI technology grows increasingly abstract and ever more distant from ordinary people, the significance of such visualization attempts perhaps lies not in how much they reveal, but in making more people willing to stop and think about what these behemoths actually mean.

For tech enthusiasts who follow AI, LLM City is worth experiencing firsthand—when you truly "step into" that city built from weights, your understanding of large models may gain an unprecedented new dimension.