[KongchangAI]
· 4 min read· 2,404 words

Tsinghua Expert Breaks Down the Five-Layer Architecture of AI Large Models: From Applications to Energy Across the Full Chain

Tsinghua Expert Breaks Down the Five-Layer Architecture of AI Large Models: From Applications to Energy Across the Full Chain

Using the five-layer framework of application, model, infrastructure, chip, and energy to break down the full AI industry and the real progress behind the news.

The article addresses why ordinary people get more confused the more they read AI news by proposing a five-layer framework: application layer, model layer, infrastructure layer, chip layer, and energy layer. The application layer requires distinguishing wrappers from native, understanding Token billing and API calls, and grasping the trend from prompts to Agent intelligences; the model layer distinguishes training and inference, with DeepSeek lowering costs through algorithm optimization and fine-tuning as a low-cost path to specialization; the infrastructure layer's threshold lies in ten-thousand-card-level parallel collaboration and network transmission, with liquid cooling and site selection as keys to cost reduction; the chip layer explains why GPUs suit matrix operations and Nvidia's moat in the CUDA ecosystem; the energy layer points out that electricity is the ultimate bottleneck, with giants already signing directly with nuclear plants. Mastering these five layers lets you quickly locate the substantive progress of any tech news.

Why Do You Get More Confused the More AI News You Read?

Pick up your phone: today one company releases a new model, tomorrow another launches a new app, and the stock market is buzzing about compute power and chips every day. Many people get more and more confused. According to a judgment made by a creator on Bilibili in his "Complete Zero-Foundation Course on AI Large Models," the problem isn't how complex AI is — it's that most people only see the few most surface-level software products.

The framework he proposes is straightforward: from the most surface-level tools all the way down to the underlying physical world, there are actually just 5 links — the application layer, the model layer, the infrastructure layer, the chip layer, and the energy layer. Once you understand these 5 layers, the entire industry's threads become clear. The value of this layered perspective is that it lets you instantly locate "which layer has made real progress" whenever you see any piece of tech news.

Or tons on the water meter

Application Layer: Wrappers vs. Native, Tokens and APIs

Distinguish Wrappers from Native

The application layer is what everyone is most familiar with — ChatGPT, Doubao, Kimi, and image and video editing tools all sit at this layer. To understand this layer, just grasp three key points.

First, distinguish what is a wrapper and what is native. People used to use Word; now some old software adds a tiny AI window on the side, with the original logic completely unchanged — this can only count as icing on the cake. True AI-native software has all its operation logic designed around the large model from the very beginning. The creator judges that the former may survive, but the latter is what truly has vitality in the future.

Token Is the Unit of Measure, API Is the Pipe

Second, what Token and API are. In plain terms, Token is the unit of measure in the AI world — equivalent to kilowatt-hours on an electricity meter or tons on a water meter. When you send AI a passage of text or an image, the backend cuts it into individual Tokens for billing. API is a pipe: big companies train models in the cloud and deliver compute power to various applications through this pipe, and everyone pays by usage flow.

The business models are also clearly divided: ordinary individual users are mainly free or charged a small monthly fee, and vendors use individual users to test software and build reputation; the real money is on the enterprise customer side. Enterprises do the math realistically — buying AI isn't for trying something new, but for calculating a return: can it save labor costs, can it raise efficiency?

The way Tokens are split directly affects cost and model performance. Different models use different tokenizers, with common algorithms such as BPE (Byte Pair Encoding). An English word may be cut into one or more Tokens, and a Chinese character usually takes 1-2 Tokens. This means the same passage of Chinese may differ by nearly double in billing across different models. In addition, a model's "context window" is also measured in Tokens — for example, a 128K context means the model can "remember" about 128,000 Tokens of conversation at once, and beyond that, earlier information gets truncated. This is also a key parameter to watch in long-document processing scenarios.

From "Talking" to "Doing"

Third, and the biggest trend right now: AI is shifting from "talking" to "doing." In the past, using AI depended entirely on how well you wrote prompts; now everyone is building Agent intelligences, and the gap between the two is huge. Previously, when you asked AI to organize invoices, it would at most write you a guide on how to organize them, and in the end you still had to manually paste them. Today's Agent has "hands and feet" connected — it can go through your email, download attachments, identify invoice amounts, fill them into spreadsheets, and finally even send an email directly to finance. It is no longer just an assistant chatting with you, but has become a "digital employee" that can finish the work itself.

Model Layer: Training Is Like Educating a PhD, Inference Is Going to Work

No matter how good the software is, there must be a smart brain behind it — this brings us to the model layer. How do large models operate? Actually just two stages: training and inference.

The training stage is like educating a child from kindergarten all the way to a PhD — you have to hire the world's best tutors and stuff hundreds of billions of web pages, books, and code into his head. This process is not only mentally demanding but also expensive — tens of thousands of top-tier GPUs running together for months, and just the electricity bills and machine depreciation are astronomical. The inference stage is this PhD graduating and going to work: you ask him a question, and he thinks and gives you an answer.

You ask him a question

The creator specifically mentions that an important reason DeepSeekAd caused such a global shock is that through particularly clever algorithms, they managed to push down the cost of both training and inference, making everyone realize that you don't have to spend so much money to build a very smart brain.

How is the brain polished? First, data — sheer volume is useless; it must be high-quality data cleaned very thoroughly. Second, parameter count — the tens of Bs and hundreds of Bs you often hear; B means billion, and you can think of it as the connection density of brain neurons. Generally, the larger the parameters, the stronger the ability to handle complex problems. But in reality, you can't mobilize a super-large model for everything — the cost is too high. So the most common approach in the industry is fine-tuning: take an already smart large model and specifically teach it one or two years of finance or medical expertise to turn it into a specialized expert, with lower cost and more accurate answers.

Infrastructure Layer: Ten Thousand Cards Pulling One Net Together

With a brain, there must be a place to put it — this is the infrastructure layer, usually called infra in the industry, the hardest-core "heavy industry" of the entire AI industry.

Cloud vendors (Alibaba Cloud, Tencent Cloud, AWS, Microsoft Cloud, etc.) package compute power like water, electricity, and coal. Ordinary companies don't need to spend millions buying servers themselves — they can just open an account on the cloud and rent compute power by the hour. But what goes on inside the data center is far more complicated than imagined. Many people think doing AI is just buying GPUs; in fact, the industry has a pain point: buying one card is not hard, but connecting ten thousand cards to work together is the real technical threshold. During large model training, tens of thousands of cards are computing one big problem simultaneously — like ten thousand people pulling one large net together. If even one card goes offline, or data transmission slows by half a second, all the other cards have to stop and wait for it.

It's like ten thousand people pulling one large net together

So what big companies compete on, besides the cards in hand, are switches, optical modules, and the network transmission speed inside the data center. These servers generate enormous heat that traditional fans simply can't cool; many data centers now use direct liquid circulation cooling, i.e., liquid cooling. Many domestic computing centers are built in Guizhou, Inner Mongolia, or the northwest precisely because the weather there is cool, natural air cooling costs are low, and the data center can save a great deal on electricity.

Another key technology at the Infra layer is distributed parallel training. When a single GPU cannot hold a super-large model, the model must be split across multiple cards for collaborative computation. Common strategies include data parallelism (each card processes different data), tensor parallelism (splitting single-layer matrix operations), and pipeline parallelism (assigning different layers to different cards). The combination of different parallel strategies directly determines communication volume and training efficiency — which is why network bandwidth and latency become bottlenecks: the speed of transmitting gradient data between cards often determines the overall training cycle more than single-card compute power. Technologies such as RDMA and NVLink were born precisely to solve high-speed inter-card communication.

Chip Layer: Why GPU Instead of CPU

Digging further down, we reach the most core part of hardware — chips. The term heard most in this link is GPU. Why not use the CPU of an ordinary computer, and insist on grabbing GPUs? By analogy: a CPU is like an old university professor who understands calculus and logical reasoning and excels at handling single tasks with very complex procedures; a GPU is more like thousands of elementary school students sitting in a workshop, each of whom only knows simple addition, subtraction, multiplication, and division, but they can all do it at the same time. It just so happens that the underlying mathematical operations of large models are all massive matrix multiplications — essentially very simple but extremely numerous arithmetic problems. Thousands of elementary school students working in parallel can leave one old professor far behind in speed.

When it comes to chips, everyone surely thinks of Nvidia first. But what really gives Nvidia's competitors headaches is not just hardware, but the software ecosystem CUDA. More than a decade ago, Nvidia gave this software toolset free to universities and engineers worldwide, and today almost everyone who writes AI programs is accustomed to Nvidia's standards. It's like every auto mechanic holding a wrench specifically designed to turn Nvidia screws — even if other chip vendors make hardware performance roughly comparable, programmers take one look and find their existing code simply won't run, and they'd have to relearn and rewrite. Naturally, people are unwilling to switch.

Still have to relearn and rewrite

This is also why chips have become the front line of contention — it is no longer just a matter of a silicon wafer, but a whole chain from chip design and foundry manufacturing to software ecosystem. Whoever gets stuck will have their subsequent AI development slow down by half a beat.

Beyond CUDA, there is another often-overlooked concept: process node and foundry. Chip performance depends not only on design but also on manufacturing process — such as 5nm and 3nm process nodes. The smaller the number, the more transistors can be packed per unit area and the higher the energy efficiency. The world's most advanced foundry capacity is concentrated in the hands of a few players such as TSMC, while high-end lithography machines depend on equipment makers such as ASML. This is the concrete meaning of the "whole chain" mentioned in the article: design (Nvidia/AMD), manufacturing (TSMC/Samsung), equipment (ASML), and software ecosystem (CUDA) are all interlocked, and any restricted link will transmit to the entire AI industry.

Energy Layer: The End of AI Is Electricity

At the very bottom beneath the chips, there is another layer — and the one that global tech giants are most anxious about right now: the energy layer. There is a realistic saying in the industry — the end of AI is actually electricity.

Large models look like virtual code, but in the physical world they are real power hogs. When we casually ask it to write an article or generate a few seconds of video, the electricity consumed is dozens of times that of traditional search; when a super data center with hundreds of thousands of cards connected together runs, it consumes as much electricity as an entire small city. The bottleneck now is not just whether you can buy chips, but that in many places the power grid simply cannot connect such a large load. This is also why giants like Microsoft and Amazon have started directly signing decades-long power supply agreements with nuclear plants, and everyone is desperately laying out wind power, photovoltaics, and energy storage. No matter how beautifully the code is written, it ultimately comes down to physical laws — without massive, stable, and cheap electricity, superintelligence simply cannot run.

Five Layers Connected: No Longer Led Astray by News

Run through the five links from beginning to end: at the top is the visible and tangible application layer; supporting the operation of applications is the thinking model layer; providing a home for the brain is the supercomputing cluster of the infrastructure layer; the power core within the cluster is the parallel compute power of the chip layer; and at the very bottom, truly supporting the operation of all machines, is the continuous electricity of the energy layer.

Behind every seemingly effortless answer on the screen, there is actually an enormous industrial chain supporting it. This five-layer framework does not involve any specific product parameters, yet it is the most practical "judgment framework" for understanding the AI industry — next time you see a dazzling variety of tech news, you will clearly know which layer has made real progress, and specifically who will be affected along the chain by that progress.

One-line Summary

Using the five-layer framework of application, model, infrastructure, chip, and energy to break down the full picture of the AI industry and the real progress behind the news.

Paragraph Summary

The article addresses the phenomenon of ordinary people getting more confused the more they read AI news by proposing a five-layer framework: application layer, model layer, infrastructure layer, chip layer, and energy layer. At the application layer, distinguish wrappers from native, understand Token billing and API calls, and grasp the trend from prompts to Agent intelligences; at the model layer, distinguish the two stages of training and inference — DeepSeek lowers costs through algorithm optimization, and fine-tuning is the low-cost path to specialization; the threshold at the infrastructure layer lies in ten-thousand-card-level parallel collaboration and network transmission, with liquid cooling and site selection being key to cost reduction; the chip layer explains why GPUs suit matrix operations, and Nvidia's moat is the CUDA ecosystem; the energy layer points out that electricity is the ultimate bottleneck, and giants have already directly signed with nuclear plants. Master these five layers, and you can quickly locate the substantive progress of any tech news.

#AI large model#five-layer architecture#industry chain#Token#API#Agent#GPU#CUDA#infrastructure
Share:

Related articles