From AI to Large Models: Understanding the Conceptual Landscape and Technological Evolution of Artificial Intelligence

Many newcomers to AI share the same confusion: What exactly is the relationship between artificial intelligence, machine learning, deep learning, large models, and generative AI? Some think AI is just ChatGPT, others believe large models represent all of AI, and still others see machine learning, deep learning, and large models as completely independent technologies.

In reality, there's a very clear developmental thread connecting them all. Understanding these concepts is the first step toward getting into large model application development. Based on the introductory large model course series by Bilibili creator Jin Ze, this article traces AI's technological evolution from its origins to today, and explores how ordinary people can break into this field.

Artificial Intelligence: A Field of Technology, Not a Product

The core of artificial intelligence (AI) research is enabling computers or machines to possess human-like abilities in learning, understanding, reasoning, and action. Two keywords stand out here: first, "machine" — AI research focuses on computer programs, software systems, or robots, not humans themselves; second, "intelligence."

Views on what constitutes "intelligence" vary widely. By science fiction standards — say, Jarvis from Iron Man, with independent consciousness and autonomous thought — current technology is clearly nowhere close. This kind of AI with general cognitive abilities, capable of understanding and handling any task like a human, is academically known as Artificial General Intelligence (AGI), one of AI research's ultimate goals. Everything we interact with today, including ChatGPT, falls under Narrow AI — systems that excel only at specific domains or tasks. But from a more practical perspective: if a machine can accomplish tasks that previously required humans, it can be considered to possess a certain degree of intelligence. The voice assistant on your phone understands commands; map apps plan routes — these are all AI applications.

Therefore, artificial intelligence isn't a specific product but an entire field of technology, involving computer science, mathematics, statistics, linguistics, and even neuroscience. For application developers, the focus should be on "Can AI help me solve real problems?" rather than debating whether it has achieved sci-fi-level intelligence.

Traditional AI: The Rules and Search Behind Deep Blue

Two landmark moments stand out in AI history. In 1997, IBM's Deep Blue defeated world chess champion Garry Kasparov; in 2016, AlphaGo defeated world Go champion Lee Sedol.

Deep Blue's significance lay in proving that in domains with well-defined rules, machines can surpass humans through sheer computational power. But Deep Blue's approach was entirely different from today's large models — it primarily relied on search algorithms: analyzing vast numbers of possible board positions in advance, evaluating how opponents might respond after each move and how the game would develop, then selecting the option with the highest win probability based on preset evaluation rules.

Specifically, Deep Blue's core technology was the Minimax Algorithm combined with Alpha-Beta pruning. In the game tree, each node represents a board state, and each edge represents a move. Deep Blue could evaluate approximately 200 million positions per second, achieving this speed through dedicated hardware chips running parallel computations. Its evaluation function was designed with input from chess grandmasters and contained over 8,000 feature parameters for assessing position quality. This approach was essentially exhaustive — traversing as many possible move combinations as possible within a limited search depth, then choosing the most advantageous move.

This method worked for chess because the game has well-defined rules: how pieces move and the impact of each move can all be predefined in the program. Chess has an average of about 35 legal moves per turn, and reaching grandmaster level only requires a search depth of around 12 moves. However, Go has approximately 250 legal moves per turn, causing the search space to expand exponentially — which is precisely why AlphaGo had to introduce entirely new methods like deep learning and Monte Carlo Tree Search (MCTS) to conquer Go.

Many real-world tasks similarly lack well-defined rules — writing an article, analyzing complex information, determining whether an image contains a cat. Traditional AI's logic of "humans define rules first, then machines compute according to rules" hit a bottleneck, which is exactly why AI entered its next phase.

Machine Learning: Letting Machines Learn from Data

The core idea behind machine learning is: instead of telling machines all the answers, let them discover hidden patterns through large amounts of data. Take cat-vs-dog recognition as an example. The traditional approach required manually summarizing features like "cats have pointed ears, dogs are larger," but in reality, cats and dogs appear at countless angles and in diverse environments — rules simply can't cover everything. Machine learning instead feeds massive amounts of data directly to the machine and lets it generalize on its own.

Machine learning has three classic paradigms:

Supervised Learning

Provide the machine with labeled data — essentially "problems + answers." It's like a student studying math by reviewing many example problems with standard solutions. Image A is labeled as a cat, Image B as a dog, and the model continuously learns the relationship between images and labels until it can correctly predict new images. Spam detection, financial risk management, and similar systems heavily use supervised learning.

From a mathematical perspective, supervised learning is a function approximation problem. Given a set of mapping samples between input space X and output space Y, the model's goal is to learn a function f such that f(x) approximates the true y as closely as possible. This is achieved by minimizing a Loss Function — which measures the gap between the model's predictions and the actual values. Common loss functions include Mean Squared Error (MSE) for regression tasks and Cross Entropy for classification tasks. The model continuously adjusts its internal parameters through the Gradient Descent algorithm to progressively reduce the loss. In practice, one of the biggest challenges in supervised learning is data labeling costs — for example, the ImageNet dataset contains over 14 million manually labeled images, requiring enormous human effort. This also spurred the development of semi-supervised learning and self-supervised learning methods that reduce labeling dependency.

Unsupervised Learning

The data has no labels and no answers; the machine finds patterns on its own. For example, given massive user data (age, browsing habits, purchase history), the model independently discovers which users exhibit similar behaviors and likely belong to the same group. Large model training also extensively borrows from this concept: feeding models massive text corpora and having them learn language patterns through tasks like "predict the missing content" (e.g., "Tonight I'm going to ____").

Reinforcement Learning

The core idea is adjusting behavior through "trial and error + feedback," similar to a child learning to walk — fall down, adjust; succeed, get positive reinforcement.

In the large model domain, reinforcement learning plays a critical role. After ChatGPT completes its base training, the model has acquired extensive knowledge but doesn't know what kind of answers better meet user needs or what expressions are safer. Human feedback is then introduced for optimization — this is the commonly heard RLHF (Reinforcement Learning from Human Feedback) — teaching the model not just to know answers, but to answer better.

The complete RLHF training pipeline consists of three stages. The first stage is Supervised Fine-Tuning (SFT): collecting high-quality human-written Q&A pairs to fine-tune the pretrained model, giving it basic conversational ability. The second stage is training a Reward Model: having the model generate multiple answers to the same question, then having human annotators rank these answers, and training a reward model to simulate human preferences. The third stage uses the Proximal Policy Optimization (PPO) algorithm, using the reward model's scores as feedback signals to further optimize the language model's output policy. The key insight of this pipeline is that humans are better at comparing which of two answers is better than writing a perfect answer from scratch, so using rankings instead of direct writing significantly reduces annotation difficulty. Subsequent teams like DeepSeek proposed methods such as GRPO that further simplify this pipeline by eliminating the need for a separate reward model.

Deep Learning and Transformer: The Technical Foundation of Large Models

As tasks like image recognition, speech understanding, and natural language processing grew dramatically in information volume, traditional machine learning hit bottlenecks, and deep learning emerged. It's an important branch of machine learning, with neural networks as its core technology — a computational model inspired by the way neurons connect in the human brain.

Neural networks connect large numbers of computational nodes, with information passing through multiple layers: when recognizing images, the first layer learns edges and colors, the second learns shapes and structures, the third learns object compositions. "Deep" refers to the number of layers — the more layers, the richer the hierarchical information the network can learn. To solve information loss in deep networks, researchers proposed residual connections, allowing information to skip certain layers and pass through directly.

Residual connections were introduced by Kaiming He and colleagues in the 2015 ResNet paper, marking a milestone in deep learning history. In deep neural networks, information gradually attenuates as it passes through successive layers, and during backpropagation, gradients become extremely small (the vanishing gradient problem), making it nearly impossible to update parameters in shallow layers. The concept behind residual connections is remarkably simple: add the layer's input directly to its output, i.e., output = F(x) + x. This means even if the increment F(x) learned is very small, information can still pass losslessly through the x "shortcut." This design made training networks with over 100 or even 1,000 layers possible and was the key enabler for Transformer architectures to stably stack dozens of layers.

In 2017, eight researchers at Google published the paper Attention Is All You Need, introducing the Transformer architecture — the critical foundation of modern large models. The core challenge it solved was enabling machines to understand contextual relationships in human language.

For example: "Apple released a new phone, and it was well-received by many users." What does "it" refer to? Humans easily figure this out from context, but traditional computers just see "it" as a character. Transformer's Attention mechanism allows the model to attend to relevant contextual information while processing each word — when processing "it," the model attends to "Apple phone" earlier in the text, establishing accurate semantic connections.

The complete Transformer architecture consists of an Encoder and a Decoder, but different large models choose different combinations: the GPT series uses only the decoder, BERT uses only the encoder, and T5 uses both. The specific implementation of the attention mechanism is Self-Attention: for each word in the input sequence, it computes relevance scores with all other words, generating three sets of vectors — Query, Key, and Value — and computing attention weights through dot product operations. To capture semantic relationships across different dimensions, Transformer uses Multi-Head Attention — splitting the attention computation into multiple parallel subspaces, with each "head" focusing on different types of semantic associations. Additionally, since Transformer doesn't inherently have sequential order information like RNNs, it introduces Positional Encoding to inform the model of each word's position in the sentence.

More importantly, compared to traditional Recurrent Neural Networks (RNNs) that tend to "forget" earlier content when processing long texts, Transformer can simultaneously analyze multiple parts of the text through its attention mechanism, with attention at all positions computed in parallel, making training efficiency far superior to RNNs. This makes it possible for models to handle longer, more complex information. Researchers discovered that continuously scaling up — adding more training data and increasing compute — would continuously improve model capabilities. This was the foundation for the large model explosion.

Generative AI and Large Models: From Judgment to Creation

Large Language Models (LLMs) are AI systems trained on massive text data that can understand and generate natural language, producing corresponding answers to your questions.

Looking back at AI's development reveals an obvious shift: past AI was mostly about "classification and judgment" (Is this spam? Is the image a cat?), while the biggest breakthrough of recent generative AI is "creating content from scratch" — generating articles, code, and images. Previously, getting computers to write essays was extremely difficult because writing requires understanding topics, organizing structure, and choosing expressions; now, large models can generate complete content from a simple prompt.

ChatGPT's explosion was no accident — three factors converged behind it: the accumulation of data, the leap in compute power (GPU clusters, etc.), and architectural breakthroughs (Transformer). On the compute side, the computational demands of large model training are staggering. Take GPT-3 as an example: with 175 billion parameters, training consumed approximately 3,640 PetaFLOP/s-days of computation. NVIDIA's A100 and H100 GPUs are the primary hardware for current large model training, and training a GPT-3-scale model typically requires clusters of thousands of GPUs running continuously for weeks or even months. Distributed training techniques are critical enablers, including data parallelism (distributing training data across multiple GPUs), model parallelism (splitting model parameters across different GPUs), and pipeline parallelism (assigning different model layers to different GPUs). GPT-4's training cost is estimated to exceed $100 million — this enormous resource barrier is an important reason why individual developers should focus on the application layer. These three factors together drove the breakthrough from quantitative to qualitative improvements in model capabilities.

Currently, major global large models each have their strengths: OpenAI's GPT series covers the broadest range of scenarios; Anthropic's Claude excels at long-text analysis and logical reasoning, popular among programmers; Google's Gemini is a multimodal model that can process text, images, audio, and video. In China, notable models include Alibaba's Qwen (open source), Baidu's ERNIE series, and DeepSeek, among others.

How Ordinary People Can Get Started: Build Applications, Don't Build Models

For ordinary people wanting to enter the AI field, it's important to clarify your positioning. If the goal is to become a fundamental researcher developing new model architectures, you'll need deep expertise in mathematics, statistics, deep learning algorithms, and multiple other disciplines — the barrier to entry is extremely high.

But for the vast majority of application developers, the goal isn't to reinvent large models, but to learn how to build practical applications on top of existing large models — such as building enterprise knowledge bases, developing intelligent customer service systems, or constructing automated Agent systems. It's like after the automobile was invented: most people don't need to study engine mechanics; they just need to learn to drive to improve their travel efficiency.

From hands-on experience testing various large model applications, large models are currently best suited not for completely replacing a job role, but rather as highly efficient assistants that help us handle large amounts of repetitive work, freeing up our time for goal setting, decision-making, and creative work.

Looking ahead, large models have three clear development directions: first, continuously strengthening capabilities (reasoning, code, multimodal); second, moving toward industry specialization (vertical large models for finance, law, healthcare, etc.); and third, Agent systems — which not only answer questions but can understand objectives (perception), formulate plans (planning), and invoke tools to execute tasks, forming an intelligent closed loop.

Agent systems represent the most cutting-edge form of current AI applications. Their core architecture typically includes four modules: a perception module (receiving user instructions and environmental information), a memory module (short-term memory storing current conversation context, long-term memory storing historical information via vector databases), a planning module (decomposing complex tasks into executable sub-steps, using methods like ReAct, Chain-of-Thought, and other reasoning frameworks), and a tool-calling module (invoking external tools via APIs such as search engines, code interpreters, database queries, etc.). Unlike simple Q&A-style AI, Agents can form a "perceive-think-act-observe" loop: after executing a step, they observe the results and adjust their next plan accordingly.

For example, when asked to analyze a company's sales performance, an Agent can automatically read data, analyze trends, generate reports, and offer recommendations. Representative Agent frameworks like LangChain, AutoGPT, and MetaGPT have already built relatively complete development ecosystems. This autonomy enables Agents to handle multi-step, cross-system complex tasks, making them the most noteworthy form of AI application in the years ahead.

Key Takeaways