5 Core Modules Every AI Engineer Must Know

Why These Modules Keep Reappearing

From ChatGPT to Waymo's self-driving cars, most of today's AI products aren't invented from scratch — they're assembled from a recurring set of core components. Once you understand these "building blocks," you can quickly grasp the skeleton of any AI system. This article is based on a systematic breakdown from a YouTube tech channel, unpacking five core modules: large language models, embedding models, retrieval-augmented generation, agent harnesses, and world models.

These modules don't exist in isolation — a coding agent is an LLM plus a harness, an internal customer service bot is very likely a RAG system, and recommendation systems are almost entirely driven by embedding models. Master them, and you've got a universal toolbox for dissecting AI products.

1. Large Language Models: The Brain of the System

An LLM is a neural network built on the Transformer architecture, and its core capability is predicting the next word given a sequence of words. It's precisely this capability that powered the early ChatGPT experience.

But "predicting the next word" is only the surface. The real value of an LLM lies in its ability to process context, understand meaning, and decide what to do next — it's the "brain" of the entire AI system. That's why, among the five modules, the LLM sits at the very core.

The Three Stages of Training a Frontier LLM

A frontier model typically comes into being in three steps:

• Pre-training: The model reads trillions of words from the internet and acquires grammar, facts, and foundational knowledge.
• SFT (Supervised Fine-Tuning): By learning from curated Q&A examples, the model learns to "actually answer questions" rather than just continue text.
• RL (Reinforcement Learning): The model generates multiple responses and is weighted toward answers humans prefer — essentially repeated practice.

The division of labor across these three stages is worth remembering: pre-training gives "knowledge," SFT gives "conversational format," and RL gives "alignment with human preferences."

The Transformer architecture is the neural network structure proposed in the 2017 Google paper Attention Is All You Need, whose core is the self-attention mechanism: every word in a sequence can directly attend to all other words and dynamically compute their relative weights. Compared with recurrent neural networks (RNNs), which must process words one by one in sequence, self-attention can compute over the entire sequence in parallel, greatly improving training efficiency and enabling the model to capture long-range dependencies. Mainstream large models such as GPT, Claude, and Gemini are all built on Transformer's decoder structure, and "predicting the next word" is achieved step by step by stacking dozens of layers of attention and feed-forward networks. Understand this, and you'll see why context length and compute scale directly determine the boundaries of a model's capabilities.

The difference between SFT and RLHF is often confused. SFT (supervised fine-tuning) uses high-quality human-written "question–answer" pairs to perform standard next-word prediction training; it's still imitation at heart, switching the model from "continuing web pages" to "conversational responses." RL (here usually referring to reinforcement learning from human feedback, RLHF) doesn't directly tell the model the correct answer; instead, it has the model generate multiple candidate responses to the same prompt, has humans (or a reward model) label which is better, and then uses reinforcement learning algorithms (such as PPO or DPO) to adjust the model's parameters so it favors producing the preferred responses. The former teaches "how to answer," the latter teaches "how to answer better" — only by combining the two do you get the usable conversational models of today.

2. Embedding Models: Making Search Possible

Embedding models are also neural networks, but their training objective is completely different — they don't predict the next word; they map inputs into a high-dimensional space (the embedding space). The training data consists of millions of sample pairs: related samples end up close to each other in the space, while unrelated ones end up far apart.

This property directly gives rise to search and retrieval capabilities. At runtime, mapping a user query into the same embedding space lets you find semantically similar data. Google Search is a classic example: search for "cheap flights to Europe," and pages about affordable airfare still show up, because the query and the pages are mapped into the same space.

The recommendation systems of YouTube and TikTok are likewise built on embedding models under the hood. Understand this, and you understand the foundation of modern search and recommendation.

3. Retrieval-Augmented Generation (RAG): Bolting External Knowledge onto the Model

LLMs are knowledgeable, yet they can't answer specific questions about your internal documents. RAG fills this gap: first, use an embedding model to build your internal documents into a knowledge base; at runtime, embed the user query and retrieve the relevant chunks, then hand "query + retrieved content" together to the LLM to generate an answer.

Notebook LLM works exactly this way — you upload documents and ask questions, and the answers come only from those files, with citations to their sources. RAG is an efficient, low-cost way to give a model knowledge it was never trained on and to build truly customized systems that fit your data.

The retrieval step of RAG usually relies on a vector database. Documents are split by the embedding model into chunks and converted into vectors, which are stored in a purpose-optimized vector database (such as Pinecone, Weaviate, Milvus, or pgvector). At runtime, the system performs approximate nearest neighbor search (ANN, commonly using index algorithms like HNSW) on the query vector, recalling the most similar entries from millions of chunks within milliseconds. Retrieval quality depends heavily on the chunking strategy and the choice of embedding model: chunks that are too large carry too much noise, while chunks that are too small lose context. You can also add a rerank model to re-score the recalled results, further improving the relevance of the context handed to the LLM.

4. Agent Harness: Giving the LLM Hands and Feet

LLMs can answer most questions, but they're powerless in scenarios that require real-time information — say, this weekend's sports events. Many queries require web search or code execution, and the agent harness is the system layer built around the LLM: it safely executes functions, calls external tools, passes results back to the LLM, and loops until the model produces a final answer.

The harness is also responsible for managing the model's limited context window, maintaining memory across sessions, and determining which operations require human approval. The core value of powerful agents like Claude Code and ChatGPT lies precisely in this system layer built around the LLM — not in the model itself.

Function calling (tool use) is the key mechanism that makes an agent harness work. Developers declare to the model, via a structured schema, the names, purposes, and parameter formats of the available tools; during reasoning, if the model judges that external capability is needed, it outputs a call request conforming to the schema rather than ordinary text. The harness intercepts that request, actually executes the corresponding function (such as searching, querying a database, or running code), and then passes the result back to the model as a new message, and so on in a loop. The model itself never actually executes any operation — it only "decides what to call," while the actual execution and safety boundaries are entirely controlled by the harness. This is precisely the technical reason the original article emphasizes that an agent's core value lies in the system layer rather than the model itself.

5. World Models: Predicting the Environment's Next Step

LLMs can predict the next word, but they don't necessarily understand physical laws and object interactions, and they can't produce the environment's next state. World models are another kind of neural network, trained to predict what will happen next in an environment, usually presented in video form.

During training, a world model watches massive amounts of video and predicts subsequent frames, thereby learning how objects move and interact. At runtime, it takes the current state (such as the last few frames) and optional actions (such as "move right") as input, and outputs the next few frames. DeepMind's Genie 3 generates interactive 3D worlds you can roam in real time, while NVIDIA's Cosmo models let robots and self-driving cars train in simulation before entering the real world.

Putting It Together: The Assembly Logic of AI Systems

The significance of the five modules lies in their combinations. Coding agent = LLM + agent harness; robot = world model + LLM; recommendation system = embedding model; internal customer service bot ≈ RAG system. Put these modules in your toolbox, and building AI applications becomes far more manageable.

The original author emphasizes that these modules will "reappear again and again" in real AI systems. Rather than chasing every new framework, it's better to thoroughly master these five building blocks first — they are the stable knowledge that transcends product cycles.

One-line Summary

Breaking down the five core modules that make up today's AI products and the logic of how they combine.

Paragraph Summary

The article systematically breaks down the five core components that recur across today's AI products: the large language model as the system's "brain" (trained in three stages: pre-training, SFT, and RL), embedding models that map inputs into high-dimensional space to power search and recommendation, retrieval-augmented generation (RAG) that bolts internal knowledge with citations onto the model, the agent harness that executes tool calls and manages context around the LLM, and world models that predict the environment's next state. The author stresses that these modules don't exist in isolation — coding agents, customer service bots, and recommendation systems are all combinations of them. Rather than chasing every new framework, it's better to thoroughly master these five "building blocks" and thereby gain stable knowledge that transcends product cycles.