How ChatGPT and AI Agents Actually Work: A No-Math Breakdown of the Core Mechanics

Many people use ChatGPT every day but can't explain how it actually works. A common misconception is that AI "remembers" you. It doesn't. Based on the core content of the popular YouTube explainer video ChatGPT vs AI Agents: How They Actually Work (No Math), this article breaks down the essential differences between large language models and AI Agents in the plainest possible language—without a single math formula.

Large Language Models: A Guessing Game About "the Next Word"

From Text to Numbers: Tokenization

The model has never seen a letter. The text you type is first chopped into small pieces called tokens, and each token is then converted into a number. In a real tokenizer, "the cat sat on the" is split into five pieces, while a long word like "unbelievably" gets broken into three. This step looks mechanical, but it determines the basic granularity at which the model understands language.

Tokens are the smallest unit of text a language model processes, but a token is not the same as a "word" or a "character." Common subword tokenization algorithms (such as BPE, byte pair encoding) start from characters and repeatedly merge high-frequency adjacent pieces, ultimately forming a vocabulary that balances keeping common words intact with allowing rare words to be split apart. As a result, common words like "the" often occupy a single token, while "unbelievably" may be split into three tokens: "un," "believ," and "ably." Different models use different tokenizers and vocabulary sizes (GPT-3's was about 50,000), which directly affects the text granularity the model can "see" and the actual capacity of the context window—the same word count can correspond to very different token counts.

Neural Networks: A Pile of Countless Tiny Calculators

These numbers flow into a neural network. Think of it as layers upon layers of tiny calculators: each calculator takes in numbers, multiplies them by its own settings (called weights), sums them up, and passes the result to the next layer. A single calculator is simple, but a large model has billions of weights—GPT-3 in 2020 had 175 billion. It is this scale of stacking that lets simple multiplication operations give rise to language ability.

Training: Guess Wrong, Adjust a Little

The training process is astonishingly simple: the model reads massive amounts of text, but the next word is hidden, and it has to guess that word. When it guesses wrong, every weight is adjusted a tiny bit so that the correct answer becomes more likely next time. After repeating this billions of times, language patterns and factual knowledge settle into the weights. A second round of training introduces human feedback to teach it to follow instructions like an assistant.

Attention: Letting "bank" Know Whether It's a Riverbank or a Bank

There is also a key internal trick called attention. Each word "looks at" the other words to determine what it means in this particular place. In "river bank," bank looks at river and means riverbank; in "bank account," it looks at account and means money. This is where the T in GPT comes from—Transformer, an architecture design published by Google researchers in 2017.

The core innovation of the Transformer architecture is the self-attention mechanism, which allows each position in a sequence to interact directly with all other positions, thereby computing the relevance weights between words in parallel. This stands in sharp contrast to recurrent neural networks (RNNs), which must process data step by step in order, and it enables Transformers to efficiently use GPUs to train extremely large models in parallel. Google's 2017 paper Attention Is All You Need first proposed this architecture, originally for machine translation, but because of its scalability it became the foundation of almost all large language models since. The "T" in GPT stands for Transformer, while "pre-trained" means the model first learns language patterns from massive amounts of unlabeled text and is then fine-tuned for specific tasks.

Output: A Probability Game of Generating Word by Word

The model outputs a list of candidate tokens, each with a probability attached. For "the cat sat on the," "mat" scores very high. The model picks one, appends it to the end of the text, and runs another round to predict the next one. This is why answers appear word by word, and it is also the root of why the model can sound utterly confident yet still be wrong—it is simply picking high-probability words, and at no point in the entire process does it verify facts.

Chat: No Memory, All Relying on "Rereading the Whole Text"

The model itself has no memory between messages. Every time you send a message, the app resends the entire conversation: the company's hidden instructions, everything you've said, everything it has said, and your new message. The model reads this whole package and predicts a reply. This package is called the context window, and it has a size limit.

This also explains a common everyday confusion: why does the model start to get "fuzzy" late in a long conversation? Because the context window can't hold everything, and earlier information gets squeezed out. The practical advice given in the video is—start a new conversation for a new task and paste in the necessary information, rather than continuing to ask follow-up questions in one extremely long conversation.

The context window refers to the maximum number of tokens a model can process in a single forward computation, including system instructions, historical conversation, and the current input. Window sizes vary significantly across models: early GPT-3 had 2,048 tokens, GPT-3.5 Turbo had 4,096, and later models have expanded to tens of thousands or even hundreds of thousands of tokens. The larger the window, the longer the conversation the model can "remember," but computational cost and memory usage also grow quadratically (because the attention mechanism must compute the relationship between every pair of tokens). When a conversation exceeds the window, applications typically adopt strategies such as truncating early messages or summarizing and compressing them, which explains why models in long conversations omit early information or contradict themselves.

Agents: Tools Plus a Loop, From "Saying" to "Doing"

An Agent has two extra things compared with a chat model: tools and a loop. Suppose you ask it to book a table for six at a restaurant on Friday. The model's reply is not direct execution but a request: check the calendar. The app runs this tool and puts the result back into the conversation; the model reads the result and decides the next step; the loop continues until the task is complete. This is exactly how Anthropic defines an Agent: a model using tools in a loop.

The value of this definition is that it reveals both the boundaries and the risks of Agent capabilities. The model itself still only guesses the next token, but it can now affect the real world through tools. The more tools there are and the longer the loop, the greater the cost of mistakes.

Tool use (tool use / function calling) is the technical foundation for an Agent's ability to act. Developers register available tools with the model using structured descriptions (name, function, parameter schema). When generating a reply, if the model determines that an external operation is needed, it outputs a call request that conforms to that schema rather than natural language. The application layer executes the tool (such as querying a calendar, sending an email, or calling a payment API) and appends the result to the conversation as a new message, based on which the model continues reasoning. This division of labor—"the model decides, the application executes"—means the model itself does not directly touch real systems, and permission control falls entirely at the application layer. This is also why retaining human approval for irreversible operations is a necessary safety design.

Three Habits for Using AI at Work

The video concludes with three practical principles that anyone using AI at work should remember:

First, verify every important fact. The model picks high-probability words, not the truth. Anything involving data, citations, or regulations must be manually checked.

Second, start a new conversation for a new task. The model rereads the whole text every time, and a long conversation will exhaust the context space. Rather than patching an old conversation, it's better to start clean.

Third, authorize Agents strictly. Agents act through tools, so give them only the tools necessary for the task; for irreversible operations such as sending, paying, or deleting, keep human approval in place.

Scaling Simple Steps Is the Source of the "Magic"

To sum up this mechanism in one sentence: a language model guesses the next token, chat sends the entire conversation each time, and an Agent gives it tools and a loop. Each step is simple enough to describe in a single sentence, but when scale (billions of weights) and length (word-by-word generation, looped execution) are combined, they produce an astonishing sense of intelligence. Understanding this lets you use it more clear-headedly—neither over-mythologizing it nor underestimating the risks.