Understand Machine Learning and GPT Principles to Write Better Prompts

Most people's first encounter with a large language model (LLM) is through a chat box. You type a sentence, it gives an answer, and it feels like talking to a well-read colleague. That experience is useful, but it is also the source of most of the frustration developers run into later. The model is not a colleague — it is a statistical system with particular habits. Once you understand those habits, your prompts get better, your Agents get more controllable, and your token bill gets smaller.

Three historical milestones explain everything that follows

No need for a history lesson, but three things are enough to explain most of this article.

In 2017, Google researchers published "Attention Is All You Need," introducing the Transformer architecture. Its core idea — the attention mechanism — lets the model weigh each token in the input against all other tokens, rather than reading strictly left to right. Then OpenAI's GPT series (starting in 2018) proved that training a Transformer on massive amounts of text with a single simple objective — "predict the next token" — leads the model to pick up grammar, facts, code, and reasoning patterns along the way. Finally, ChatGPT in late 2022 added instruction tuning and reinforcement learning from human feedback, turning these models from "autocomplete engines" into "assistants."

The key takeaway: beneath the shiny exterior, it is still next-token prediction. Everything below follows from that.

The attention mechanism (Attention) is the core component of the Transformer. It allows the model, when processing a given token, to dynamically weigh how relevant every other token in the input sequence is to it, and to aggregate information accordingly. This breaks the bottleneck of RNNs, where information must pass strictly left to right and be relayed step by step, enabling long-range dependencies to be modeled directly and making large-scale parallel training possible — which is exactly what allowed the GPT series to scale to hundreds of billions of parameters.

Next-token prediction is a self-supervised training objective: given the preceding text, the model outputs a probability distribution over the vocabulary for each token being the next one, and training means continually increasing the predicted probability of the actual next token. The objective looks simple, but to keep reducing prediction error on massive text corpora, the model must implicitly acquire grammatical structure, factual associations, code logic, and even reasoning steps — which is why it "incidentally" learns those abilities.

Instruction tuning and RLHF are two post-pretraining adjustment stages: first, human-written "instruction–response" samples teach the model to follow the format of a request; then a reward model is trained on human preference rankings over multiple responses, and reinforcement learning nudges the model toward producing more preferred responses. This process does not change the nature of what the model does at runtime — it is still predicting the next token, just steered toward a distribution that better matches human expectations.

What the model is actually doing

An LLM does not "look things up." It has no fact database to consult. During training, its weights are adjusted so that, given a token sequence, it assigns high probability to plausible next tokens. When you enter a prompt, it generates one token at a time, each based on your prompt plus everything it has already generated.

For developers, two consequences matter enormously:

• The model optimizes for "plausible," not "true." A fluent, confident answer and a correct answer are not the same thing, and the model has no built-in ability to tell them apart.
• The model only knows what is in its weights and what is in the current context window. Nothing else. It does not remember yesterday's conversation unless you send it again.

How hallucinations happen

A hallucination is what happens when "the most plausible continuation just isn't true." There are several common causes:

• Gaps in the training data. Ask about an obscure library or a recent API change, and the model's signal is weak. But it still has to produce something, so it generates what that kind of answer usually looks like: a function name that sounds right, an argument that seems plausible.
• Pattern completion overriding factual recall. If every fetchUser function in the training data takes an id, the model will happily invent an id parameter for your custom function.
• Pressure imposed by the prompt. If you ask "What does the autoRetry option do?", you have already presupposed that the option exists. The model tends to follow the premise.
• Decoding randomness. Sampling means a low-probability wrong token sometimes gets chosen, and then every token after it is built on top of it.

Hallucination is not a bug that can be completely fixed in some version. It is a natural property of this mechanism — it can be reduced, but not eliminated. Treat output as a draft that needs verification, not a source you can trust outright.

The "sampling" mentioned in decoding randomness refers to the fact that at each step the model does not always pick the highest-probability token, but instead draws randomly according to the probability distribution. Common approaches include temperature scaling (higher temperature flattens the distribution and makes output more random), top-k (sampling only from the k highest-probability tokens), and top-p / nucleus sampling (sampling only from the smallest set whose cumulative probability reaches a threshold). The purpose is to avoid stiff, repetitive output, but the cost is that low-probability tokens occasionally get selected. Once a wrong token enters the sequence, every subsequent step continues predicting on the premise of it, and the error is carried forward in a "rationalized" way — which is one reason hallucinations tend to snowball in longer answers.

A hypothesis: does stuffing the context make things worse?

Here is an idea worth discussing. I must be frank: this is an informed guess, not an established conclusion, so treat it as a hypothesis you can test in your own work.

Many developers stuff everything they can into the prompt: entire files, long documents, old conversation history, pasted logs. You can call this context stuffing. A more restrained version of the same thing is often called context engineering: deliberately choosing what the model gets to see.

LLMs are trained on enormous amounts of data mixing natural language, programming languages, markup formats, and industry jargon. That breadth is an advantage, but it also means many tokens are ambiguous. "Table" could be furniture, a database table, or an HTML element; "class" could be a school class, a CSS class, or an object-oriented class; the identifier order could be a purchase, a sort direction, or a command. A variable name in one language may look like a word in another. Mixed-language comments, English prose wrapping Python, JSON, and shell output, all crammed into a single window.

The more heterogeneous material you dump in, the more room the model has to grab the wrong meaning of a token or blend patterns from different domains together. Add stale or contradictory snippets, and the model can only guess which one you want. A guess packaged as a confident answer looks exactly like a hallucination.

Research provides some support for this guess: studies on long-context behavior (for example, the "lost in the middle" finding) show that a model's use of information varies unevenly depending on where it sits in the context, and that irrelevant material drags down answer quality. As for whether language breadth itself is a driver, I cannot say, but the practical advice holds either way: give the model less, cleaner, more relevant context, and spell out what your terms mean.

"Lost in the middle" is an empirical finding about long-context utilization efficiency: when key information is placed in the middle of the input, the model often retrieves and uses it noticeably worse than when it is placed at the beginning or end. This means that even if the context window is large enough to hold all the material, it does not mean the model will attend evenly to every part. This phenomenon is related to attention allocation and the positional distribution in training data, and it also explains why "just dump the whole document in" often produces vague or missed-content answers. A practical countermeasure is to place the most critical instructions and materials near the beginning or end of the context, and to proactively delete irrelevant content rather than relying on the window capacity itself.

Agent applications: this is where misunderstanding gets expensive

In chat, the cost of a bad answer is one retry. In an Agent application, the model plans, calls tools, reads results, and loops — and a single wrong step can trigger real actions. Understanding how the model works explains most of the surprises.

Why an Agent produces results you did not want, or does things you did not ask it to do:

• Ambiguous goals. "Clean up the project" has many plausible readings. The model picks the statistically most common one, which may include deleting files.
• Overlapping or loosely described tools. The model selects tools by reading descriptions. If two tools sound similar, or a description is too broad, it will pick the wrong one, or call one unnecessarily.
• Excessive tool permissions. If the Agent can write to disk, run shell commands, and access production APIs, then "a plausible next action" includes using those capabilities. Capability invites use.
• Helpfulness training. Instruction-tuned models lean toward doing more. Without clear boundaries, "fix this bug" can turn into "fix this bug and refactor the three nearby functions while you're at it."
• Context drift in long loops. As tool results, errors, and intermediate thoughts pile up, your original instruction gets buried. The model follows the most prominent content, and that may already be a stray log line rather than your goal.
• Error accumulation. A hallucinated assumption in step two becomes a "fact" in the context by step three. The Agent confidently builds on top of it.
• Untrusted content in the context. Text scraped from web pages, emails, or files may contain instructions, and the model has no reliable way to distinguish your instructions from data. This is prompt injection, and it is a direct consequence of "everything is just tokens in the same window."

What helps: state goals and boundaries explicitly (including what not to do), give each tool a narrow, distinct, and precisely described purpose, grant only the minimum permissions needed, require confirmation before destructive operations, restate the original task near the end of the context, and verify tool outputs before they re-enter the loop.

How this understanding helps you save tokens

Tokens are both what you pay for and what fills the context window. Understanding the mechanism saves money in several ways:

• The model is stateless. Every call resends the entire conversation. In a 20-step Agent loop, early context gets paid for over and over. Trimming irrelevant material early compounds into meaningful savings.
• Less stuffing, fewer tokens, and usually better answers. Sending the relevant function instead of the whole codebase costs less and gives the model fewer sources of confusion.
• Fewer retries. A precise prompt that gets it right the first time beats three vague attempts. Every hallucination you have to discover and re-prompt costs you twice.
• Fewer runaway loops. An Agent with a clear scope and explicit stop conditions will not wander into unrequested work — and unrequested work is billable work too.
• Cacheable structure. Many providers offer discounts on repeated prompt prefixes. Put stable instructions at the top and variable content below, and you get that discount.
• Choose the right model size and output. Knowing that output tokens are usually more expensive than input tokens, and that small models are good at simple tasks, lets you request concise formats (like structured JSON) and route simple work to cheaper models.

Tokens and tokenization are prerequisites for understanding cost: the model does not process characters or words directly. It first splits text into tokens — usually subword units. An English word may be split into one or more tokens; Chinese characters often map to one token per character or a few characters per token; code, punctuation, and spaces each count too. Pricing, context window limits, and request lengths are all measured in tokens, not in word or character counts. As a result, the same content expressed in different formats (for example, compact JSON versus verbose prose) will have noticeably different actual token counts. Knowing how your model tokenizes lets you estimate costs more accurately, judge which materials are worth putting into the context, and understand why some content that looks short is actually quite "expensive."

Conclusion

Prompts are not magic spells. They are communication with a "next-token predictor" whose ground truth comes only from training data and your context window. Once you internalize that, a lot of "mysterious" behavior stops being mysterious: hallucinations are plausible-sounding fill-in-the-blanks, Agents step out of bounds because you left room for them to, and your token bill is largely a record of how much unnecessary material you sent.

You do not need a PhD. A few hours spent understanding how Transformers work, how tokenization splits text, how context windows behave, and how sampling works can improve every prompt you write afterward.

If you have seen mixed languages or messy context lead to strange output, feel free to share in the comments. This is exactly the kind of hypothesis that gets better with more real-world cases.