Prompt Engineering in Practice: A Programmer's Core Methodology for Prompt Tuning

Prompt Engineering: The Programming Language of the AGI Era

As generative AI rapidly proliferates today, Prompt Engineering has evolved from a niche topic into a fundamental skill that almost everyone needs to master. There's an elegant analogy that captures its essence perfectly: Prompts are the programming language of the AGI era.

Background on the Concept of AGI

AGI (Artificial General Intelligence) refers to an AI system capable of matching or exceeding human-level performance on any cognitive task, as opposed to today's "Narrow AI." There is significant disagreement in academia over whether AGI has been or will soon be achieved. Organizations like OpenAI and DeepMind list it as a core objective, but most researchers believe that current large language models are still highly complex pattern-matching systems, fundamentally far from true AGI. Nevertheless, the term "AGI era" has been widely adopted in the industry to describe the paradigm shift brought about by large models. The discipline of Prompt Engineering itself only began to systematize after GPT-3's release in 2020, and it rapidly went mainstream following the explosive popularity of ChatGPT in 2022, with "prompt engineer" briefly becoming one of the hottest positions in Silicon Valley.

The role of traditional programming languages is to control computers, making machines work according to our requirements. In the AGI era, prompts play exactly this role—controlling AI through natural language instructions to make models produce results according to our intentions. As a result, those who specialize in debugging prompts are called "prompt engineers," who are, in a sense, the programmers of this new era.

A prompt, put simply, is the message you send to a large model: asking it to tell a joke, write a love letter, code a game, or explain an error message. It seems extremely simple—"if you can talk, you can use it"—but this simplicity hides enormous depth.

Low Barrier to Entry, High Ceiling

Large model technology has a core characteristic: a low barrier to entry, but a high ceiling. You can start using it within minutes, but truly mastering prompts to perfection is extremely difficult. This is why some half-jokingly call prompts "spells"—how well you cast your "spell" directly determines how effectively the AI works for you.

Interestingly, even OpenAI's Sam Altman has admitted that prompt engineering may not become a lasting standalone profession. As AI continues to evolve, writing prompts will become increasingly simple, until eventually everyone can do it. This raises a key question: if everyone can do it, what advantage do we gain from learning it?

The Programmer's Unique Advantage: Understanding Principles and Understanding Programming

This is the core perspective that distinguishes this from the vast majority of prompt engineering courses on the market. Many prompt courses and books target people who don't know how to program, presenting the content in a rather "mystical" way—essentially "blindly guessing" without understanding the underlying principles.

Those who truly possess a moat are the people who simultaneously have the following two capabilities:

Advantage One: Understanding the Principles

The core operating mechanism of large models is generating the next token based on probability—simply put, continuously selecting the token with the highest probability and stitching them together into a complete sentence. Once you understand this, you can truly grasp:

• Why some instructions work and others don't;
• Why the same instruction sometimes works and sometimes fails;
• How to systematically increase the probability of an instruction being effective.

A Deeper Understanding of the Token Mechanism and Generation Parameters

A token is the basic unit through which large models process language, sitting somewhere between characters and words. In English, one token corresponds roughly to 3/4 of a word; in Chinese, one character usually corresponds to 1-2 tokens. The context window of mainstream models like GPT-4 is measured in tokens (e.g., 128K tokens). Understanding tokens helps design prompts more efficiently and avoid information truncation caused by exceeding the context length.

The generation process of a large model is essentially a chain of conditional probabilities: at each step, it samples the next token from the vocabulary (which typically contains 50,000 to 100,000 tokens), conditioned on the existing sequence. This process is governed by two key parameters—Temperature and Top-p. Higher temperature means more random and diverse output; lower temperature means more conservative and deterministic output. When Temperature is 0, the model almost always selects the highest-probability token, which suits precision tasks like code generation and data extraction; when Temperature approaches 1, output becomes more creative, suiting copywriting and brainstorming. Top-p (also known as Nucleus Sampling) controls output diversity by limiting the cumulative probability range of candidate tokens, and combining it with Temperature enables finer output control. A tightly constrained, information-rich prompt can maintain output quality even at higher temperatures—this is precisely the value of understanding the principles.

Additionally, there are two often-overlooked parameters—Frequency Penalty and Presence Penalty. The former penalizes tokens that have already appeared frequently, suppressing repetitive word choices; the latter applies a fixed penalty to any token that has already appeared, encouraging the model to explore new topics. Understanding the interplay of these parameters is a key step in advancing from "knowing how to use AI" to "mastering AI"—together they form an adjustable probability control panel, whereas those who don't understand the principles can only grope blindly inside a black box.

Here's an important insight: we can never make an instruction 100% effective. Think of AI as analogous to a person—people make mistakes, and large models will certainly have a probability of error too. What we can do is find every possible way to improve the success rate, rather than pursuing absolute reliability.

Advantage Two: Understanding Programming

Some say "AI can program now, so knowing how to program isn't important anymore," but the truth is exactly the opposite—AI's ability to program actually makes programming skills more important.

The key lies in judgment: which tasks are more efficiently solved with prompt engineering, and which are more efficiently solved with traditional programming. This judgment directly affects the system's cost, response time, and even its success or failure. AI must connect and collaborate with a vast number of computer systems to realize its value. If it doesn't integrate with business systems, it can only "chatter away spitting out words," still requiring manual operation—which is extremely inefficient.

This also determines the two application positionings of prompt engineering:

1. Obtaining specific results for specific problems—for example, directly asking "how do I fix this code error," which can be done through a graphical interface and is easy for anyone to master;
2. Embedding prompts into programs to become part of a system's functionality—for example, automatically generating a company briefing every day, building an AI customer service agent, or doing Q&A based on an enterprise knowledge base.

RAG: The Mainstream Architecture for Enterprise Knowledge Base Q&A

The "enterprise knowledge base Q&A" in the second category of applications is currently mainly implemented through RAG (Retrieval-Augmented Generation). Its core idea is: before generating an answer, first retrieve document fragments relevant to the question from an external knowledge base, inject them into the prompt as context, and then have the large model synthesize the answer.

RAG's value manifests across three dimensions: First, it breaks through the large model's knowledge cutoff date limitation, allowing the system to access the latest data; second, it anchors the model's answers to verifiable documents, significantly reducing the risk of hallucination—the problem of the model fabricating facts out of thin air; third, compared to full fine-tuning, RAG doesn't require retraining the model, greatly reducing deployment costs.

Building a RAG system requires three core components working in concert: a vector database (such as Pinecone, Chroma, Milvus) for storing and retrieving the semantic vectors of documents; an Embedding model (such as text-embedding-ada-002) responsible for converting text into high-dimensional vectors so that semantically similar text is closer in vector space; and an LLM responsible for the final answer generation. In actual implementation, engineers also need to handle details like document chunking strategy, similarity retrieval tuning, and prompt template design—these are all typical scenarios that "only those who understand programming can command," and where prompt engineering truly generates commercial value.

It's worth mentioning that RAG is not a silver bullet. When a knowledge base scales beyond the million-document level, retrieval precision noticeably degrades; for questions requiring compound reasoning across multiple documents, a single retrieval is often inadequate. To address this, the industry has developed advanced variants such as Graph RAG (knowledge-graph-based retrieval) and Agentic RAG (multi-step autonomous retrieval). Understanding the applicability boundaries of these architectures is precisely the core competitive edge that programming-savvy engineers have over ordinary users.

The latter is where those who understand programming hold their unique advantage, and where prompt engineering truly generates commercial value.

Prompt Tuning: Essentially Iteration, Not One-Shot Creation

Using "tuning" rather than "writing" to describe this reveals an important essence: writing a prompt isn't accomplished in one go—it's an iterative process of repeated debugging and continuous experimentation, gradually approaching the optimal result.

Cater to the Model's Preferences: Reference the Training Data Style

The most ideal state of tuning is to understand the model's training data style, then write prompts by referencing it. The reasoning is straightforward: the model learned from that kind of data, so it's most adapted to that language style and naturally performs best.

Understanding this through the lens of "catering to preferences" is very intuitive: if you know someone loves reading Dream of the Red Chamber, you chat with them about it; if you know they worked at Alibaba for ten years, you speak Alibaba's internal jargon to them; if you know they're an anime fan, you compliment them with "kawaii." Interacting with AI follows the same logic.

But the reality is that we usually can't see the training data of a base large model (unless we do our own fine-tuning).

Fine-tuning: A Deeper Form of Customization Than Prompts

Fine-tuning is the technique of further training a pre-trained large model with domain-specific data to make it better adapted to specific tasks or styles. Compared to prompt engineering, fine-tuning can permanently change model behavior at the parameter level, with more lasting and stable effects, but at the cost of requiring substantial labeled data and considerable computing power.

The technical forms of fine-tuning are also constantly evolving. Traditional full fine-tuning requires updating all of the model's parameters, which is extremely costly for a model with hundreds of billions of parameters. In contrast, parameter-efficient fine-tuning methods (PEFT) like LoRA (Low-Rank Adaptation) inject a low-rank decomposed adapter layer alongside the original weight matrices, training only a small number of newly added parameters (usually less than 1% of the original model's parameter count). This greatly lowers the barrier, making it affordable even for small and medium-sized enterprises. OpenAI's Fine-tuning API allows developers to fine-tune GPT-3.5/GPT-4o-mini with their own data, making the output more aligned with an enterprise's specific tone or professional domain norms.

In practice, prompt engineering and fine-tuning are not an either-or proposition: it's generally advisable to first validate requirements with prompt engineering, and only consider investing in fine-tuning after confirming clear benefits. Understanding the boundaries of both is precisely the core advantage of those who understand the principles over those who "blindly guess." There's also an intermediate option—System Prompt caching (such as Anthropic's Prompt Caching feature)—which pre-caches fixed system instructions, maintaining a consistent behavioral baseline while saving token costs on each call. It's a cost-effective choice for moderate customization needs.

At this point, you can leverage two paths:

• Reference the public recommendations in official documentation: OpenAI's GPT series is particularly friendly to Markdown format, and organizing prompts in Markdown yields better results; while Anthropic's Claude is friendly to XML format, and the official recommendation is to organize prompts in XML to improve performance.
• Continuous experimentation and validation: Very often, adding or removing a single character, or swapping in a synonym, can have a huge impact on generation probability. Because the model predicts subsequent text based on preceding text, any change in a single character alters the probability of subsequent generation. In comparison, punctuation has a relatively smaller weight of influence.

Why Are Markdown and XML Formats Effective?

The effectiveness of these two formats is rooted in the distribution patterns of large model training data. Markdown is a lightweight markup language that gives structured semantics to plain text through symbols like #, **, and -, widely used on platforms like GitHub README files, technical blogs, and Stack Overflow. The pre-training corpora of GPT series models contain large amounts of such content, so the models are "naturally familiar" with Markdown-formatted structured instructions and can more accurately identify the semantic relationships between heading levels, list items, and emphasized content.

XML (Extensible Markup Language)'s advantage lies in the explicitness of its boundaries. Through custom tags like <system_prompt>...</system_prompt> and <user_input>...</user_input>, content with different semantic roles can be clearly separated, effectively eliminating semantic ambiguity between paragraphs. Anthropic's official documentation explicitly recommends using XML tags in Claude prompts to distinguish role settings, background information, and specific task instructions, which has been shown in practice to significantly improve output consistency for complex multi-step tasks.

It's worth noting that format preferences change with model version iterations. Choosing which format to use is essentially leveraging prior knowledge of the model's training data distribution—the technical implementation of the "cater to preferences" logic, and an important reason to keep an eye on official documentation updates. Moreover, from an information theory perspective, structured formats essentially reduce the ambiguity entropy of a prompt: clear boundaries mean the model needs to expend less of its "attention budget" on disambiguation when parsing instructions, thereby concentrating more computational resources on executing the task itself.

The Core of High-Quality Prompts: Specific, Rich, and Low-Ambiguity

This is one of the most important insights in all of prompt engineering. The key points of a high-quality prompt can be condensed into three words: specific, rich, and low-ambiguity.

All the various prompt "techniques" and "templates" on the market ultimately serve to satisfy these three points:

• Specific: Make the instruction's objective clear and unambiguous;
• Rich: Provide sufficient information, background, and context;
• Low-ambiguity: Minimize the possibility of multiple interpretations.

Those mystical tricks like "tapping on the heating pipe works better" or "hanging a CPU around your neck works better"—as long as you firmly grasp these three words, whether or not you follow such formulas really doesn't matter much.

The Essence of Classic Technical Frameworks Is Also These Three Points

Several widely circulated prompt frameworks in the industry, when broken down, are all different expressions of "specific, rich, and low-ambiguity":

The CRISPE framework (Capacity, Role, Insight, Statement, Personality, Experiment) increases richness by clarifying the role the model plays (Role) and the task background (Insight), and improves specificity through concrete output requirements (Statement).

Few-shot Prompting is a powerful tool for improving low-ambiguity: by providing 2-5 input-output example pairs in the prompt, the model "induces" the task format from the examples, which eliminates comprehension bias better than pure text descriptions. Research shows that for structured tasks like classification and format conversion, Few-shot can improve accuracy by 10%-30% compared to Zero-shot (no examples).

Chain-of-Thought (CoT) improves the output quality of complex tasks by guiding the model to "reason step by step." Its core is adding "Let's think step by step" to the prompt or directly providing examples of the reasoning process, so the model breaks down long-range reasoning into multiple intermediate steps before arriving at the final answer. This technique was proposed by Google researchers in 2022 and has significant effects on tasks like mathematical reasoning and logical judgment—essentially, it also reduces the error probability of leaping across complex reasoning in one step by "enriching" the intermediate steps.

More cutting-edge variants include Tree-of-Thought (ToT), which allows the model to explore multiple parallel paths during reasoning and perform self-evaluation, similar to the human "comparing multiple solutions" way of thinking when solving complex problems. The evolutionary trajectory of these frameworks clearly demonstrates: the direction of all technical innovation is to help the model more fully "unfold" its existing knowledge in more sophisticated ways, rather than creating new capabilities out of thin air—understanding this helps you more rationally evaluate the endless stream of new prompt techniques.

The Skill Is Built in Everyday Life: Cultivating It Through Daily Writing

How do you improve these three capabilities? The answer is "the skill is built in everyday life." There's an interesting cultural phenomenon here: the group chat habits that Chinese people are accustomed to actually run completely counter to writing good prompts.

In group chats, we casually send short sentences with colloquial expressions full of ambiguity—two people chatting on WeChat can generate tons of misunderstandings. In contrast, the email-writing culture common in Western work settings is closer to the ability training needed for writing good prompts: writing an email requires rigorous logic, cause and effect, and background introduction—essentially writing a "mini essay."

Therefore, you can try treating your everyday group chat questions as practice, organizing your language by the standard of "specific, rich, and low-ambiguity." This deliberate practice, sustained over the long term, will significantly improve your prompt engineering abilities.

Two Questions Left for Reflection

Finally, there are two questions worth pondering:

1. If the underlying large model changes, do prompts need to be re-tuned? Since different models have different format preferences (Markdown vs XML) and different training data, the answer is clearly not a simple "no need to change." Different models have varying parameter counts, training corpora, and alignment methods (RLHF strategies). A prompt that performs excellently on GPT-4 may need to be rewritten specifically when migrated to Claude or a domestic large model. This is also why enterprise-level AI systems often need to maintain a separate prompt version repository for each underlying model, and establish systematic evaluation benchmarks to quantify performance differences between different models and different prompt versions.

   RLHF: The Key Technology for "Aligning" Models with Human Preferences

   RLHF (Reinforcement Learning from Human Feedback) is currently the core mechanism for mainstream large model alignment. Its basic process has three steps: first, human annotators rank different model outputs to train a "Reward Model" that predicts human preferences; then, a reinforcement learning algorithm (usually PPO) is used to optimize the language model so its output achieves higher reward scores; finally, through repeated iteration, the model's behavior converges toward human expectations. Mainstream models like ChatGPT, Claude, and Gemini have all undergone alignment training with RLHF or its improved versions (such as Anthropic's Constitutional AI). Precisely because each company's alignment strategy and training data differ, the same prompt produces stylistically very different outputs on different models—this fundamentally determines the inevitability of needing to re-tune prompts when migrating across models.

   It's worth noting that RLHF is not without limitations. The problem of "Reward Hacking" has always plagued practitioners: the model may learn to "please" the reward model rather than genuinely satisfy user needs, for example giving lengthy, floridly worded but information-sparse answers. To address this, Anthropic's Constitutional AI (CAI) has the model self-critique and revise based on a set of explicit rules, reducing reliance on human annotation to some extent; OpenAI's RLHF + PPO continues to iterate on reward modeling approaches. Understanding these differences helps explain why different models exhibit vastly different "personalities" when facing the same sensitive topics—this "personality" is not random, but a direct manifestation of alignment strategy choices.

2. Can prompts be written in a dialect? Theoretically feasible—a dialect is also a language to the model, and the key lies in whether there's enough text of that dialect in the training data, and whether it can be "aligned" with other languages. A large model's multilingual capability heavily depends on the distribution of the training corpus: English typically accounts for over 50% of mainstream pre-training datasets (such as Common Crawl and The Pile), while low-resource languages account for very little, causing the model's reasoning and logical abilities in these languages to be significantly weaker than in English. The challenge with dialects (such as Cantonese and Hokkien) is even more complex: not only is the corpus scarce, but there are also problems like inconsistent writing systems and large gaps between spoken and written forms. Some research even shows that for complex reasoning tasks in low-resource languages, having the model "think" in English first and then translate back to the target language can sometimes yield more accurate results—this indirectly confirms the decisive influence of training data distribution on the model's capability boundaries, and the same applies to dialects.

These questions precisely confirm that core judgment: prompt engineering has a low barrier to entry and a high ceiling, and everything we learn is aimed at continuously improving our odds in a world of probabilities.

Key Takeaways