Prompt Engineering in Practice: A Programmer's Core Methodology for Tuning Prompts

Prompt Engineering: The Programming Language of the AGI Era

As generative AI rapidly spreads today, Prompt Engineering has evolved from a niche topic into a fundamental skill that almost everyone needs to master. There's an elegant analogy that captures its essence perfectly: Prompts are the programming language of the AGI era.

Background on the Concept of AGI

AGI (Artificial General Intelligence) refers to AI systems that can match or surpass human performance across any cognitive task, as opposed to the current "Narrow AI." There is significant disagreement in academia over whether AGI has been or will soon be achieved. Organizations like OpenAI and DeepMind list it as a core goal, but most researchers believe that current large language models remain highly complex pattern-matching systems, still fundamentally distant from true AGI. Nonetheless, the phrase "AGI era" has been widely used in industry to describe the paradigm shift brought about by large models. The discipline of Prompt Engineering itself only began to systematize after the release of GPT-3 in 2020, and quickly went mainstream after ChatGPT's explosive popularity in 2022, with "prompt engineer" briefly becoming one of the hottest jobs in Silicon Valley.

Traditional programming languages serve to control computers, making machines work according to our requirements. In the AGI era, prompts play exactly this role—using natural language instructions to control AI and make models produce results according to our intentions. Therefore, those who specialize in debugging prompts are called "prompt engineers," and in a sense, they are the programmers of this new era.

A prompt, put simply, is the sentence you send to a large model: asking it to tell a joke, write a love letter, code a game, or explain an error message. It seems extremely simple—"if you can talk, you can use it"—but hidden behind this simplicity lies tremendous depth.

Low Barrier, High Ceiling

Large model technology has a core characteristic: low barrier, high ceiling. You can learn to use it in a few minutes, but truly mastering prompts to perfection is extremely difficult. This is why some半-jokingly call prompts "incantations"—how well you "chant your spell" directly determines how effectively the AI works for you.

Interestingly, even OpenAI's Sam Altman admits that prompt engineering may not become a long-lasting standalone profession. As AI continues to evolve, writing prompts will become easier and easier, until eventually everyone can do it. This raises a key question: if everyone can do it, what advantage do we gain by learning it?

The Programmer's Unique Advantage: Understanding Principles and Programming

This is the core perspective that distinguishes this from the vast majority of prompt engineering courses on the market. Many prompt courses and books target people who don't know how to program, presenting the content in a rather "mystical" way—essentially "blind guessing" without understanding the underlying principles.

Those who truly possess a moat are people who simultaneously have the following two capabilities:

Advantage One: Understanding the Principles

The core operating mechanism of large models is generating the next token based on probability—simply put, continuously selecting the token with the highest probability and stitching them into complete sentences. Understanding this, you can truly grasp:

• Why some instructions work and others don't;
• Why the same instruction sometimes works and sometimes fails;
• How to systematically increase the probability of instructions being effective.

Deep Dive into the Token Mechanism and Generation Parameters

Tokens are the basic units through which large models process language, sitting between characters and words. In English, one token corresponds to roughly 3/4 of a word; in Chinese, one character typically corresponds to 1-2 tokens. The context windows of mainstream models like GPT-4 are measured in tokens (e.g., 128K tokens). Understanding tokens helps design prompts more efficiently and avoid information truncation caused by exceeding the context length.

The generation process of a large model is essentially a conditional probability chain: at each step, it samples the next token from the vocabulary (typically containing 50,000 to 100,000 tokens) conditioned on the existing sequence. This process is regulated by two key parameters—Temperature and Top-p. Higher temperature makes outputs more random and diverse; lower temperature makes outputs more conservative and deterministic. When Temperature is 0, the model almost always selects the highest-probability token, suitable for precision tasks like code generation and data extraction; when Temperature approaches 1, outputs become more creative, suitable for copywriting and brainstorming. Top-p (also known as Nucleus Sampling) controls output diversity by limiting the cumulative probability range of candidate tokens, and used in conjunction with Temperature, enables finer output tuning. Prompts with strict constraints and rich information can maintain output quality even at higher temperatures—this is precisely the value of understanding the principles.

Additionally, there are two often-overlooked parameters—Frequency Penalty and Presence Penalty. The former penalizes tokens that have already appeared frequently, suppressing repetitive wording; the latter applies a fixed penalty to any token that has already appeared, encouraging the model to explore new topics. Understanding how these parameters interact is a key step from "being able to use AI" to "mastering AI"—together they form an adjustable probability control panel, whereas those who don't understand the principles can only grope blindly in a black box.

There's an important insight here: we can never make instructions 100% effective. Comparing AI to humans—humans make mistakes, and large models certainly have a probability of erring too. What we can do is try every means to increase the success rate, rather than pursue absolute reliability.

Advantage Two: Understanding Programming

Some say "AI can already program, so programming skills are no longer important," but the truth is exactly the opposite—AI being able to program actually makes programming skills more important.

The key lies in judgment: which tasks are more efficiently solved with prompt engineering, and which are more efficiently handled with traditional programming. This judgment directly affects the system's cost, response time, and even its success or failure. AI must connect and collaborate with numerous computer systems to deliver value; without integrating with business systems, it can only "babble out words" and still requires manual operation, making it extremely inefficient.

This also determines two application positionings for prompt engineering:

1. Obtaining specific results for specific problems—for example, directly asking "how do I fix this code error," which can be done through a graphical interface and easily mastered by anyone;
2. Solidifying prompts into programs, making them part of system functionality—such as automatically generating a company briefing every day, building an AI customer service system, or doing Q&A based on an enterprise knowledge base.

RAG: The Mainstream Architecture for Enterprise Knowledge Base Q&A

The "enterprise knowledge base Q&A" in the second type of application is currently mainly implemented through RAG (Retrieval-Augmented Generation). Its core idea is: before generating an answer, first retrieve document fragments relevant to the question from an external knowledge base, inject them as context into the prompt, and then have the large model synthesize the answer.

The value of RAG manifests across three dimensions: First, it breaks through the knowledge cutoff date limitation of large models, allowing the system to access the latest data; second, it anchors the model's answers to verifiable documents, significantly reducing the risk of hallucination—the problem of models fabricating facts out of thin air; third, compared to full fine-tuning, RAG requires no retraining of the model, greatly reducing deployment costs.

Building a RAG system requires three core components working in concert: a vector database (such as Pinecone, Chroma, Milvus) for storing and retrieving semantic vectors of documents; an Embedding model (such as text-embedding-ada-002) responsible for converting text into high-dimensional vectors, so that semantically similar texts are closer in vector space; and the LLM, responsible for the final answer generation. In actual implementation, engineers also need to handle details such as document chunking strategies (Chunking), similarity retrieval tuning, and prompt template design—these are all typical scenarios that "only those who understand programming can command," and also where prompt engineering truly generates commercial value.

It's worth mentioning that RAG is not a silver bullet. When the knowledge base scales beyond the million-document level, retrieval precision noticeably degrades; for questions requiring compound reasoning across multiple documents, a single retrieval is often insufficient. To address this, the industry has developed advanced variants such as Graph RAG (retrieval based on knowledge graphs) and Agentic RAG (multi-step autonomous retrieval). Understanding the applicable boundaries of these architectures is precisely the core competitive advantage that programming-savvy engineers have over ordinary users.

The latter is where those who understand programming hold their unique advantage, and where prompt engineering truly generates commercial value.

Prompt Tuning: Essentially Iteration, Not One-Shot Creation

Using "tuning" rather than "writing" to describe this reveals an important essence: writing prompts is not accomplished in one stroke, but rather is an iterative process of repeated debugging and continuous experimentation, gradually approaching the optimal effect.

Cater to Its Preferences: Reference the Model's Training Data Style

The ideal state of tuning is to understand the model's training data style, then write prompts with reference to it. The reasoning is straightforward: the model learned from this kind of data, so it adapts best to this language style, and naturally performs best.

Understanding this principle through "catering to preferences" is very intuitive: if you know someone loves reading Dream of the Red Chamber, you talk to them about it; if you know they've worked at Alibaba for ten years, you speak in Alibaba jargon; if you know they're an anime fan, you praise them as "kawaii." Interacting with AI follows the same logic.

But the reality is that we usually cannot see the training data of a base large model (unless we do fine-tuning ourselves).

Fine-tuning: A Deeper Customization Method Than Prompts

Fine-tuning is a technique that further trains a pre-trained large model with domain-specific data, making the model better adapted to specific tasks or styles. Compared to prompt engineering, fine-tuning can permanently change model behavior at the parameter level, with more lasting and stable effects, but at the cost of requiring substantial labeled data and considerable computational resources.

The technical forms of fine-tuning are also constantly evolving. Traditional full fine-tuning requires updating all model parameters, which is extremely costly for a model with hundreds of billions of parameters; whereas parameter-efficient fine-tuning methods (PEFT, Parameter-Efficient Fine-Tuning) such as LoRA (Low-Rank Adaptation) inject low-rank decomposed adapter layers alongside the original weight matrices, training only a small number of new parameters (typically less than 1% of the original model's parameter count), greatly lowering the barrier and making it affordable even for small and medium enterprises. OpenAI's Fine-tuning API allows developers to fine-tune GPT-3.5/GPT-4o-mini with their own data, making its output better conform to specific corporate tones or professional domain standards.

In practice, prompt engineering and fine-tuning are not mutually exclusive: it's generally recommended to first validate requirements with prompt engineering, and only consider investing in fine-tuning after confirming clear benefits. Understanding the boundaries of both is precisely the core advantage that those who understand the principles have over those who "blindly guess." There's also a solution that sits between the two—System Prompt caching (such as Anthropic's Prompt Caching feature), which pre-caches fixed system instructions, saving token costs on each call while maintaining a consistent behavioral baseline—a cost-effective choice for moderate customization needs.

At this point, we can leverage two paths:

• Reference the public recommendations in official documentation: OpenAI's GPT series is particularly friendly to Markdown format, and organizing prompts in Markdown yields better results; while Anthropic's Claude is friendly to XML format, and officially recommends organizing prompts in XML to improve performance.
• Continuous experimentation and validation: Often, adding or removing a single word, or swapping a synonym, can have a very large impact on generation probability. Because the model predicts subsequent text based on preceding text, any single-word change alters the subsequent generation probability. In comparison, punctuation has a smaller weight of influence.

Why Are Markdown and XML Formats Effective?

The effectiveness of these two formats is rooted in the distribution patterns of large model training data. Markdown is a lightweight markup language that gives plain text structured semantics through symbols like #, **, and -, widely used on platforms like GitHub README files, technical blogs, and Stack Overflow. The pre-training corpora of GPT series models contain a large amount of such content, so the model is "naturally familiar" with structured instructions in Markdown format, and can more accurately recognize the semantic relationships between heading levels, list items, and emphasized content.

The advantage of XML (Extensible Markup Language) lies in the explicitness of boundaries. Through custom tags such as <system_prompt>...</system_prompt> and <user_input>...</user_input>, content with different semantic roles can be clearly separated, effectively eliminating semantic ambiguity between paragraphs. Anthropic's official documentation explicitly recommends using XML tags in Claude's prompts to distinguish role settings, background information, and specific task instructions, which has been empirically shown to significantly improve output consistency for complex multi-step tasks.

It's worth noting that format preferences change with model version iterations. Which format to choose is essentially about leveraging prior knowledge of the model's training data distribution—the technical implementation of the "cater to preferences" logic, and an important reason to continuously follow official documentation updates. Additionally, from an information theory perspective, structured formats essentially reduce the ambiguity entropy of prompts: clear boundaries mean the model needs to spend less of its "attention budget" on disambiguation when parsing instructions, thereby concentrating more computational resources on the execution of the task itself.

The Core of High-Quality Prompts: Specific, Rich, Low-Ambiguity

This is one of the most important insights in all of prompt engineering. The essentials of a high-quality prompt can be condensed into three words: specific, rich, low-ambiguity.

Various prompt "techniques" and "templates" on the market all ultimately satisfy these three points:

• Specific: Make the instruction's goal clear and unambiguous;
• Rich: Provide sufficient information, background, and context;
• Low-ambiguity: Minimize the possibility of multiple interpretations.

As for those mystical tricks like "tapping on the radiator pipe works better" or "hanging a CPU around your neck works better"—as long as you firmly grasp these three words, whether you follow such formulas doesn't really matter that much.

The Essence of Classic Technical Frameworks Is Also These Three Points

Several widely circulated prompt frameworks in the industry, when broken down, are all different expressions of "specific, rich, low-ambiguity":

The CRISPE framework (Capacity, Role, Insight, Statement, Personality, Experiment) increases richness by clarifying the role (Role) and task background (Insight) the model plays, and improves specificity through concrete output requirements (Statement).

Few-shot Prompting is a powerful tool for improving low-ambiguity: by providing 2-5 input-output example pairs in the prompt, the model "induces" the task format from the examples, eliminating comprehension bias more effectively than pure text descriptions. Research shows that for structured tasks like classification and format conversion, Few-shot can improve accuracy by 10%-30% compared to Zero-shot (giving no examples).

Chain-of-Thought (CoT) improves the output quality of complex tasks by guiding the model to "reason step by step." Its core is adding "Let's think step by step" to the prompt or directly providing reasoning process examples, causing the model to decompose long-range reasoning into multiple intermediate steps before arriving at the final answer. This technique was proposed by Google researchers in 2022 and has significant effects on tasks like mathematical reasoning and logical judgment—essentially also reducing the probability of error in a single leap of complex reasoning by "enriching" the intermediate steps.

More cutting-edge variants include Tree-of-Thought (ToT), which allows the model to explore multiple parallel paths during reasoning and conduct self-evaluation, similar to the "multi-solution comparison" thinking style humans use when solving complex problems. The evolutionary trajectory of these frameworks clearly demonstrates: the direction of all technical innovation is to help the model more fully "unfold" its existing knowledge in more ingenious ways, rather than creating new capabilities out of thin air—understanding this can help you more rationally evaluate the endless stream of new prompt techniques.

The Skill Is Built in Everyday Life: Cultivating Through Daily Writing

How do you improve these three abilities? The answer is "the skill is built in everyday life." There's an interesting cultural phenomenon here: the group-chat style that Chinese people are accustomed to is exactly the opposite of writing good prompts.

In group chats, we casually send short sentences and colloquial expressions full of ambiguity—two people chatting on WeChat can generate plenty of misunderstandings. In contrast, the email-writing culture common in Western work settings is closer to training the ability to write good prompts: writing an email requires logical rigor, cause and effect, and background introduction—essentially writing a "mini-essay."

Therefore, you can try treating your daily questions in group chats as practice too, organizing your language according to the standard of "specific, rich, low-ambiguity." This deliberate practice, sustained over the long term, will significantly improve your prompt engineering ability.

Two Questions Left for Reflection

Finally, there are two questions worth pondering deeply:

1. If the underlying large model is switched, do prompts need to be re-tuned? Since different models have different format preferences (Markdown vs XML) and different training data, the answer is clearly not a simple "no need to change." Different models vary in parameter count, training corpus, and alignment methods (RLHF strategies), so a prompt that performs excellently on GPT-4 may need targeted rewriting when migrated to Claude or a domestic large model. This is also why enterprise-level AI systems often need to maintain independent prompt version repositories for each underlying model, and establish systematic evaluation benchmarks to quantify performance differences between different models and different prompt versions.

   RLHF: The Key Technology for Aligning Models with Human Preferences

   RLHF (Reinforcement Learning from Human Feedback) is currently the core mechanism for mainstream large model alignment. Its basic process consists of three steps: First, human annotators rank different model outputs to train a "Reward Model" that predicts human preferences; then a reinforcement learning algorithm (usually PPO) optimizes the language model so that its outputs achieve higher reward scores; finally, through repeated iteration, the model's behavior converges toward human expectations. Mainstream models like ChatGPT, Claude, and Gemini have all undergone alignment training with RLHF or its improved versions (such as Anthropic's Constitutional AI). Precisely because each company's alignment strategy and training data differ, the same prompt produces stylistically distinct outputs on different models—this fundamentally determines the inevitability of needing to re-tune prompts when migrating across models.

   It's worth noting that RLHF is not without limitations. The "Reward Hacking" problem constantly plagues practitioners: models may learn to "please" the reward model rather than genuinely satisfy user needs, for example by giving lengthy, floridly worded but information-sparse answers. To address this, Anthropic's Constitutional AI (CAI) has the model self-critique and revise according to a set of explicit rules, reducing dependence on human annotation to some extent; OpenAI's RLHF + PPO continuously iterates on reward modeling approaches. Understanding these differences helps explain why different models exhibit vastly different "personalities" when facing the same sensitive topics—this "personality" is not random, but a direct reflection of alignment strategy choices.

2. Can prompts be written in dialects? Theoretically feasible—a dialect is also a language to the model, and the key is whether there's a sufficient amount of text in that dialect in the training data, and whether it can be "aligned" with other languages. The multilingual capabilities of large models are highly dependent on the distribution of training corpora: English typically accounts for over 50% in mainstream pre-training datasets (such as Common Crawl, The Pile), while low-resource languages account for very little, causing the model's reasoning and logical abilities in these languages to be significantly weaker than in English. The challenge of dialects (such as Cantonese and Hokkien) is even more complex: not only is the corpus scarce, but there are also issues like inconsistent writing systems and large gaps between spoken and written language. Some research even suggests that for complex reasoning tasks in low-resource languages, having the model "think" in English first and then translate back to the target language can sometimes yield more accurate results—which indirectly confirms the decisive influence of training data distribution on the boundaries of model capabilities, and the same applies to dialects.

These questions precisely confirm that core judgment: prompt engineering has a low barrier and a high ceiling, and everything we learn is to continuously improve our win rate in a world of probabilities.

Key Takeaways