Introduction to Prompt Engineering: A Complete Guide to Core Concepts, Methodologies, and Limitations

In an era where AI has become a productivity tool, knowing how to effectively converse with large models is becoming a core competency. The first lesson in a recent prompt engineering series on Bilibili systematically reviews the fundamental differences between prompts and prompt engineering, their core value, and the limitations they cannot avoid. Based on the course content, this article offers an in-depth breakdown of the foundational understanding of prompt engineering.

What Is a Prompt: The Most Direct Way to Control AI

The definition of a prompt is actually quite simple—any text you send to the AI, whether a question or a description, is a prompt. The course uses a vivid analogy: a prompt is like a "neural signal" you send to the AI. Whatever signal you send, the AI thinks in that direction.

To understand the technical logic behind this analogy, you need to understand how large language models (LLMs) fundamentally work. An LLM is essentially a statistical prediction system trained on the Transformer architecture. The Transformer was introduced by Google in the 2017 paper Attention Is All You Need, and its core innovation is the "self-attention mechanism." When each Token is processed, it generates three sets of vectors—Query, Key, and Value. By computing the dot product of the Query with all Keys, attention weights are obtained, which are then used to compute a weighted sum of the Values. This allows the model to dynamically focus on the most relevant contextual information in the sequence when processing each word. This mechanism enables the model to process the entire input sequence in parallel, capturing semantic dependencies at any distance—the fundamental reason LLMs can understand complex context. The model learns the probabilistic relationships between words from massive amounts of text. When you input a prompt, the model doesn't truly "understand" the meaning; rather, it predicts the most likely next Token based on the context.

A Token is the atomic unit through which large models process text—it is not simply equivalent to a "character" or "word." Modern tokenizers typically use the BPE (Byte Pair Encoding) algorithm, repeatedly merging the most frequently occurring adjacent byte pairs in the corpus until a preset vocabulary size is reached, thereby splitting high-frequency subwords into individual Tokens. On average, an English word corresponds to about 0.75 Tokens, while a Chinese character corresponds to roughly 1-2 Tokens due to different character set densities. This difference has a direct economic impact in practice: at GPT-4o's pricing of about $5 per million input Tokens, a Chinese prompt containing the same amount of information typically consumes more Tokens than an English one. This cost difference is especially significant in enterprise scenarios involving high-frequency batch calls, and it directly affects the actual usable capacity of the context window. Different word orders and expression frameworks activate different "knowledge pathways" in the model's weights, guiding it toward drastically different answers. This is precisely why the same question, phrased differently, yields completely different results.

Why is it that with the same AI, some people can only use it to write elementary school essays, while others can generate professional industry reports? The difference lies not in the AI itself, but in the quality of the input prompt. A prompt is the most direct means we have to control AI—bar none.

The course summarizes the role of prompts in four points, each worth understanding:

• Communication Bridge: Translating the vague needs in your mind into clear instructions the AI can understand. Just like ordering takeout—you can't just say "bring me some food." You need to specify the dish, spice level, and whether you want extra rice.
• Task Navigation: Giving the AI a clear goal and boundaries. If you say "help me deal with this file," the AI has no idea whether you want statistical analysis, visualization, or a summary, so it can only pick one at random.
• Style Control: By default, the AI outputs a neutral, generic style. Through prompts, you can make it switch to a Lu Xun style, a schoolchild's tone, or rigorous academic language.
• Boundary Constraints: This is one of the most effective ways to solve AI "nonsense." By restricting it to focus only on a specific data range and avoid irrelevant macro background, the output becomes more precise.

Vague Prompts vs. Precise Prompts: An Intuitive Comparison

The course uses the example "write an article about spring." The problem with this basic prompt is that it's too broad—about the north or the south? An essay or a narrative? How long? What style? The AI can only guess, and the results end up formulaic.

The optimized prompt, by contrast, specifies an identity (essayist), word count (800 words), setting (spring in Jiangnan), and style (beautiful, lyrical language, emphasizing the misty water-town atmosphere). Going from vague requirements to concrete, executable rules naturally produces vastly different output quality.

Prompt vs. Prompt Engineering: The Difference Between a One-Off Technique and a Systematic Method

Many people conflate "prompt" and "prompt engineering," but the two differ enormously.

A prompt is a one-off, like writing a line of code and executing it once to solve a specific problem. The course particularly emphasizes: so-called "universal prompts" are all gimmicks. There is no prompt that works everywhere—only prompts suited to specific scenarios.

Prompt engineering, on the other hand, is a complete methodology. It doesn't modify the AI's own code or parameters; it only systematically designs input instructions to make the AI reliably produce the desired results. It relies not on inspiration, but on a reproducible, scientific workflow.

It's worth noting that prompt engineering emerged as a formal research field around 2020. With the release of GPT-3, researchers discovered that carefully designed input formats could dramatically improve performance without modifying model weights. Two techniques in particular are crucial: Few-shot Prompting, which includes several input-output examples in the prompt so the model can grasp the task format through In-Context Learning—this capability is essentially an emergent property that appears suddenly once a model's parameter count exceeds a certain threshold; without gradient updates, the model can infer task patterns from context examples alone. Chain-of-Thought (CoT), proposed by the Google Brain team in 2022, guides the model to explicitly output intermediate reasoning steps, activating deeper reasoning abilities within the weights, and can improve accuracy by 30%-50% on tasks like mathematical reasoning and logical judgment. The Self-Consistency method formed by combining the two (sampling multiple paths and then voting) pushes reasoning accuracy to new heights. Institutions such as Stanford, Google, and OpenAI successively published systematic papers, defining prompt engineering as an "interface optimization" discipline situated between the user and the model. On the industry side, dedicated Prompt Engineer positions once commanded annual salaries exceeding $300,000, and also gave rise to specialized engineering frameworks such as LangChain and PromptFlow.

LangChain was released by Harrison Chase in late 2022. Built around the core concept of "chained composition," it abstracts components such as prompt templates, model calls, tool integration, and memory management into reusable modules. Its GitHub repository surpassed 60,000 stars within a few months of release, becoming one of the de facto standards for LLM application development. Its Agent module is especially noteworthy—based on the ReAct (Reason+Act) framework, the model serves as the reasoning core, alternately invoking external tools during reasoning steps to achieve multi-step autonomous task execution. This paradigm is becoming the standard engineering model for complex AI applications. Microsoft's PromptFlow, deeply integrated into the Azure ML platform, provides a visual workflow orchestration interface along with complete evaluation, debugging, and version management capabilities, making it better suited for enterprise-grade team collaboration scenarios.

The Six-Step Workflow of Prompt Engineering

The course breaks prompt engineering down into six steps, which are logically highly similar to software development—except you're writing natural language instead of code:

1. Requirement Decomposition: Breaking a complex big problem into small problems the AI can solve
2. Solution Design: Designing corresponding prompts and roles for the decomposed problems
3. Execute and Test: Actually running it once and observing the results
4. Evaluate Results: Judging the quality of the output against specific criteria
5. Strategy Iteration: Continuously optimizing to address shortcomings
6. Consolidate and Reuse: Saving effective solutions so they can be directly reused for similar problems

What it solves is no longer a single problem, but a category of problems.

Four Criteria for Measuring the Success of Prompt Engineering

There are four core metrics for evaluating how well prompt engineering is done:

• Improve Accuracy: Making the AI output exactly what you need
• Reduce Hallucination: Making the AI generate less erroneous or fabricated information
• Enhance Stability: Keeping results consistent regardless of who uses it or when
• Reduce Cost: Getting the best results with the fewest Tokens

Why Learning Prompt Engineering Is Worthwhile Now

The reason the course gives is very practical: AI capabilities are already very strong, but over 90% of users only use it for basic Q&A. It's like buying a sports car and only ever driving in first gear—wasting all that performance.

The barrier to entry for prompt engineering is extremely low—you don't need to know programming, you don't need to understand algorithms; as long as you can articulate things clearly, you can get started. And the returns after mastering it are considerable:

• Efficiency Leap: Work that used to take a full day might now be done in an hour
• Side Hustle Opportunities: Helping companies and content creators optimize prompts and generate content in bulk, with almost zero startup cost
• Lower Entrepreneurship Costs: Work that used to require team collaboration can now be handled by one person, dramatically reducing the cost of trial and error
• Faster Content Production: A content creator's monthly output rising from 10 articles to dozens, with quality still on point

From this perspective, prompt engineering is not just a skill—it's a low-barrier, high-reward window of opportunity in the AI era.

The Limitations of Prompts: Not a Master Key

The most valuable part of this course is its honest acknowledgment of the limitations of prompts—precisely the blind spot that many beginners tend to overlook.

First, a prompt cannot break through the AI's own capability ceiling. If the model's training data only goes up to a certain point in time, no prompt, however clever, can extract information about events after that; if the built-in capabilities don't support a complex mathematical derivation, no amount of instruction tweaking will help.

Three Hard Technical Limitations

From a technical foundation standpoint, prompts have three problems that cannot be fundamentally cured:

• Limited Context Length: Every model has a fixed context window—the maximum number of Tokens it can process in a single pass. The expansion path of context windows has undergone rapid evolution: from 4K (GPT-3) → 32K (GPT-4) → 128K (GPT-4 Turbo) → 200K (Claude 3) → the million-level range (Gemini 1.5 Pro). However, research has found that a model's utilization of information at different positions within a long context exhibits a "Lost in the Middle" effect: information at the beginning and end of the context is effectively used, while key information positioned in the middle tends to be ignored—this is precisely the technical root of why AI gradually "forgets" what was requested at the very start. The larger the window, the higher the cost per call, and attention gradually gets "diluted." Therefore, reasonably organizing the placement of prompt content is equally crucial.
• Poor Stability on Complex Tasks: When multi-step reasoning or writing long-form solutions is involved, the AI is prone to skipping steps and breaking logic, so tasks must be decomposed into small tasks and handled one by one in advance.
• Lack of True Long-Term Memory: Every conversation is a fresh start for the AI; it cannot remember your preferences and history, and can only rely on external databases (such as RAG, Retrieval-Augmented Generation) to compensate.

RAG (Retrieval-Augmented Generation) was proposed by Meta AI in 2020. Its core idea is to dynamically inject retrieval results from an external knowledge base into the prompt context. A typical architecture involves three stages: documents are chunked and converted into vectors via an embedding model, then stored in a vector database (such as Pinecone or Chroma); when the user asks a question, the most relevant document fragments are retrieved simultaneously; the retrieval results are then spliced into the prompt, and the LLM generates the answer—giving the model's responses a verifiable basis. Function Calling is a standardized interface introduced by OpenAI in 2023, allowing the model to declare during inference that it needs to call a certain external function (such as querying a database or calling an API), with the application layer responsible for actually executing it and returning the results to the model. Both are deeply integrated with Agent architectures: RAG focuses on "passive retrieval" and is suited to knowledge-intensive Q&A; Function Calling focuses on "active action" and is suited to tasks requiring real-time data or external operations. Together, they form the core technical foundation of current enterprise-grade AI applications.

Three Practical Challenges in Real-World Deployment

From a real-world deployment perspective, there are three more pitfalls worth knowing about in advance:

• Safety and Alignment Constraints: Illegal or non-compliant content will never be generated, but sometimes compliant requests are mistakenly flagged and blocked—a common issue among all mainstream large models.
• Pure Prompts Are Hard to Engineer: For personal use it's fine, but turning it into a product for large numbers of users to call brings the dilemma of chaotic version management and hard-to-trace issues. Enterprise-grade applications are almost always a "prompt + code" combination.
• The Cost Trap Cannot Be Ignored: Large models charge by the Token, so the longer the prompt, the more expensive it is—and the cost does not grow linearly. Hidden fees may exceed expectations during high-frequency calls. When writing prompts, always strive for conciseness.

More fundamentally, there is the hallucination problem—prompt optimization can only reduce the probability of hallucinations occurring; it cannot eliminate them entirely. The root of hallucination lies in the model's training objective: mainstream LLMs use "next-Token prediction" as their pretraining objective, combined with RLHF (Reinforcement Learning from Human Feedback) for alignment fine-tuning. The optimization direction is to generate "responses humans find satisfying" rather than "factually correct responses"—this is precisely the fundamental tension between the pretraining objective and the fact-alignment objective. When the model encounters long-tail questions where training data is sparse, there is no clear corresponding knowledge pathway in its weights, but the generative mechanism still drives it to output fluent text with high confidence—this is precisely the technical root of "confidently talking nonsense." Academia divides hallucinations into two categories: "intrinsic hallucinations" (contradicting the input information) and "extrinsic hallucinations" (unable to be inferred from the input but not necessarily wrong). In engineering practice, beyond RAG and Function Calling, common approaches also include lowering the Temperature parameter to make output more deterministic, asking the model to annotate confidence levels, and using automated evaluation pipelines based on factual consistency to proactively manage hallucination risk. The industry generally believes that under the current Transformer paradigm, hallucinations cannot be completely eliminated—they can only be managed and controlled. As long as it's a large model, there will inevitably be a risk of "confidently fabricating information." Whenever important decisions or fact-checking are involved, always verify for yourself.

Conclusion

This introductory course establishes a clear-headed cognitive framework: a prompt is the most direct means of controlling AI, while prompt engineering is the systematic methodology that enables general-purpose models to reliably solve specific problems. It is low-barrier and high-value, and worth serious mastery by everyone working in the AI era.

But equally important is maintaining a sense of boundaries—a prompt is a useful tool, but not a master key. Where human judgment is called for, humans must still make the final call. Only by truly understanding its capability boundaries can you make AI your most handy assistant, rather than a headache-inducing black box.

Key Takeaways