Andrew Ng's Prompt Engineering Course Essentials: Core Methodology Every Developer Must Learn

The course ChatGPT Prompt Engineering for Developers, jointly launched by AI education authority Andrew Ng and OpenAI technical team member Isa Fulford, is hailed as an essential course for progressing from LLM beginner to advanced practitioner. Andrew Ng previously served as the founder of Google Brain and Chief AI Scientist at Baidu, and is also a co-founder of Coursera. His company DeepLearning.AI has provided AI courses to over 5 million learners worldwide. Isa Fulford, meanwhile, was deeply involved in the development and promotion of the ChatGPT API. Hosted on the DeepLearning.AI platform, this course centers on hands-on Python coding, with all examples directly calling the OpenAI API—filling the gap between "theory-focused" and "engineering-focused" prompt tutorials. Based on the opening content of the course, this article organizes the core concepts of Prompt Engineering, model classifications, and key practical principles to help developers wield large language models more efficiently.

Why Developers Need to Rethink Prompt Engineering

Content about prompts has flooded the internet—articles like "30 Prompts Everyone Must Know" are everywhere. But Andrew Ng points out that most of this content focuses on the ChatGPT web interface, used to complete one-off specific tasks.

What is truly underrated is the potential of large language models (LLMs) as a developer tool—that is, calling LLMs via API to rapidly build software applications. Andrew Ng's team AI Fund has already collaborated with numerous startups to apply these technologies across various scenarios, and the rapid development capabilities demonstrated by LLM APIs are exhilarating.

It's worth noting that Prompt Engineering, as an independent discipline, only truly entered the public eye after the release of GPT-3 in 2020. GPT-3's emergence proved for the first time that ultra-large-scale pretrained models possess powerful "Emergent Abilities"—the capacity to perform diverse tasks such as reasoning, translation, and summarization purely from natural language descriptions, even on tasks they were never explicitly optimized for during training.

In-Depth Background: The Academic Debate Over Emergent Abilities Emergent abilities refer to capabilities that are nearly absent in smaller models but suddenly appear once a certain scale threshold is crossed, exhibiting nonlinear jumps resembling a "phase transition." A Stanford University research team systematically documented this phenomenon in a 2022 survey paper, covering abilities such as arithmetic reasoning, multi-step logic, and code generation. However, this concept is also academically contested: Schaeffer et al. from Google DeepMind (2023) argued that so-called "emergence" may be an illusion caused by inappropriate choice of evaluation metrics—when using finer-grained continuous metrics, capability improvements are actually smooth and gradual rather than abrupt. This debate has direct implications for prompt engineering: certain complex reasoning tasks may have a hard threshold on model scale, and no amount of prompt optimization can break through on smaller models. Understanding this boundary helps developers make more pragmatic technology choices.

This characteristic makes "how to write good inputs" itself a subject worthy of systematic study. Its core research object is: how to effectively guide large language models to produce high-quality, expected outputs by designing input text. The rise of this field is essentially because the capabilities of large language models are highly dependent on how inputs are phrased—the same question, asked differently, may yield entirely different answers.

From an academic perspective, prompt engineering encompasses multiple technical paradigms, which can be understood along the axis of "amount of information injected": Zero-shot learning provides no examples and drives the model to complete tasks directly through natural language instructions, relying on the generalization ability the model accumulated during pretraining; Few-shot learning embeds several "input-output" example pairs in the prompt, leveraging the model's In-context Learning ability to quickly align it with the target format or style—without updating any weights. This property holds because the Transformer architecture's attention mechanism naturally supports "reading" and "analogizing" the example structure within a single forward pass, essentially serving as an implicit simulation of gradient updates.

In-Depth Background: The Transformer Architecture and Attention Mechanism The Transformer architecture is the foundation of modern large language models, proposed by Vaswani et al. in the 2017 paper Attention is All You Need. Its core innovation—the Self-Attention mechanism—allows the model to simultaneously "attend to" information from all other positions in a sequence when processing each position, dynamically weighting them by relevance. This mechanism overcomes the vanishing gradient problem that RNNs/LSTMs face with long-range dependencies, enabling the model to capture semantic associations spanning thousands of tokens. Multi-Head Attention further allows the model to extract different types of linguistic features (such as syntax, semantics, and coreference) in parallel from multiple "subspaces," greatly enhancing expressive power. It is precisely this architecture's natural support for parallel computation that made pretraining GPT-series models on ultra-large corpora possible, and it is also the underlying foundation that enables Few-shot In-context Learning to perform "analogical reasoning" within a single forward pass.

Chain-of-Thought prompting goes a step further by requiring the model to output its reasoning process before giving the final answer—it is currently one of the most consistently effective prompting strategies known for complex reasoning tasks. These three paradigms are not mutually exclusive; in practical engineering they are often combined. For instance, Few-shot CoT (examples with reasoning steps included) often outperforms using any single paradigm alone. From an engineering perspective, prompt engineering directly affects the development efficiency and product quality of AI applications.

The course systematically covers several key modules: best practices for prompts in software development, common application scenarios (summarization, inference, transformation, expansion), and how to build chatbots with LLMs. The goal of this framework is to spark developers' imagination about new application scenarios.

Two Types of Large Models: Base Models vs. Instruction-Tuned Models

To understand prompt engineering, you first need to grasp the two basic types of large models. Andrew Ng distinguishes them as Base LLM and Instruction-tuned LLM.

Base Models: Predicting the Next Word Based on Context

The training of a Base LLM is essentially self-supervised learning: the model learns the statistical patterns of language and world knowledge by predicting the "next word" over massive amounts of unlabeled text. The key innovation of self-supervised learning lies in requiring no human annotation—it automatically constructs training signals directly from raw text, optimizing over trillions of tokens with the objective function of "predicting the conditional probability of the next word." From an information-theoretic perspective, this objective is equivalent to minimizing the cross-entropy between the model's distribution and the true language distribution. To continuously reduce this loss, the model must internalize the grammatical structure of language, long-range semantic dependencies, and even a vast amount of factual world knowledge—because accurately predicting the next word fundamentally requires understanding the logic and factual meaning of the preceding text. It is precisely this training mechanism that "forces the model to understand the world" that enables base models like the GPT series and LLaMA to exhibit surprising generalization abilities even on tasks they were never specifically trained for.

For example, given the input "Once upon a time there was a unicorn," it might continue with "It lived in a magical forest and lived with all its unicorn friends." But if asked "What is the capital of France," a base model likely won't answer directly. Instead, it may continue with "What is the largest city in France" or "What is the population of France"—because the pretraining corpus comes from real internet text (including a large volume of Q&A posts, forums, encyclopedias, etc.), and articles about France online often take the form of quiz-style questions themselves. This is a direct reflection of the model's data distribution, not a "flaw" in the model.

Pure base models have obvious limitations: they are good at "continuation" but not at "answering," because following instructions is not their training objective. This is precisely the fundamental reason instruction tuning was invented.

Instruction-Tuned Models: Learning to Understand and Follow Instructions

Instruction-tuned models are the mainstream direction in current LLM research and practice—they are specifically trained to follow natural language instructions. Asked the same question "What is the capital of France," an instruction-tuned model will directly output "The capital of France is Paris."

Instruction Fine-Tuning does not require training from scratch. It only requires relatively lightweight fine-tuning on top of a base model—typically less than 1% of the base training compute—to significantly change the model's behavior pattern. Its training path is: first train the base model, then fine-tune it using input-output pairs like "instruction-ideal response," often further combined with RLHF (Reinforcement Learning from Human Feedback) technology, making the model better at being helpful and following instructions.

The basic RLHF process consists of three stages: first, supervised fine-tuning (SFT) of the base model to teach it standard Q&A formats; second, training a "Reward Model," where human annotators rank the model's multiple responses by preference. The essence of this reward model is to convert hard-to-quantify "human preferences" into a differentiable scalar signal—the training data usually comes from annotators making pairwise comparisons of multiple model responses to the same question. By learning from these preference pairs, the model establishes an implicit "quality scoring function" that maps discrete human judgments into continuous rewards suitable for reinforcement learning optimization. Finally, reinforcement learning algorithms such as PPO are used to continuously optimize the LLM's outputs to maximize the reward model's score. OpenAI's InstructGPT paper (2022) was the first to systematically validate RLHF's effectiveness in improving model alignment, and mainstream models such as ChatGPT, Claude, and Gemini have since adopted similar approaches.

However, RLHF also faces inherent challenges: the reward model itself may have preference bias (Reward Hacking), meaning the model learns to "please" the reward model rather than genuinely improving response quality—manifesting as increasingly verbose responses with increasingly agreeable phrasing, but not necessarily more accurate substance. PPO's computational overhead is about 3x or more that of SFT, with high engineering implementation complexity, requiring four model copies to be maintained simultaneously in memory: the policy model, reference model, reward model, and value network. For this reason, alignment methods that avoid reinforcement learning, such as Direct Preference Optimization (DPO), have received widespread attention in recent years. DPO mathematically proves that the optimization objective of RLHF can be equivalently transformed into a language model classification loss that depends only on preference data, thereby completely bypassing the reward model training and the PPO reinforcement learning loop, compressing the three-stage process into a single-stage fine-tuning. It has been adopted by mainstream open-source models such as Llama 3 and Mistral.

In-Depth Background: The LLaMA Open-Source Ecosystem and the Democratization of Instruction Tuning Meta's LLaMA series (7B to 65B parameters), released in 2023, is a major milestone in the open-source LLM ecosystem. LLaMA itself was released as a base model, but the open-source community quickly launched numerous instruction-tuned variants on top of it: Alpaca (Stanford, based on Self-Instruct data), Vicuna (based on ShareGPT conversation data), WizardLM (based on evolved instruction data), and more. These projects proved a key fact: high-quality instruction tuning only requires tens of thousands of curated data points, not billions—corroborating findings from Google's FLAN series research. The LLaMA 3 series released in 2024 further introduced DPO alignment training, and its instruction-tuned versions approach or surpass early versions of GPT-4 on multiple benchmarks, marking the ongoing narrowing of the capability gap between open-source and closed-source models. This trend means for prompt engineers that applications built on open-source models have higher prompt portability and reproducibility.

The core value of both RLHF and DPO lies in converting "human preferences," which are difficult to describe with rules, into optimizable training signals, thereby allowing the model to strike a better balance among helpfulness, truthfulness, and harmlessness.

Because instruction-tuned models are trained to be more "helpful, honest, and harmless," the probability of them outputting harmful content is greatly reduced. This also explains why the open-source community can rapidly derive large numbers of instruction-tuned versions for specific scenarios based on base models like LLaMA.

Core Course Advice: Focus on Instruction-Tuned Models

Andrew Ng clearly advises: many prompt techniques found online are actually more suited to base models, but for the vast majority of today's practical applications, developers should focus their energy on instruction-tuned models.

There are two reasons: first, instruction-tuned models are easier to use; second, thanks to the continued investment by companies like OpenAI, these models are becoming increasingly safe and increasingly aligned with human values. Therefore, the entire course revolves around best practices for instruction-tuned models.

When using instruction-tuned model APIs, there is a key abstraction layer often overlooked by beginners: the System Prompt. In the OpenAI API message structure, prompts are divided into three roles: System, User, and Assistant. The system prompt is set by the developer before the conversation begins and remains in effect throughout the entire session, used to define the model's overall behavioral norms, role setting, output format constraints, and safety boundaries—equivalent to giving the assistant a "code of conduct" before each conversation.

In-Depth Background: The Engineering Value and Safety Boundaries of System Prompts From an implementation standpoint, the system prompt is not truly "privileged" in most models—it is still concatenated as ordinary text input to the Transformer at the underlying level. It's just that during the training phase, using large numbers of "system prompt-user instruction-assistant response" triplets, the model learns to grant higher "instruction authority" to the System role. This design brings two important engineering implications: First, a carefully designed system prompt can customize a general-purpose model into a domain-specific expert without any fine-tuning, greatly lowering the development barrier for vertical applications. Second, the system prompt is not an absolute safety boundary; targeted "Jailbreak" prompts can sometimes induce the model to bypass system-level restrictions, which is why serious AI applications need to layer an independent content filtering layer on top of the system prompt. One of the core competitive moats of many commercial AI applications (such as GitHub Copilot and Notion AI) is precisely system prompt engineering that has been iterated hundreds of times and is highly robust on edge cases.

From an engineering perspective, this is the core abstraction layer for building reusable AI applications: through carefully designed system prompts, developers customize general-purpose instruction-tuned models into specialized assistants for specific domains, without any model fine-tuning. The core competitive moat of many AI applications is precisely the extensively iterated and optimized system prompt itself.

The Core Mindset: Treat the Model as a Smart Assistant Who Doesn't Know You

This is one of the most enlightening analogies in prompt engineering. Andrew Ng suggests that when using an instruction-tuned model, you should imagine you are giving instructions to another person—someone very smart, but who knows nothing about the background of your task.

When an LLM underperforms, many times it's not because the model lacks capability, but because the instructions themselves aren't clear enough.

Take "Please help me write something about Alan Turing" as an example—this instruction is too vague. To get an ideal result, you need to add explicit information:

• Content focus: Should it focus on his scientific contributions, personal life, or historical significance?
• Tone of the text: The tone of a professional journalist, or a casual note written to a friend?
• Reference materials: Specify which text snippets the model should reference—equivalent to providing a newly hired college graduate with the background materials needed to complete the task.

The reason this "newly hired college graduate" metaphor is so apt has a deeper layer: a language model's "knowledge" comes from the statistical distribution of its pretraining corpus. It knows nothing about your personal work context, audience preferences, or writing conventions—just like a brilliant but freshly hired graduate—with enormous potential, but who can only reach their best performance when given sufficient contextual information. There's an important physical constraint worth noting here: the Context Window sets an upper limit on the number of tokens the model can process at once. Early GPT-3 had a context window of only 4096 tokens; current mainstream models have expanded it to 128K or even millions. Research shows that the model's attention to information at different positions in the context is uneven—the "Lost in the Middle" phenomenon (Liu et al., 2023) shows that key information tends to be processed better when placed at the beginning or end of the context rather than in the middle. This means that when writing long prompts, the most important instructions and constraints should be placed at the beginning or end, rather than buried in the middle of a large block of background description. This "newly hired college graduate" metaphor runs throughout the course: the more sufficient the context you provide and the more specific your requirements, the more the output will meet your expectations.

The Two Fundamental Principles of Prompt Engineering

The course distills two core principles of prompt engineering that form the cornerstone of the entire methodology:

1. Be clear and specific: Instructions must be explicit, avoid ambiguity, and provide sufficient context and constraints.
2. Give the LLM time to think: The second principle, taught by Isa Fulford, emphasizes guiding the model to reason step by step rather than rushing to a conclusion.

The second principle technically corresponds to an important discovery in the AI field—Chain-of-Thought Prompting (CoT). In 2022, Google researchers Wei et al. demonstrated in a paper that when a prompt contains examples of step-by-step reasoning, or when the model is explicitly asked to "think step by step," the model's performance on complex reasoning tasks (such as math word problems and multi-step logical inference) significantly improves.

The working principle behind this is worth understanding in depth: large language models are essentially autoregressive generative models—when generating each token, they can only "see" the previously generated prefix content. When the model is asked to directly output the final answer, it must complete all reasoning within the hidden layer representation of a single forward pass, which often exceeds the model's "working memory" limit for multi-step math or logic problems. The core mechanism of chain-of-thought lies in externalizing intermediate reasoning steps into a visible text sequence, where each step's output becomes the explicit input for the next step's reasoning, effectively extending the model's reasoning depth. Computationally, this is equivalent to trading more token generations (i.e., more forward-pass steps) for deeper reasoning capability—a core trade-off of exchanging inference time for inference quality.

The reason the Zero-shot CoT trigger phrase "Let's think step by step" (proposed by Kojima et al. in 2022) is effective is rooted in the data distribution of the pretraining corpus: internet corpora are full of "step-by-step derivation" text patterns—solution processes in math textbooks, debugging steps on Stack Overflow, argumentation structures in Wikipedia, and so on. During pretraining, by minimizing the next-word prediction loss, the model internalizes these step-by-step reasoning text patterns as implicit knowledge in its weights. When trigger phrases like "step by step" appear in the prompt, the model's attention mechanism activates weight paths highly correlated with such patterns, thereby favoring structured, step-by-step output—which also explains why this technique generalizes to some extent across different languages and task types.

In-Depth Background: The o1/o3 Reasoning Models and the Paradigm Shift in Prompt Engineering The o1 model released by OpenAI in 2024 represents a new paradigm: internalizing Chain-of-Thought (CoT) from an external prompt-level technique into the model's training objective itself. Through large-scale reinforcement learning training, o1 learns to autonomously generate implicit "internal reasoning chains" before producing the final answer. These reasoning chains are not fully exposed externally but profoundly influence the final output. A side effect of this mechanism is that traditional CoT prompting techniques have diminishing marginal returns on reasoning models like o1/o3, because the model already has "slow thinking" capability built in. For developers, this means the focus of prompt design is shifting—from "how to guide the model to reason step by step" to "how to precisely define the problem's constraint boundaries, format requirements, and evaluation criteria." Understanding this generational evolution helps developers make the right prompt strategy choices across different models, avoiding mechanically applying techniques suitable for the GPT-3.5 era to reasoning models.

It's worth noting that reasoning models like o1 and o3 released in 2024 internalize CoT into the model's training objective itself—by rewarding reasoning chains that produce correct answers during the reinforcement learning phase, the model learns to autonomously generate implicit chains of thought, marking the evolution from prompt-level chain-of-thought to model-level "deep thinking." This also foreshadows that prompt engineering itself will continue to transform as model architectures evolve: when CoT becomes an intrinsic capability of the model rather than an external triggering technique, the focus of a prompt engineer's work will shift from "how to guide the model to think" to "how to define problem boundaries and evaluation criteria."

These two principles may seem simple, but they are the true watershed distinguishing "casual questioning" from "professional prompt engineering."

Conclusion

The value of Andrew Ng's course lies not in listing flashy prompt templates, but in establishing a set of underlying cognitive frameworks: understanding the differences between model types, clarifying the applicable scenarios of instruction-tuned models, and mastering the two core principles of "being clear and specific" and "giving the model time to think."

For developers hoping to advance from beginner to expert, rather than blindly collecting prompt techniques from the internet, it's better to first internalize this methodology. When you learn to converse with the model as you would with a smart assistant, prompt engineering ceases to be mysticism and becomes a reusable, sustainably iterable engineering practice.