Advanced Prompt Engineering: Six Core Techniques from CoT to Agents

Prompt Engineering Is More Than Just Writing Prompts

Prompt Engineering is often oversimplified as "learning to write prompts," but once it enters production environments, it becomes more of a systematic engineering discipline. From zero-shot prompting to self-optimizing prompt loops, different maturity levels correspond to entirely different tech stacks and evaluation methods.

A deep-dive guide from the tech community divides prompt engineering into six maturity levels: from improvised zero-shot prompting, to structured templates, few-shot + chain-of-thought, RAG integration, Agent prompting, and finally self-optimizing loops. Each leap in level means higher system complexity and stronger output reliability.

Determining which level your project is at directly determines how much engineering resources you should invest—using zero-shot prompts for complex reasoning tasks, or Agent architectures for simple classification problems, are both common resource misallocations.

Foundational Techniques: Zero-Shot, Few-Shot, and Chain-of-Thought

Zero-Shot Prompting: Start with Templating

The core of zero-shot prompting isn't "just writing a random sentence"—it's separating system instructions and user instructions into reusable templates. For example, summarization, entity extraction, and classification tasks can each be defined as independent PromptTemplate data classes, with flexible composition achieved through variable injection.

The value of this approach: when task descriptions change, you only need to modify the template itself rather than adjusting each calling code individually.

Few-Shot Prompting: The Power of Examples

Few-shot prompting guides model behavior by providing input-output pairs. The key detail is format consistency across examples—if examples use Input: / Output: markers, test inputs must follow the same format.

Taking natural language to SQL as an example, three to five examples covering different query patterns can significantly improve model performance on unseen queries. The guide recommends organizing task descriptions, examples, and test inputs as structured concatenation, with output format requirements also explicitly declared as part of the prompt.

Chain-of-Thought (CoT): Making Reasoning Visible

Chain-of-thought prompting improves accuracy on complex tasks by guiding the model to reason step by step. The basic form is asking the model to "think step by step" in the prompt and explicitly listing reasoning steps.

A more advanced approach is Self-Consistency: running CoT reasoning multiple times on the same question and taking the majority vote. This method trades computational cost for accuracy improvement, with particularly notable results in mathematical reasoning and logical inference tasks.

Self-Consistency derives its benefit from a statistical fact: single greedy decoding easily falls into locally optimal reasoning paths, while multiple samplings can cover different reasoning branches. Since correct reasoning paths tend to be more easily reproduced independently by the model than incorrect ones, majority voting naturally converges toward the correct answer. In practice, the temperature parameter is typically set between 0.5 and 0.7 to obtain sufficient path diversity, with sampling counts starting at 5 and scaling up to 20-40 for scenarios with high accuracy requirements. Note that this method works best for tasks with normalizable answers (numerical values, options, short entities), while for open-ended generation tasks it's difficult to define a "majority."

Advanced Patterns: ToT, ReAct, and RAG

Tree of Thoughts (ToT)

If CoT is a reasoning chain, ToT is a reasoning tree. It achieves more global search by generating multiple candidate thoughts at each step, evaluating their quality, and retaining the best branches.

In implementation, ToT maintains a Thought data structure containing content, score, and parent-child relationships. Each layer generates branches through the LLM, evaluates branch quality, and prunes to retain top-K. The solution is ultimately obtained by backtracking along the optimal path. The cost is multiplied LLM calls, making it suitable for high-value scenarios requiring exploratory reasoning.

ReAct: Combining Reasoning and Acting

The ReAct pattern has the model alternate between reasoning and tool calls. The typical flow is: think → select tool → execute → observe results → continue thinking, until a final answer is reached.

The prompt template needs to define the available tool list, tool call format (Action/Action Input), and termination conditions (Final Answer). The parsing layer needs to extract structured information from model output, distinguishing between "continue calling tools" and "provide final answer" states.

ReAct gets its name from the combination of Reasoning and Acting, first proposed by Yao et al. in 2022. Its key insight is: pure reasoning (like CoT) cannot access external facts, while pure acting (like directly calling tools) lacks the ability to integrate results and correct errors. Alternating between the two allows the model to re-plan when observing abnormal tool returns, rather than following an erroneous path to the end. ReAct is also the foundational paradigm for most subsequent Agent frameworks—LangChain's AgentExecutor and AutoGPT's execution loop can both be viewed as engineered variants of it.

RAG Prompting: Generation with Citations

Prompt design in RAG scenarios has two core requirements: constraining the answer scope and requiring source citations. The prompt must explicitly require the model to answer only based on the provided context, and to refuse to answer when context is insufficient.

The structured output version goes further, requiring the model to return JSON format containing answer text, cited fragments (annotated with source document IDs and original text quotes), and confidence scores. This provides structured evidence for downstream quality monitoring and human review.

Agent System Prompt Design

Once entering the Agent level, prompts are no longer a single template but a set of collaboratively working role definitions.

Supervisor Agent is responsible for decomposing tasks, delegating subtasks to specialist Agents, and synthesizing results. Its prompt needs to define the delegation protocol (how to specify target Agent and task description), response format (continue delegating or provide final answer), and historical result tracking mechanism.

Specialist Agent focuses on specific capability domains. Research Agents emphasize information retrieval and cross-validation, while coding Agents emphasize code quality and design decision documentation. Each specialist Agent's prompt includes capability descriptions, available tool lists, working principles, and output format requirements.

Prompt Optimization and Evaluation Frameworks

Automated Prompt Optimization

The efficiency bottleneck of manually tuning prompts has given rise to automated optimization methods. One approach draws from evolutionary algorithms: initialize a batch of prompt variants (adding CoT, adding few-shot examples, adding format constraints, etc.), then evaluate, select, cross, and mutate across multiple generations to gradually approach the optimal prompt.

DSPy-style optimization is more lightweight: by adding examples (demos) and automatic compilation (compile), the framework automatically selects the best example combination based on evaluation metrics.

DSPy is a declarative prompt programming framework proposed by the Stanford NLP group. Its core idea is to abstract prompts from "hand-written strings" into "optimizable modules": developers only need to define input/output field names and signatures (Signature), along with an evaluation metric, and the framework's compiler (Teleprompter) automatically searches for the best instruction wording and few-shot example combinations. The difference from manual tuning is similar to the difference between hand-writing regex and training a classifier—the former relies on experience, the latter on data-driven search.

Evaluation Metrics and A/B Testing

Production prompt evaluation requires multi-dimensional metrics: accuracy, consistency (result stability across multiple runs), instruction adherence, format compliance rate, latency, and cost. LLM-as-Judge is a common automated evaluation method—using another model to score output quality.

A/B testing frameworks allow simultaneous evaluation of two prompt versions, determining superiority through statistical significance. Interestingly, accuracy differences below 2% are typically difficult to distinguish and require larger sample sizes for reliable conclusions.

LLM-as-Judge effectiveness depends on several prerequisites: the judging model's capability is typically stronger than or equal to the model being tested; the judging task is decomposed into specific dimensions rather than vague scoring; the judging prompt includes reference answers or scoring rubrics. Common biases include position preference (tending to choose options listed earlier), length preference (tending toward longer answers), and self-preference (tending toward self-generated content). Mitigation approaches include randomizing option order, requiring reasoning before scores, and averaging across multiple judging models.

Production-Grade Prompt Management

Version Control and Registry

Prompts should be version-managed like code. A SQLite-based PromptRegistry can implement prompt saving, version querying, historical backtracking, and other functions. Each prompt modification creates a new version, recording metadata and parent version ID, making troubleshooting and rollback possible.

Canary Releases and Feature Flags

New prompt versions should not be deployed all at once. Through traffic percentage control and conditional routing (such as only taking effect for specific user tiers), progressive deployment can be achieved. Once metrics are abnormal, quickly roll back to the previous version.

Production Checklist

The guide provides a checklist covering four dimensions:

• Prompt Quality: Role definition, task description, output format, constraints, few-shot examples, CoT steps, edge case handling, self-verification steps
• Evaluation: Gold-standard test set (50+ cases), automated metrics, LLM review, A/B testing, CI/CD regression detection
• Management: Version control, registry, feature flags, canary deployment, rollback capability (<1 minute), usage analytics
• Security: Input sanitization, injection detection, PII handling, output validation, rate limiting

Decision Framework: How to Choose Prompt Patterns

The decision tree provided by the guide is logically clear: simple deterministic tasks use zero-shot; tasks requiring specific format or style use few-shot; complex reasoning uses CoT or ToT; tasks requiring external information use RAG; multi-step tool calls use ReAct or Agent; high-risk production scenarios require the trifecta of optimization + evaluation + version management.

Choosing an overly heavy approach wastes resources, while choosing an overly light approach fails to meet quality requirements. Understanding the applicable boundaries of each pattern is more important than blindly pursuing the "most advanced" methods.

One-line Summary

Prompt engineering has six maturity levels, from zero-shot to self-optimizing loops, requiring engineering investment matched to task complexity.

Paragraph Summary

The article redefines prompt engineering from "learning to write prompts" to a systematic engineering discipline, divided into six maturity levels: zero-shot prompting, structured templates, few-shot plus chain-of-thought, RAG integration, Agent prompting, and self-optimizing loops. The foundational techniques section covers zero-shot templating, few-shot format consistency, and chain-of-thought with self-consistency voting. Advanced patterns cover Tree of Thoughts branch search, ReAct's reasoning-acting alternation, and RAG's citation constraints and structured output. The Agent level requires designing collaborative role prompts for Supervisor and Specialist. The production section emphasizes automated optimization (evolutionary algorithms, DSPy), multi-dimensional evaluation (LLM-as-Judge, A/B testing), version control, canary releases, and security checklists. The final decision framework recommends selecting patterns based on task complexity, format requirements, and risk level to avoid resource misallocation.