Andrew Ng's AI Engineering Skills Map: A Complete Capability Breakdown from Building to Deployment

Introduction: AI Engineering Is Becoming a Standalone Discipline

As Large Language Models (LLMs) and generative AI move from the lab to production environments, a new professional role is rapidly emerging—the AI Engineer. Unlike traditional machine learning engineers, AI engineers focus more on leveraging existing Foundation Models to quickly build, evaluate, and deploy usable AI applications.

Andrew Ng's "AI Engineering Skills Map" systematically outlines the core competencies required for this role. It's not just a learning guide—it reflects a profound shift in the industry's demand for AI talent: from "training models" to "applying models," from algorithm research to engineering implementation.

The Core Logic of the Skills Map: Application First

From Model Training to Model Invocation

Over the past decade, core machine learning skills centered on data cleaning, feature engineering, model training, and hyperparameter tuning. In the foundation model era, the vast majority of developers no longer need to train a model from scratch. Instead, the focus is on how to efficiently invoke, orchestrate, and optimize existing powerful models.

The concept of Foundation Models was formally introduced by Stanford's Human-Centered AI Institute (HAI) in 2021, referring to large models pre-trained on massive datasets through self-supervised learning that can be adapted to a wide range of downstream tasks. Unlike traditional task-specific models, foundation models exhibit Emergent Abilities—new capabilities that suddenly appear once model scale crosses certain thresholds, even though these abilities were never explicitly optimized during training. This paradigm shift moves AI development from "one task, one model" to "one model, many applications," dramatically lowering the barrier to AI application development while giving rise to entirely new engineering methodologies.

This shift means a fundamental restructuring of the AI engineer's skill stack. Ng's skills map emphasizes that mastering Prompt Engineering, Retrieval-Augmented Generation (RAG), and model evaluation methods has become a more urgent practical capability than understanding backpropagation algorithms.

Building and Deploying Are Equally Important

Another key signal from the skills map: Building and Deploying are placed on equal footing. There is an enormous engineering gap between a demo that runs in a Notebook and an application that reliably serves millions of users in a production environment. AI engineers must possess both rapid prototyping ability and production deployment capability.

Core Skills Breakdown for AI Engineers

Understanding and Utilizing Foundation Models

While you don't need to train models from scratch, AI engineers must deeply understand the capability boundaries and limitations of foundation models. This includes:

• Model Selection: Making reasonable trade-offs between cost, latency, and performance across models like GPT, Claude, Gemini, and open-source Llama variants.
• Context Management: Understanding context window limitations, organizing input information effectively, and avoiding critical content truncation.
• Hallucination Issues: Recognizing that models may generate plausible-sounding but factually incorrect information, and designing corresponding verification mechanisms.

The "hallucination" problem in LLMs fundamentally stems from their training objective—Next Token Prediction. Models are optimized to generate statistically probable subsequent text rather than verify factual accuracy. Hallucinations fall into two categories: intrinsic hallucinations (contradicting input information) and extrinsic hallucinations (unverifiable from input or training data). Current industry mitigation strategies include: introducing verifiable external knowledge sources via RAG, using factual consistency detection models for output verification, designing self-verification prompts for models to check their own outputs, and reducing hallucination tendencies during training through RLHF (Reinforcement Learning from Human Feedback).

Prompt Engineering and Agent Orchestration

Prompt engineering has evolved from scattered "tips and tricks" into a systematic engineering practice. The theoretical foundation of Prompt Engineering originates from the discovery of LLMs' "In-Context Learning" capability. The 2020 GPT-3 paper first systematically demonstrated that models could complete new tasks using only a few examples in the prompt, without updating model parameters. Subsequently, Chain-of-Thought (CoT) prompting was proposed by the Google Brain team in 2022, significantly improving model performance on mathematical and logical reasoning tasks by demonstrating step-by-step reasoning processes in the prompt. Further developments include Tree-of-Thought, Self-Consistency, and other techniques, evolving prompt engineering from simple instruction writing into a complete engineering practice encompassing design patterns, testing methods, and version management.

Excellent AI engineers need to master the following capabilities:

• Design patterns for structured prompts
• Flexible application of Few-shot Learning
• Reasoning enhancement techniques like Chain-of-Thought
• Agent orchestration and workflow construction for multi-step tasks

AI Agents refer to intelligent systems capable of autonomously planning and executing multi-step tasks. Their theoretical foundation traces back to the "plan-act" loop in reinforcement learning, but in the era of large models, Agents replace traditional policy networks with natural language reasoning. Typical Agent architectures include the ReAct (Reasoning + Acting) pattern, where the model alternates between reasoning and tool invocation. Mainstream Agent orchestration frameworks like LangChain, LlamaIndex, and AutoGen provide standardized capabilities for tool registration, memory management, and multi-agent collaboration. Andrew Ng himself has proposed four design patterns for Agentic Workflows: Reflection, Tool Use, Planning, and Multi-Agent Collaboration, providing methodological guidance for Agent system design.

Retrieval-Augmented Generation (RAG) Architecture

RAG is one of the most mainstream technical architectures for enterprise AI applications today. By combining external knowledge bases with large models, it effectively mitigates model knowledge staleness and hallucination issues. Retrieval-Augmented Generation was first proposed by Facebook AI Research (now Meta AI) in 2020. Its core idea combines parametric memory (knowledge stored in model weights) with non-parametric memory (externally retrievable document stores). Technically, RAG relies on Vector Embedding technology to transform text into high-dimensional numerical vectors, then uses Approximate Nearest Neighbor (ANN) search for semantic-level similarity matching. Current RAG architectures have evolved from the simple "retrieve-generate" two-stage approach into multiple variants, including Adaptive RAG (deciding whether retrieval is needed based on the query), Iterative RAG (multi-round retrieval to progressively refine answers), and Graph RAG (structured retrieval combined with knowledge graphs).

Core areas AI engineers need to master include:

• Document Chunking and Vectorization: Choosing appropriate embedding models and setting reasonable chunking granularity.
• Vector Database Selection and Usage: Comparing and implementing solutions like Pinecone, Weaviate, Milvus, and others.
• Retrieval Quality Optimization: Improving recall accuracy through reranking, hybrid search, and other techniques.

Vector databases are the core infrastructure of RAG architectures, specifically designed for storing and retrieving high-dimensional vector data. They typically employ Approximate Nearest Neighbor (ANN) algorithms under the hood, such as HNSW (Hierarchical Navigable Small World) graph algorithms or IVF (Inverted File) indexes, balancing retrieval precision and speed. Major solutions each have their focus: Pinecone offers fully managed cloud services to reduce operational complexity; Weaviate supports hybrid search (vector + keyword); Milvus, open-sourced by Zilliz, is suited for large-scale deployment scenarios; and Chroma is known for being lightweight and developer-friendly, ideal for prototyping. Selection should consider data scale, query latency requirements, metadata filtering needs, and deployment environment constraints.

Evaluation and Iteration: From "Black Art" to "Science"

In AI application development, "how do you know it got better" is often harder than "how do you make it better." Andrew Ng consistently emphasizes the importance of Evaluation—establishing a quantifiable evaluation system is the critical turning point for AI engineering to move from experience-driven to data-driven.

An effective evaluation system typically operates on three levels:

• Automated Evaluation Metrics: Programmatically measurable dimensions such as answer relevance and faithfulness.
• Human Evaluation Processes: Standardized human review mechanisms designed for open-ended tasks.
• Business-Specific Test Sets: Proprietary test case collections built for specific application scenarios.

Notably, with the rise of "LLM-as-Judge" approaches (using large models to evaluate other large models' outputs), evaluation itself is undergoing an automation revolution. Through carefully designed evaluation prompts, strong models like GPT-4 can score target model outputs across multiple dimensions, finding a balance between cost and efficiency. Open-source evaluation frameworks like RAGAS (focused on RAG system evaluation) and DeepEval are standardizing these best practices, enabling evaluation processes to be seamlessly embedded into CI/CD pipelines.

Deployment: The Underestimated Engineering Challenge

The Gap from Demo to Production

Many AI projects fail not because the model isn't good enough, but because it can't be reliably deployed to production. Production deployment involves a series of engineering challenges that cannot be ignored:

• Cost Control: LLM API calls are expensive, requiring optimization through caching strategies, batch processing, and model downgrading.
• Latency Optimization: Users are extremely sensitive to response speed; streaming output and model distillation techniques are essential.
• Observability: Monitoring key metrics including model input/output logs, error rates, and user satisfaction.
• Security and Compliance: Defending against prompt injection attacks, sensitive data leakage, and other security risks.

Among these, Prompt Injection is a unique security threat facing LLM applications, where attackers use carefully crafted inputs to override system prompts and induce unintended model behavior. Defense measures include input/output filtering, permission layering design, and using dedicated safety guardrail models to pre-screen requests. Additionally, Knowledge Distillation technology, which trains smaller models to learn the output distribution of larger models, can reduce inference costs by several to tens of times while maintaining most of the performance—a key technique for balancing performance and cost in production environments.

The Evolution from MLOps to LLMOps

As large model applications proliferate, the traditional MLOps framework is expanding toward LLMOps. Version management must cover not just code and data, but also prompt templates, evaluation datasets, and model configuration parameters. This places higher demands on AI engineers' DevOps capabilities and has spawned a new generation of toolchains and best practices.

LLMOps is the evolved form of MLOps for the large model era, focusing on the full lifecycle management of LLM application development, deployment, and operations. Unlike traditional MLOps which focuses on model training pipelines, LLMOps core concerns include: prompt version management (e.g., PromptLayer, Humanloop), LLM observability (e.g., LangSmith, Helicone, Arize), evaluation frameworks (e.g., RAGAS, DeepEval), safety guardrails (e.g., Guardrails AI, NeMo Guardrails), and cost tracking and optimization. This ecosystem is still rapidly evolving, with tool standardization far below that of traditional software development—one reason AI engineers need to continuously track industry developments.

Implications for Practitioners: Opportunities and Challenges Coexist

Ng's skills map sends a clear signal: The barrier to AI engineering is shifting from "deep mathematical and algorithmic foundations" to "solid engineering capabilities and product thinking." This represents a tremendous opportunity for software engineers at large—they don't need to become machine learning PhDs to enter the AI application development arena.

But this doesn't mean it's easy. What's truly scarce are cross-functional talents who can both rapidly build prototypes and polish them into reliable production systems. Mastering prompt engineering, RAG architecture, systematic evaluation, and production deployment—this set of "new engineering fundamentals"—will become one of the most competitive skill combinations of the future.

From an industry trend perspective, the rise of the AI engineer role also reflects the deeper logic of technology democratization. Just as web development evolved from needing to understand the TCP/IP protocol stack to using high-level frameworks for rapid application building, AI development is undergoing a similar abstraction level leap. But unlike web development, AI system outputs are non-deterministic, meaning AI engineers need to develop a unique "probabilistic mindset"—accepting that system outputs aren't fully deterministic, and instead managing that uncertainty through statistical methods, guardrail mechanisms, and human-AI collaboration.

Conclusion

From popularizing deep learning to guiding the AI engineering era, Andrew Ng's skills map is essentially answering a question all developers care about: In an era dominated by foundation models, what should I learn?

The answer is clear—understand models, leverage tools effectively, prioritize evaluation, and excel at deployment. This is both a skills checklist and a ticket to the era of AI application implementation.