SJTU Open-Sources 'Hands-On AI': From API Calls to RAG and Agent Development

An Open-Source Tutorial That's Got the AI Community Talking

A hands-on tutorial series has been making the rounds in the AI community lately — Hands-On AI, open-sourced by a team at Shanghai Jiao Tong University. Led by Professor Zhang Zhuosheng and co-developed with several industry experts, the series bills itself as a practical, beginner-friendly learning path.

Unlike the many courses that stay firmly in theory-land, this one is built around doing. It starts with the basics of API calls and works its way up to model deployment, fine-tuning, safety and defense, and automated Agent development. A key emphasis is local deployment: by the end, you should be able to run a dedicated LLM on your own machine — no external internet required, with better privacy and data security as a bonus.

For newcomers trying to break into LLM engineering, this kind of "spoon-fed" practical approach genuinely lowers the barrier to entry. In this article, we'll dig into two representative case studies from the tutorial to unpack what it actually teaches — and whether the approach is worth following.

Case Study 1: Structured Information Extraction with an LLM

The first hands-on case is a "tourist attraction extraction" task: given a passage of text, have the model identify Chinese tourist attractions and return them as a structured list.

Environment Variables and API Key Management

This case demonstrates a basic engineering best practice — never hardcode your API key into your code. Using the Volcano Engine (Ark) platform as an example, the instructor walks through creating and copying a key in the "API Key Management" panel, writing it to a .env file, and reading it in via os.getenv().

Notably, the demo hit an error along the way: missing credits, pass an API key. The culprit? A missing load_dotenv() call — the environment variables were never loaded. This "oops" moment is actually more instructive than a flawless demo: failing to load environment variables is one of the most common rookie mistakes, and adding that one line fixed everything.

Prompt Engineering Is the Key to Quality Output

The instructor repeatedly emphasizes the importance of writing good prompts and summarizes a few universal principles:

• Role assignment: Explicitly tell the model, "You are a master at extracting tourist attractions."
• Task scoping: Constrain the model to extraction only — no elaboration.
• Output format enforcement: Require a specific format to make downstream parsing easier.
• Few-shot Learning: Guide the model with examples.

When building the message list, the user's query is injected into the prompt template via prompt.format(query=...) and passed as the user role's content. Since this is a one-shot task rather than a conversational system, there's no need to maintain a history message list.

Few-shot Learning is a key technique in prompt engineering: by providing a small number of input-output examples in the prompt, you let the model "reason by analogy" toward the desired output format and style — without touching model weights. This contrasts with Zero-shot (giving instructions with no examples) and Fine-tuning (retraining on large datasets). Few-shot is especially effective for structured extraction tasks. Show the model one or two complete examples of "input text → output list," and it will automatically align to your format, dramatically reducing formatting errors. Two to five examples is usually the sweet spot — too many and you burn tokens while potentially distracting the model.

Parsing Model Output with Regular Expressions

Model output typically comes back as formatted text (e.g., pipe-delimited values wrapped in brackets), which needs to be parsed into a Python list before your program can use it. Interestingly, the instructor demonstrates a "lazy but effective" approach: just ask the LLM to write the parsing script — drop a prompt like "Write me a Python regex script that parses this into a list" and let the model generate the function.

The core parsing logic it generates: use regex to match the pattern inside the brackets, then split on the pipe character (|) to get the final list. For robustness, the regex also handles cases where there might be extra text before or after the brackets.

Two more engineering details worth noting: first, the parsing logic is wrapped in try...except, so if the model's output can't be parsed, an empty list is returned instead of crashing the program. Second, stray double quotes in the output are cleaned up, leaving a tidy single-element list. This combination of "structured output + structured parsing" is the foundational pattern for wiring an LLM into a real application.

Case Study 2: Logic Reasoning with Chain-of-Thought

The second case involves a logic puzzle: four employees — A, B, C, and D — are assigned to four roles (Operations, Design, Engineering, HR). Given a set of constraints (e.g., A doesn't do Engineering or Design; B handles employee relations and attendance; C is proficient in code but not Operations; D doesn't do Design), deduce each person's role.

This kind of problem is notoriously tricky for LLMs — the constraints have multiple interdependencies, and models tend to "skip steps" and make mistakes. The tutorial's solution is to guide the model with Chain-of-Thought (CoT) prompting: explicitly instruct it to "complete the reasoning in strict chain-of-thought fashion, breaking down the solution process step by step."

Pair a role-setting prompt ("You are a rigorous reasoning master") with a hard constraint to reason step by step, and you can significantly boost model performance on logic puzzles like this. It reinforces a widely-held industry consensus: for reasoning-heavy tasks, prompting the model to "show its work" is far more reliable than just asking for the answer.

Chain-of-Thought (CoT) was introduced by Google researchers (Wei et al.) in 2022. The key finding: when a prompt asks the model to "think step by step" or includes examples with intermediate reasoning steps, the model's accuracy on arithmetic, commonsense reasoning, and symbolic reasoning tasks improves significantly. The underlying mechanism is that the model "writes out" its intermediate reasoning in the token sequence before producing a final answer — essentially giving itself a scratchpad. Follow-on research produced Zero-shot CoT (just adding a trigger like "Let's think step by step"), Tree-of-Thought, Self-Consistency, and other variants, collectively known as reasoning-enhanced prompting techniques. CoT is especially effective on multi-constraint logic puzzles, since step-by-step elimination aligns naturally with the model's token-by-token generation process.

Who Is This Tutorial For?

Based on the content covered, Hands-On AI has a clear audience: aspiring engineers with little to no background, looking to build a complete pipeline from API calls and prompt engineering through structured parsing and reasoning enhancement — with later chapters extending into RAG, Agents, MCP, model deployment, fine-tuning, and safety.

Its value isn't in explaining deep theory. It's in thoroughly demonstrating how to actually build things. Details like environment variable management, try...except error handling, and regex parsing are exactly the kind of things textbooks gloss over but that come up every single day in real engineering work. The live errors and debugging that appear during the demos are, arguably, more educational than a polished, issue-free walkthrough.

For beginners, the recommendation is to follow along with the official open-source resources and type out the code yourself. Get each case running locally, then try rewriting the prompts and observing how the output changes. That's the only way to truly internalize knowledge that's been handed to you on a silver platter.

RAG (Retrieval-Augmented Generation) and Agents are the two core advanced topics covered in later chapters of the tutorial, and understanding them helps assess the learning path's value. RAG's approach: rather than stuffing all knowledge into model parameters, dynamically retrieve relevant documents from an external knowledge base at inference time, inject relevant snippets into the prompt, and then have the model generate an answer — addressing LLM knowledge cutoff and hallucination issues. Agents go a step further: the model doesn't just "talk," it "acts" — completing multi-step tasks by calling tools (search, code execution, file I/O, etc.). MCP (Model Context Protocol) is a recently emerged standard for normalizing Agent tool-calling interfaces. Both technologies are currently core architectural components for deploying LLMs in production, and mastering them means being able to build AI applications that are genuinely ready for real-world use.