Complete Spring AI Tutorial: ChatModel, RAG, and MCP Practical Guide

The Java Developer's Path to AI Engineering

Python's ecosystem, led by LangChain, dominated the early narrative around AI application development. But with frameworks like Spring AI, LangChain4j, and Spring AI Alibaba now reaching maturity, Java developers finally have a complete, well-structured toolchain for building large language model (LLM) applications.

This article is based on a comprehensive Spring AI course outline, tracing the full technical journey from basic conversational AI to RAG (Retrieval-Augmented Generation) and AI Agent development. The goal is to give readers a clear picture of what Spring AI can do and where its boundaries lie.

Environment Setup and Getting Started

Spring AI has explicit minimum requirements for JDK and Spring Boot versions — these must be verified before you write a single line of code.

The introductory section uses DeepSeek to demonstrate the most basic LLM interaction flow. The core objective is to help developers understand how Spring AI abstracts model calls — the underlying HTTP requests and response parsing are encapsulated behind a unified interface, so you never need to deal with the implementation details.

One important principle runs throughout the entire course: learn one, apply to all. Whether it's ChatGPT, Claude, or DeepSeek, Spring AI officially supports a wide range of models, and the integration pattern is nearly identical across all of them — usually just a matter of swapping a few configuration parameters. Master one model integration, and the rest will come naturally.

Spring AI's Four Core Model Abstractions

Spring AI abstracts LLM capabilities into several core Model types. Understanding these is the key to understanding the entire framework.

ChatModel: The Conversational Model

ChatModel is the most commonly used abstraction, handling text-based conversations with LLMs. Using DeepSeek as an example, it covers three dimensions:

• Basic chat: Synchronous request-response calls
• Streaming chat: Token-by-token responses that significantly improve the interactive experience
• Parameter control: Fine-tuning generation results via settings like temperature and max tokens

EmbeddingModel: The Embedding Model

If ChatModel is the front door, EmbeddingModel is the foundation of RAG. The essence of embedding is converting a piece of text into a vector. Local knowledge base content needs to be vectorized, and user queries need to go through the same process. The two are then matched via vector similarity to identify the most relevant text segments, which are then passed to the LLM to generate a final answer.

Using Zhipu AI as an example, three progressively advanced scenarios are built around Embedding:

1. Find similar text — understanding the basic concept of vector similarity
2. Manual RAG implementation — building the retrieval pipeline from scratch without relying on framework abstractions
3. RAG optimization — improving on the limitations of the manual implementation

The "build it from scratch" teaching approach is particularly valuable: it bypasses Spring AI's built-in Advisor API, forcing developers to truly understand every step of the logic behind RAG. This manual approach is also completely viable in real-world projects.

ImageModel and AudioModel

Spring AI also includes ImageModel for image generation and AudioModel for speech-to-text and text-to-speech. In practice, ChatModel and EmbeddingModel see the widest use — image and audio models are good to know, and their APIs follow a very similar pattern to the first two.

Spring AI with Ollama for Local Deployment

Ollama is currently the most popular tool for managing locally-run models. Combining Spring AI with Ollama means you can run both chat and embedding operations entirely on-premises, with no dependency on cloud APIs. For enterprise scenarios with strict data privacy requirements or a need to control API costs, local deployment is an important option worth considering.

ChatClient and ChatMemory

ChatClient: A Higher-Level Conversation Client

ChatModel is a relatively low-level object. ChatClient is Spring AI's higher-level, more fully featured client wrapper. It reduces boilerplate around parameter configuration and result handling, making it the preferred entry point for real-world development.

ChatMemory: Giving the Model Conversational Context

LLMs operate in a stateless "single question, single answer" mode by default — they have no memory. If you want the model to remember previous turns in a conversation, you need to introduce the ChatMemory mechanism.

Two common storage approaches:

• In-memory storage: Lightweight and fast, suitable for temporary session scenarios
• MySQL database storage: Persistent, suitable for production environments

One worth noting detail: the official documentation explains which ChatMemory APIs exist, but doesn't systematically cover how to integrate them with Spring Boot across Controller and Service layers in a complete application. This is precisely where a structured course delivers more value than official docs alone.

Tool Calling and the MCP Protocol

Tool calling and the MCP protocol are the core capabilities for building AI Agents — and the two are closely connected.

Tool Calling: Letting the Model Invoke External Capabilities

As long as an LLM supports tool calling, developers can define tool functions and let the model invoke them autonomously during inference. Most mainstream models now support this capability, but each model has its own implementation and API — which is exactly the problem that the MCP protocol was created to solve.

MCP: A Unified Standard for Tool Calling

MCP (Model Context Protocol), introduced by Anthropic, provides a unified API specification for tool calling across different LLMs. It's important to clarify that MCP is not a Spring AI invention — Spring AI simply implements support for the protocol, enabling developers to use a single API to invoke tool capabilities across different models.

This section covers: an explanation of MCP concepts, the architecture and role breakdown in Java (Spring AI), full client and server development, multiple transport protocol options with corresponding examples. This part represents the critical leap from "model conversation" to "intelligent agent (Agent)" development.

RAG: End-to-End Practical Implementation

Building on the manual RAG implementation from the EmbeddingModel section, the comprehensive practical section takes a progressively deeper approach rather than simply repeating what came before.

Vector Database: Milvus

Enterprise-grade RAG typically deals with massive document collections that can't be held in memory — a vector database is essential. Milvus is one of the most widely used vector databases in the LLM space. You'll need to master its setup, core concepts, command-line operations, and Java API to persistently store vectorized document content.

Spring AI's Built-In RAG API

Once a vector database is in place, Spring AI's built-in RAG API can automatically handle the retrieval pipeline — performing vector similarity search on user queries, then passing the relevant segments to the LLM for a final response. This dramatically reduces development complexity.

Comprehensive Case Study: Tour Guide Exam Q&A System

The complete execution pipeline for the RAG case study looks like this:

1. Local documents are vectorized via EmbeddingModel
2. Vectors are stored in the Milvus database
3. User queries are also vectorized via Embedding
4. Relevant text segments are retrieved from the database
5. Results are passed to the LLM to generate a final answer

This case study ties together Embedding, vector database retrieval, and LLM generation into a single cohesive flow — making it the capstone demonstration of the entire knowledge system.

Conclusion: A Clear AI Learning Path for Java Developers

Spring AI lays out a clear progression for Java developers, from beginner to practitioner:

Conversation (ChatModel/ChatClient) → Memory (ChatMemory) → Tools & Agents (Tool Calling/MCP) → Retrieval Augmentation (EmbeddingModel + Milvus + RAG)

For Java engineers who have long been invested in the Spring ecosystem, there's no need to pivot to Python. LLM applications can be engineered and delivered entirely within a familiar tech stack. The key is to understand the principles behind each layer of abstraction, then reinforce that understanding with systematic hands-on practice — only then can AI capabilities be truly integrated into enterprise-grade application development.