Spring AI 2.0 in Practice: A Beginner's Guide for Java Developers Building AI Agents

Why Java Developers Should Pay Attention to Spring AI 2.0

Spring AI is gaining tremendous momentum in the Java ecosystem, and the release of version 2.0 has opened a convenient gateway for Java developers to build AI-native applications. As the official AI engineering framework from the Spring team, Spring AI's greatest value lies in enabling Java developers to build AI applications using familiar patterns — without abandoning their existing Spring Boot expertise.

For engineers who have long worked in the Java space, AI application development used to mean switching to the Python ecosystem, or grappling with complex challenges like prompt management, model integration, and enterprise data connectivity. This predicament has historical roots: when LangChain was open-sourced in Python in October 2022, its chain-based invocations, rich integrations, and active community quickly established it as the de facto standard in AI engineering, cementing Python's dominance and leaving Java developers as "second-class citizens" in the AI toolchain. Spring AI was created to bring Spring's mature design philosophy — portability, modularity, POJO-first design, and auto-configuration — into AI development, dramatically lowering the barrier to entry.

It's worth mentioning that the Java ecosystem also has another powerful AI framework: LangChain4J. It's the Java port of the LangChain framework, with an API design style closer to Python's LangChain, making it more familiar to developers with Python ecosystem experience. Spring AI, on the other hand, takes a "Spring-native" approach, seamlessly embedding AI capabilities into existing tech stacks via Spring Boot Starters, resulting in lower migration costs for pure Java engineers. Both frameworks are in rapid development and are essentially at the same starting line. Developers should explore both and choose flexibly based on their project requirements.

Core Positioning and Design Philosophy of Spring AI

To understand Spring AI, you first need to understand its positioning: it is not a large model itself and does not provide core generative AI algorithms. Instead, it serves as middleware between the Spring ecosystem and various AI models, responsible for unifying access standards.

Spring AI's core mission is to solve the fundamental challenges of AI integration. Before frameworks existed, developers had to manually handle prompt construction, integration with different model APIs, and connecting enterprise data with models. Spring AI uses a framework-level abstraction layer to efficiently connect enterprise data, external APIs, and AI models.

In terms of design inspiration, Spring AI draws from mature Python ecosystem solutions like LlamaIndex and LangChain, but it's purpose-built for the Spring ecosystem, naturally and seamlessly integrating with Spring Boot and other Spring components.

Unified API: The Core Abstraction That Hides Provider Differences

One of Spring AI's key features is its unified API abstraction. With numerous AI providers on the market, maintaining separate integration code for each model would be prohibitively expensive. Spring AI provides highly encapsulated abstractions over these APIs, allowing developers to switch underlying models through configuration alone, with virtually no code changes required.

This unified abstraction design philosophy stems from Spring's longstanding "program to interfaces" principle. By defining core interfaces like ChatModel and EmbeddingModel, the framework hides provider-specific implementations behind an abstraction layer — much like how Spring Data abstracts different databases (MySQL, MongoDB, Redis) and Spring Security abstracts authentication mechanisms (OAuth2/SAML). This design not only enables seamless multi-model switching but also provides the architectural foundation for simultaneously calling multiple models for A/B testing and cost comparison, making multi-model strategies a matter of configuration rather than code refactoring.

The underlying support for this abstraction capability comes from Spring Boot's Auto-configuration mechanism. Through the @EnableAutoConfiguration annotation and the META-INF/spring.factories mechanism, the framework automatically handles Bean registration and parameter binding at startup based on the Spring AI Starter dependencies present in the classpath. Developers only need to fill in a few parameters like API Keys in application.properties. This means switching from OpenAI to Tongyi Qianwen feels no different from switching a database driver — a direct extension of Spring ecosystem's most valuable engineering philosophy.

Full AI Model Type Support

Spring AI 2.0 provides comprehensive coverage of AI capabilities, including:

• Chat Completion: Conversational Q&A and text generation
• Embedding Models: Support for vectorization and semantic retrieval. Embedding models convert text into fixed-dimensional floating-point vectors, making semantically similar content closer in vector space. This is the core foundation of RAG technology. Spring AI provides unified interface support for mainstream embedding models including OpenAI's text-embedding-3 series and open-source models like BGE.
• Multimedia Capabilities: Text-to-image generation, speech transcription, text-to-speech
• Content Moderation: Ensuring generated content is safe and compliant
• Structured Output: AI responses can be directly mapped to Java objects, significantly improving development efficiency

Structured output is particularly valuable for Java developers — AI results can be directly mapped to entity classes, eliminating extensive manual parsing. Spring AI uses components like BeanOutputConverter and MapOutputConverter to automatically deserialize model text output into strongly-typed Java objects via Jackson under the hood, while injecting the target class's JSON Schema into system instructions through prompt templates to guide the model toward generating structured responses that conform to format constraints. This mechanism essentially uses JSON Schema as a "contract," bridging the gap between human-readable natural language interfaces (prompts) and machine-processable structured data (Java objects), fully leveraging Java's natural advantages as a strongly-typed language for data modeling.

It's worth adding that JSON Schema itself is a vocabulary for describing and validating JSON data structures, maintained by the IETF standards organization. In structured output scenarios, Spring AI uses reflection at runtime to scan the target Java class's field types and annotations (such as @JsonProperty, @NotNull, etc.), dynamically generates the corresponding JSON Schema string, and embeds it at the end of the system prompt in a form like "Please strictly return results according to the following JSON Schema format: {...}". When the model receives this constraint, it follows the format requirements during generation. For models that support "JSON Mode" or "Structured Outputs" (such as GPT-4o, Claude 3.5), the output format can also be enforced at the API parameter level, eliminating format errors at the syntax level — a significant reliability improvement over prompt-only constraints.

Complete Spring AI 2.0 Learning Path

Developing a Java Agent from scratch requires progressively mastering multiple knowledge modules. The following learning path covers the complete journey from beginner basics to enterprise-level practice.

Fundamentals: From Hello World to Tool Calling

The core goal at the beginner stage is to quickly set up your first Spring AI application:

1. Hello World Setup: Understand the basic usage of the framework
2. Cloud Platform Integration: Connect to mainstream LLM cloud services (such as OpenAI, Tongyi Qianwen, etc.)
3. Ollama Local Deployment: Run open-source large models in a local environment. Ollama is currently one of the most popular local LLM runtimes. It wraps underlying inference engines like llama.cpp through a unified REST API, supporting one-click download and execution of dozens of mainstream open-source models including Llama 3, Mistral, and Qwen2 — making it the preferred option for zero-cost AI capability testing during development. Its underlying llama.cpp is implemented in pure C/C++, deeply optimized for CPU inference, and supports 4-bit, 8-bit, and other quantization formats, enabling ordinary developer laptops to smoothly run models with 7 billion parameters. Spring AI has built-in comprehensive support for Ollama, allowing seamless switching between local debugging and cloud deployment.
4. Advisor Interceptor Pattern: Enhance AI interaction capabilities through interceptors. The Advisor mechanism is Spring AI's creative application of Aspect-Oriented Programming (AOP) thinking in the AI domain — developers can insert custom logic before requests are sent and after responses are returned, implementing cross-cutting concerns like logging, content filtering, and conversation history injection without modifying core business code. Spring AI includes built-in Advisors like MessageChatMemoryAdvisor (automatic conversation memory management) and QuestionAnswerAdvisor (automatic RAG retrieval injection), which can be combined as needed to form processing chains. This follows the same philosophy as traditional Spring AOP, where aspect logic is woven around method calls — except the join point shifts from Java method invocations to the request-response lifecycle of AI models.
5. Conversation and Prompt Engineering: Master methods for efficient communication with models. Prompt engineering is not merely a "conversation skill" — it encompasses a series of engineering techniques including System Prompt design, Few-Shot Examples injection, and Chain-of-Thought Prompting. Chain-of-Thought prompting (proposed by Wei et al. from the Google Brain team in 2022) significantly improves model accuracy on complex tasks like mathematical reasoning and logical judgment by demonstrating "step-by-step reasoning" examples in prompts. The underlying principle is guiding the model to make intermediate computation processes explicit rather than jumping directly to conclusions. Spring AI provides structured prompt management capabilities through classes like PromptTemplate and SystemPromptTemplate, supporting variable interpolation and template reuse, effectively avoiding the anti-pattern of hard-coding prompts in business logic.
6. Tool Calling (Function Calling / Tools): Give AI the ability to invoke external capabilities. Tool calling is the key mechanism for large models to evolve from "text generators" to "action executors" — developers describe callable functions to the model in JSON Schema format, and when the model determines during inference that external capabilities are needed, it returns structured invocation instructions. The application layer executes these and feeds results back, forming a "think-act-observe" Agent loop. This capability was first officially introduced by OpenAI in June 2023 with the GPT-4 API, and was subsequently adopted by Claude, Gemini, and other major models, becoming a de facto standard. Spring AI abstracts this capability into a unified Tools interface, hiding implementation differences across providers like OpenAI and Claude.

From an underlying principle perspective, tool calling capability is possible because models are specifically injected with "function calling" instruction following ability during the training phase. The model doesn't actually "execute" code — instead, it learns to output specially formatted text (typically JSON objects) at appropriate moments. The application framework parses this text, calls the real function on behalf of the model, appends the result to the conversation history, and the model continues reasoning based on the updated history. This mechanism essentially transforms "calling external tools" into a text generation task familiar to the model — a brilliant interface design between large model capabilities and software engineering practices.

Intermediate: RAG (Retrieval-Augmented Generation)

After completing the fundamentals, RAG (Retrieval-Augmented Generation) is the most critical intermediate topic. RAG was formally proposed by the Meta AI research team in 2020 in their paper Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Its core idea is to retrieve relevant document fragments from an external knowledge base before LLM inference, then inject these fragments as context into the prompt, enabling the model to provide more accurate, verifiable answers. When building AI Agents, numerous scenarios rely on RAG technology — it injects enterprise private knowledge into the model's responses through semantic vectorization and vector database retrieval, fundamentally addressing the pain points of limited model knowledge boundaries and hallucination tendencies.

To understand the necessity of RAG, you need to first understand the causes of LLM "hallucination." Large language models are essentially probability distribution estimators that learn statistical patterns of token sequences during training, rather than databases that store factual truth. When asked about events after their training data cutoff, internal enterprise data, or highly specialized domain knowledge, models tend to "plausibly fabricate" — generating content that is grammatically fluent and logically coherent but factually incorrect. RAG essentially provides "open-book exam" conditions for the model: by injecting relevant document fragments into the prompt context in real-time, it transforms "answering from memory" into "answering based on reference materials," fundamentally anchoring the factual basis of model output. Additionally, by requiring the model to cite sources, answer traceability verification can be achieved.

Notably, RAG has evolved from its initial "Naive RAG" form into multiple enhanced variants in engineering practice: Modular RAG introduces pre-retrieval processing strategies like Query Rewriting and HyDE (Hypothetical Document Embedding — having the model generate a hypothetical answer first, then using it for retrieval), effectively improving retrieval recall; GraphRAG (proposed by Microsoft in 2024) captures complex relationships between entities by constructing document knowledge graphs, handling complex Q&A scenarios requiring multi-hop reasoning; Self-RAG lets the model autonomously decide when retrieval is needed and whether to adopt retrieval results, giving the system stronger adaptive capabilities. Spring AI provides extension points for these advanced RAG patterns through its extensible DocumentRetriever and QuestionAnswerAdvisor interfaces, allowing developers to gradually evolve from the framework's basic capabilities to more sophisticated retrieval strategies.

At the engineering implementation level, RAG typically requires embedding models to convert text into high-dimensional vectors, paired with vector databases for semantic similarity retrieval. Vector databases are database systems specifically designed for storing and retrieving high-dimensional floating-point vectors. Unlike the exact matching of traditional relational databases, they use Approximate Nearest Neighbor (ANN) algorithms to find the most semantically similar content from massive vector collections in milliseconds. Among these, HNSW (Hierarchical Navigable Small World) is currently one of the best-performing ANN algorithms — proposed by Russian researchers Malkov and Yashunin in 2016, it organizes vectors into a network topology resembling "six degrees of separation" through a multi-layer graph structure: the top layer is a sparsely connected "express lane" graph, with increasingly dense node coverage in lower layers. During queries, the algorithm quickly locates the approximate region through the top sparse graph, then descends layer by layer to the bottom dense graph for precise searching — like taking a high-speed train to reach the target city, then walking to find the specific address. This achieves millisecond-level recall in million-scale vector databases while maintaining over 90% approximate precision, far superior to brute-force linear scanning. Mainstream options include dedicated vector databases Milvus, Qdrant, and Weaviate, as well as pgvector — a PostgreSQL extension that adds vector capabilities, offering extremely low migration costs for teams with existing PostgreSQL stacks. Spring AI provides out-of-the-box integration support for all these mainstream vector databases, allowing developers to operate through a unified VectorStore interface without worrying about underlying implementation differences, and flexibly switch based on production environment requirements. This component is the technical foundation for building enterprise knowledge base systems.

Practical Projects: Enterprise-Level AI Application Deployment

The learning path includes two enterprise-level practical projects:

• RAG Enterprise Knowledge Base System: An intelligent Q&A platform built on retrieval-augmented generation
• AI Intelligent Customer Service System: One of the most widely commercially deployed AI application scenarios

These two directions represent the most mainstream AI application deployment scenarios today and offer strong practical reference value. It's worth noting that a production-grade AI Agent system's core architecture typically includes four modules: Planning Module (decomposing complex tasks into subtasks using strategies like ReAct or Plan-and-Execute), Memory Module (short-term conversational context memory and long-term semantic memory based on vector databases), Tool Module (callable capabilities such as external APIs, database queries, code interpreters, etc.), and Execution Engine (orchestration logic that coordinates the above three).

The core paradigm of the Planning Module — ReAct (Reasoning + Acting) — was proposed by Google's research team in 2022. It maintains reasoning chain coherence in complex multi-step tasks by alternately generating "Thought → Action → Observation" triplets within prompts. ReAct's deeper insight is that pure Chain-of-Thought excels at reasoning but lacks the ability to interact with the external world, while pure action execution (like WebGPT's search-read loops) lacks reasoning transparency. ReAct fuses both — outputting readable thought processes before each action and incorporating environmental observations into the next round of thinking, forming a self-correcting closed loop. This design enables Agents to automatically adjust strategies and retry when tool calls fail or return unexpected results, performing significantly better than pure Chain-of-Thought on tasks requiring tool invocation. Spring AI 2.0 provides out-of-the-box infrastructure for building this complete architecture through its unified Tools interface, Advisor interceptor chains, and VectorStore abstractions — the shared technical pillars behind both practical projects.

Regarding the engineering implementation of the Memory Module, short-term memory (conversation context) and long-term memory (cross-session semantic memory) face distinctly different challenges. Short-term memory is constrained by the LLM's context window — even for models supporting 128K or even million-token contexts, the strategy of stuffing entire conversation history into prompts leads to increased inference latency and dramatically rising costs as conversation turns increase. Common engineering strategies include: sliding window truncation (keeping only the most recent N turns), summary compression (using the model to summarize conversation history into brief summaries), and selective memory injection based on vector retrieval (retrieving only history fragments semantically relevant to the current question). Spring AI's MessageChatMemoryAdvisor currently supports the first two strategies, and developers can implement the third, more refined memory management approach by combining it with VectorStore.

MCP Protocol: An Essential Component in Agent Development

The learning path also specifically covers MCP (Model Context Protocol). MCP was officially open-sourced by Anthropic in November 2024, aiming to solve the fragmentation problem of interconnection between AI models and external tools and data sources. Before MCP, every AI application needed to develop separate integration code for different data sources and tools, making maintenance costs extremely high. MCP adopts a client-server architecture and defines unified communication specifications, enabling any standards-compliant tool (file system, database, browser control, etc.) to be directly invoked by any MCP-supporting model or framework — similar to a "USB port" standard for the AI world.

From a technical implementation perspective, MCP Servers expose tool lists (tools), resources, and prompt templates via the standardized JSON-RPC 2.0 protocol. JSON-RPC 2.0 is a lightweight remote procedure call protocol that uses JSON as its data format, implementing request-response communication through standardized method, params, and id field structures — more suitable than RESTful APIs for expressing semantics like "call a function and get a result." The MCP protocol also supports two transport layer modes: stdio mode (communication via standard input/output, suitable for local tool integration, widely adopted by desktop clients like Claude Desktop) and SSE mode (Server-Sent Events, HTTP long-connection-based streaming push, suitable for remote service deployment). MCP clients (i.e., the AI application layer) obtain the server's capability manifest through an initialize handshake at runtime, then dynamically discover and invoke these capabilities — the entire process being transparent to the underlying model.

From an ecosystem evolution perspective, MCP's emergence coincides with a critical window — after AI tool calling (Function Calling) capabilities became widespread but before multi-Agent collaboration systems fully emerged. Early tool calling was a private contract between the application layer and a single model, lacking standards for cross-application reuse. MCP elevates this private contract to an open standard: the same tool server can simultaneously serve Claude Desktop's conversational completions, Cursor's code assistance, and custom Agent task execution. Tool developers only need to maintain a single MCP server implementation without adapting separately for each host platform. This carries historical significance similar to how RESTful APIs unified front-end and back-end communication standards in the mobile internet era, and how OCI specifications unified container image formats in the container era — liberating productivity through standardized interfaces and driving ecosystem-wide division of labor and prosperity.

The far-reaching significance of this decoupled design is that it transforms the tool ecosystem from "redundantly developing for each application" to "develop once, reuse everywhere" — the same MCP tool server can be simultaneously reused by multiple hosts including Claude Desktop, Cursor, Zed, and custom Agents, dramatically reducing redundant integration work. MCP shares the same spirit as REST API standards in the web domain and JDBC standards in the database domain — all breaking ecosystem fragmentation through unified specifications. Currently, mainstream AI products including Claude, Cursor, and Zed all support MCP, and Spring AI 2.0 has incorporated it into its official support scope, making it an important foundation for building complex Agent systems.

Spring AI vs. Similar Frameworks

Compared to similar frameworks like LangChain4J, Spring AI's greatest advantage is its deep integration with the Spring ecosystem. For teams already using Spring Boot, integration costs are extremely low, and they naturally benefit from Spring's auto-configuration, dependency injection, and other conveniences. LangChain4J, on the other hand, has an API design closer to Python's LangChain style, making it more approachable for developers with Python ecosystem backgrounds. The two frameworks have different emphases in their positioning.

In terms of feature completeness, both frameworks have their respective strengths and are both in rapid iteration — it's too early to definitively judge which is superior. Developers are advised to familiarize themselves with both and make choices based on their team's tech stack and project characteristics.

Prerequisites Before Learning

It's important to note that this tutorial has certain prerequisites. Learners should have:

• Solid Java fundamentals, especially core Spring Boot concepts (dependency injection, auto-configuration, Starter mechanism, etc.)
• Basic front-end knowledge

If your foundations aren't solid, it's recommended to systematically fill in core Java and Spring Boot knowledge before entering AI application development. Spring AI is essentially built on top of Spring Boot — a strong foundation is needed to build tall structures. Spring Boot's core mechanisms — Dependency Injection (DI), Aspect-Oriented Programming (AOP), and Conditional Bean Assembly (@Conditional) — are directly reflected in Spring AI's Advisor interceptors and automatic model configuration features. Understanding these underlying principles will significantly flatten the learning curve.

The Conditional Assembly mechanism (@ConditionalOnClass, @ConditionalOnProperty) in particular is the technical foundation for Spring AI automatically activating corresponding model configurations based on Starters present in the classpath. For example, when spring-ai-openai is present in the classpath, the framework auto-configures OpenAiChatModel; when spring-ai-ollama is present, it configures OllamaChatModel — without interference between the two. The underlying implementation relies on Spring Boot's AutoConfigurationImportSelector, which scans META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports files (Spring Boot 2.7+ format) across all Starter packages at application startup, collects candidate auto-configuration classes, evaluates each one's @Conditional conditions, and ultimately only includes configuration classes whose conditions are met into the ApplicationContext. Understanding this mechanism helps quickly locate configuration issues: when a model Bean isn't created correctly, it's often because the conditional annotation's trigger conditions aren't met. Using Spring Boot Actuator's /actuator/conditions endpoint allows you to directly see which auto-configurations were activated, which were skipped, and why — enabling efficient troubleshooting without diving into framework source code.

Additionally, having a foundation in Reactive Programming is highly beneficial for deep use of Spring AI, although it's not a prerequisite for getting started. Spring AI's streaming output feature is based on Project Reactor's Flux<ChatResponse> — content generated token-by-token by the model is pushed to the client in real-time via reactive streams, avoiding the need for users to wait for the complete response to be generated. For scenarios requiring real-time typewriter-effect front-end interactions or long text generation, understanding Flux's subscription model and Backpressure mechanism, as well as how to combine it with Spring WebFlux or SSE endpoints, will enable you to handle streaming AI responses more elegantly rather than forcing them into a blocking programming model.

Conclusion

Spring AI 2.0 provides Java developers with an efficient path to enter AI application development using familiar patterns. With its unified API abstraction, comprehensive model type support, structured output, and seamless integration with the Spring ecosystem, it significantly lowers the barrier to AI integration. For Java engineers looking to establish their footing in the AI wave, mastering the Spring AI framework along with key technologies like RAG, tool calling, and the MCP protocol enables building production-grade Agent systems from scratch. All major frameworks are still in their early stages, and Java developers stand at the same starting line as other ecosystems — now is the perfect time to get involved.