Java + AI in Practice: Building an Enterprise-Grade Airline Intelligent Customer Service System

LLM Capabilities Are Plateauing — AI Application Development Is the New Blue Ocean

As new models like DeepSeek^Ad V3.1 and GPT-OSS are released one after another, many developers have keenly noticed a trend: the performance improvements of large language models have noticeably slowed down. As instructor Xu Shu pointed out in this course, the benchmark scores of recent new versions don't necessarily surpass those of previous models — LLM capabilities have essentially "plateaued."

This phenomenon is known in the industry as the "Scaling Law bottleneck." Previously, organizations like OpenAI discovered that model performance follows a power-law relationship with training data volume, parameter scale, and compute (i.e., the Scaling Law) — as long as you keep increasing these three factors, model capabilities would steadily improve. However, starting from the second half of 2024, the industry broadly observed diminishing marginal returns from simply adding more parameters and data. The gradual exhaustion of high-quality training data, exponentially growing training costs, and theoretical limits of model architectures themselves have collectively created this "ceiling effect." This is the fundamental reason why the industry's focus has shifted from pre-training to post-training optimization and application-layer innovation.

What does this mean? When model capabilities stabilize, the competitive focus shifts from "building models" to "using models." An isolated LLM by itself cannot generate profitable value — it needs to be deployed into specific business scenarios to truly create value. Therefore, AI application development is becoming a rapidly growing area of demand.

For Java engineers who dominate enterprise development, this is an important signal: there's no need to switch to Python and compete in algorithms. Instead, you can layer AI capabilities on top of your Java foundation, covering both "Java development" and "AI application development" job directions in the market.

Three Types of AI Roles Compared: How Should Java Developers Choose?

The course provides a clear layering of current AI-related positions, which is worth referencing for every developer looking to enter the AI space.

AI Application Development Engineer: Highest Market Demand

This is currently the most in-demand direction. Developers only need to master framework toolchains like Spring AI, understand core skills such as RAG (Retrieval-Augmented Generation), Function Calling, Tools, and prompt engineering, to start developing AI applications.

Spring AI is an AI application development framework officially released by Spring in late 2023, aiming to provide Java developers with an AI development experience comparable to LangChain (in the Python ecosystem). It follows Spring's consistent design philosophy — shielding underlying differences through abstraction layers, allowing developers to connect to different providers like OpenAI, Anthropic, Ollama, and various Chinese LLMs through a unified API. Core modules include: Chat Client, Embedding Client, VectorStore (vector storage abstraction), and native support for patterns like Function Calling, RAG, and Advisor (interceptor chains). For Java developers, Spring AI's greatest advantage is its seamless integration into the Spring Boot ecosystem, enabling direct use of mature components like Spring Security, Spring Data, and Spring Cloud to build enterprise-grade AI applications.

Enterprise-level applications also require stability, and even the "three highs" characteristics needed for internet applications — high concurrency, high availability, and high performance — which happen to be Java's strongest domains.

Model Fine-tuning Engineer: Limited Job Openings

This direction requires mastery of Python and deep learning frameworks like PyTorch and TensorFlow. But the reality is that the number of models a company actually needs to fine-tune is very limited, and they typically hire dedicated fine-tuning engineers, so job demand is relatively low.

AI Algorithm Engineer: Highest Ceiling but Extremely High Barrier

This is the position with the highest ceiling, but it also means extremely high barriers: proficiency in Java, C++, and Python is required, along with mastery of deep learning, machine learning, and even computer vision. It may even require a master's or doctoral degree from top universities and published algorithm papers. The course states directly that for engineers with a regular undergraduate background and several years of Java development experience, this is not the recommended entry point. Moreover, with open-source LLMs thriving today, many models are pre-trained and fine-tuned based on open-source foundations like Qwen, further reducing the demand for proprietary LLM development and consequently reducing these positions.

The conclusion is clear: for most Java engineers, AI application development is the most cost-effective and realistic entry point.

Project Practice: Rebuilding a Traditional Airline Ticketing System with Spring AI

The core of the course is a hands-on project: empowering a traditional airline ticketing system with AI to transform it into an intelligent customer service system.

This transformation approach is highly representative. Traditional systems are "event-driven" — users must click buttons and links to complete queries and operations. After AI empowerment, users can complete all system operations through natural conversation. This also foreshadows a larger trend: more and more traditional systems will be restructured with AI or empowered to form intelligent AI application projects.

Role Preset: Giving the Intelligent Customer Service a Business "Persona"

When a user sends "Hello," the system responds with "Welcome to Turing Airlines." This is the function of role preset — base LLMs (like using DeepSeek directly) don't possess this kind of business identity; it must be achieved through customized prompt engineering. This is also the first essential difference between enterprise-grade AI applications and generic chatbots.

Role preset is typically implemented through a System Prompt. At the beginning of each conversation, developers inject a system-level instruction into the LLM that defines the AI's identity, responsibility boundaries, communication style, and behavioral constraints. For example: "You are a customer service representative for Turing Airlines. Only answer questions related to flight booking and ticket changes/cancellations. Politely decline irrelevant topics." This System Prompt is invisible to users but continuously influences every response from the model. A good role preset includes not only identity definition but also response format requirements, sensitive information handling rules, and fallback strategies for edge cases.

Tools and Function Calling: Enabling LLMs to Call Business APIs

The most critical capability in the project is enabling the LLM to call the system's own business methods. Taking ticket cancellation as an example, the system has a cancelBooking method that requires two parameters: booking number and passenger name. When a user expresses cancellation intent in conversation and provides the information, the LLM invokes this API through the Tools (Function Calling) mechanism, changing the order status from "Booked" to "Cancelled."

The working principle of Function Calling is: when calling the LLM API, developers pass in not only user messages but also a set of "function declarations" (including function names, parameter descriptions, and parameter types in JSON Schema format). After understanding user intent, if the LLM determines that a function needs to be called, it returns a structured JSON object containing the function name to call and parameter values. Note that the LLM itself doesn't execute the function — it only "decides" which function to call and extracts the parameters. The actual function execution is handled by the application, and the execution result is returned to the LLM for final answer generation. This design allows LLMs to safely interact with external systems while maintaining the application layer's complete control over business logic.

This step completes the closed loop from "conversation" to "executing business operations" — the core technical point for AI to truly land in business scenarios.

RAG: Making Answers Align with Real Business Rules

The project also implements RAG (Retrieval-Augmented Generation). For example, when a user asks about the cancellation policy, the system can accurately answer questions like "refunds will be processed within seven business days," "cancellation must be made at least X hours before departure," and "a fee of X will be charged after cancellation." This content is not generated from thin air by the model but retrieved from the enterprise's own business knowledge base, ensuring accuracy and compliance of answers.

The specific technical workflow of RAG includes: 1) Converting enterprise documents (such as cancellation policy files, FAQ manuals, etc.) into high-dimensional vectors using an Embedding model and storing them in a vector database (such as Milvus, Pinecone, Chroma, PGVector, etc.); 2) When a user asks a question, converting the question into a vector and performing similarity search in the vector database (typically using cosine similarity or inner product distance); 3) Concatenating the Top-K most relevant document fragments into the Prompt as context; 4) The LLM generates answers based on this real information rather than relying on potentially outdated or inaccurate knowledge memorized during training. RAG effectively solves the LLM "hallucination" problem — the phenomenon where models confidently generate fabricated information — ensuring every answer is backed by verifiable sources.

Complete Tech Stack Analysis for Enterprise-Grade AI Applications

Overall, this airline intelligent customer service project covers the key capability matrix of enterprise-grade AI application development:

• LLM Integration: Connecting to open-source LLMs as the underlying capability
• Multi-turn Conversation and Memory: Maintaining contextual coherence
• Role Preset: Customizing business identity and communication style
• Conversation Interception: Filtering non-compliant or out-of-scope requests
• Tools/MCP: Enabling LLM interaction with business system APIs
• RAG: Providing precise answers based on enterprise knowledge bases

MCP (Model Context Protocol) is a standard protocol open-sourced by Anthropic in November 2024, designed to solve the interoperability problem between LLMs and external tools/data sources. Before MCP, every AI application needed to write custom integration code for different tools, creating an "M×N" combinatorial explosion problem. MCP defines a unified communication protocol, abstracting tool providers as "MCP Servers" and AI applications as "MCP Clients," with both communicating through a standardized JSON-RPC protocol. This is similar to how the USB protocol unifies peripherals — any tool conforming to the MCP specification can be called by any AI application supporting MCP. Spring AI already natively supports the MCP protocol, enabling Java developers to conveniently expose business systems as MCP Servers or consume third-party MCP services.

Beyond this, the course also emphasizes the pain points and challenges of enterprise-grade AI application development — which is also the dividing line between real projects and demos. A demonstrable chat dialog is easy to build, but achieving stability, reliability, and the ability to handle real business traffic is the true engineering challenge.

In AI application scenarios, the "three highs" challenge is further amplified. Regarding high concurrency, LLM inference itself takes a long time (typically hundreds of milliseconds to several seconds) — supporting many concurrent requests with limited GPU resources requires strategies like asynchronous streaming output (SSE/Server-Sent Events), request queues, and rate limiting/degradation. Regarding high availability, when the primary model service fails, the system needs to quickly switch to a backup model (e.g., degrading from GPT-4o to GPT-4o-mini), which requires model routing and failover capabilities at the application layer. Regarding high performance, RAG retrieval latency, multi-round interactions in Function Calling, and storage/loading of conversation history are all performance bottleneck points. The Java ecosystem has mature solutions for these aspects (such as Spring WebFlux for reactive programming, Redis for caching conversation history, Sentinel for rate limiting and circuit breaking, etc.) — this is the unique advantage Java developers have in building AI applications compared to Python developers.

Transition Advice for Java Engineers

For Java developers with several years of experience, rather than anxiously switching to the Python algorithm track, it's better to layer AI application development capabilities onto your existing tech stack. This path aligns with actual market demand trends and maximizes your existing Java engineering experience (especially your understanding of stability and high-concurrency architectures).

When LLM capabilities converge, real value will shift from the "model layer" to the "application layer." Whoever can engineer LLM capabilities and deploy them into specific business scenarios will seize the most solid opportunity in this wave of AI. And this airline intelligent customer service project is a complete, reproducible enterprise-grade hands-on template.

Key Takeaways