Java Developer's Guide to AI Application Development: Building an Airline Intelligent Customer Service System

Large Models Have Plateaued — AI Applications Are the Next Big Opportunity

Over the past year or two, large language models have been iterating at a dizzying pace — with new models released nearly every week or two. But as we enter the current phase, this breakneck pace of updates has noticeably slowed. Recently released open-source models like DeepSeek V3.1 and GPT-OSS show no qualitative leap over their predecessors, with some benchmark scores even falling behind older models.

There are deep technical reasons behind this: the slowdown in large model performance reflects the diminishing marginal returns of deep learning's Scaling Law. This law was systematically articulated by OpenAI researcher Kaplan et al. in their 2020 paper Scaling Laws for Neural Language Models. The core finding is that model performance (measured by cross-entropy loss) follows a power-law relationship with parameter count, data volume, and compute — meaning that each doubling of model quality requires exponentially more resources. In 2022, DeepMind published the Chinchilla paper, which corrected the optimal training ratio: given a fixed compute budget, model parameters and training tokens should maintain roughly a 1:20 ratio, rather than simply scaling up parameters. This correction drove a paradigm shift from "stacking parameters" to "optimizing data quality and training efficiency." With mainstream large models already reaching hundreds of billions of parameters, the cost-benefit ratio of further parameter scaling is deteriorating, leading model providers to focus instead on Test-Time Compute, multimodal capabilities, and application-layer architectures.

Notably, behind the Scaling Law hitting its ceiling lies the "Data Wall" problem — there's a physical upper limit to the total amount of high-quality text data available on the internet, with mainstream estimates around 10^13 to 10^14 tokens, and current largest-scale model training has already consumed a significant portion. This has pushed researchers toward two new paths: Synthetic Data Generation (having models generate their own training samples to break through the data bottleneck) and Test-Time Compute (scaling inference-stage computation through Chain-of-Thought, Best-of-N sampling, MCTS tree search to achieve higher accuracy). Test-Time Compute essentially "shifts" compute from the training phase to the inference phase — instead of directly outputting answers, the model "thinks longer" through multi-path sampling, self-verification, and search tree expansion to improve accuracy. OpenAI's o1/o3 series are representative products of this approach: o1 significantly outperforms GPT-4o of similar parameter scale on reasoning-intensive tasks like math competitions and code generation, with the core difference being not parameter scale but the introduction of reinforcement learning-driven long chain-of-thought reasoning during inference. This marks a shift in the main battlefield for improving large model capabilities from "stacking compute during training" to "expanding search during inference."

This also sends an important signal: the foundational capabilities of large models have largely stabilized, and achieving major breakthroughs in pure model performance is becoming increasingly difficult.

So the question becomes — what good is a powerful large model on its own? The answer is straightforward: if a model can't be deployed in production and generate revenue, it's just an expensive chatbot. The real value lies in embedding large model capabilities into specific business scenarios to create deliverable, production-ready AI applications.

This is precisely the scarcest capability in today's market. More and more traditional systems will be rebuilt or enhanced with AI, creating massive demand for AI application development roles.

Three Types of AI Roles — Which Should Java Developers Choose?

Current AI-related technical positions fall roughly into three categories, each with different barriers to entry and career prospects.

AI Algorithm Engineer: Highest Ceiling, Highest Barrier

This direction has an extremely high ceiling, but the number of people worldwide who can truly do foundational AI algorithm research is vanishingly small. It requires solid mathematical foundations (linear algebra, probability theory, optimization theory), deep knowledge of deep learning and machine learning, often specialized skills in areas like computer vision, and has hard requirements on education (top-tier graduate programs) and publication records. Core positions are typically concentrated in a handful of leading AI labs and major tech company research institutes, with extremely limited absolute numbers of positions and competition primarily from candidates with top academic credentials.

For developers with a standard bachelor's degree and several years of Java experience, this direction is not recommended — positions are scarce, barriers are high, and the ROI doesn't make sense, unless you already hold a graduate degree from a top university in a relevant field and are willing to go all-in.

Model Fine-Tuning Engineer: Requires Python, Limited Positions

Model fine-tuning positions require proficiency in Python and deep learning frameworks like PyTorch and TensorFlow, along with parameter-efficient fine-tuning techniques such as SFT (Supervised Fine-Tuning), RLHF (Reinforcement Learning from Human Feedback), and LoRA. But the reality is: how many models does a single company actually need to fine-tune? Very few, which means the absolute volume of these positions is also limited.

Furthermore, many domestic Chinese models currently use open-source models like Qwen as their base, completing pre-training and fine-tuning before public release. This further shrinks the demand for companies to build their own models from scratch, narrowing the space for fine-tuning positions.

AI Application Development Engineer: The Direction with the Highest Demand

For Java developers, this is the direction most worth investing in. The logic is simple: large models have stabilized, what's missing are production applications, and massive numbers of AI applications are waiting to be built — naturally requiring large numbers of AI application development engineers.

More critically, Java developers don't need to switch to Python. They only need to supplement their existing skill set with an AI tech stack:

• LangChain4j / Spring AI — Native AI application development frameworks for the Java ecosystem
• RAG (Retrieval-Augmented Generation)
• Function Calling / Tools
• MCP (Model Context Protocol)
• Prompt engineering and model fine-tuning fundamentals

LangChain4j is a Java port of the widely popular Python LangChain framework. Open-sourced in 2023, it provides out-of-the-box modules for AI service abstraction, memory management, RAG pipelines, agents, and more, with support for mainstream model providers including OpenAI, Anthropic, and Ollama. LangChain4j's core design philosophy abstracts common AI application components — model invocation, tool integration, memory management — into reusable building blocks, allowing developers to rapidly assemble AI applications with conversation, retrieval, and reasoning capabilities without worrying about underlying HTTP call details. Spring AI, launched by the official Spring team in late 2023, is an AI integration framework deeply integrated with the Spring Boot ecosystem, following the "convention over configuration" design philosophy that Java enterprise developers know and love. Spring AI's advantage lies in its seamless integration with the Spring ecosystem — developers can manage AI model connections using familiar patterns like @Bean injection and application.properties configuration, blending seamlessly with existing Spring Boot microservice architectures. Both frameworks support Function Calling, RAG, MCP, and other mainstream AI application patterns, and compared to calling raw REST APIs from scratch, they encapsulate tedious details like model switching, retry logic, and token counting into unified interfaces, dramatically reducing engineering complexity.

Why Java Is the Best Choice for AI Application Development

"Should I use Java or Python for AI?" is a common question, and the answer depends on the use case.

Python undeniably has a natural advantage in training and fine-tuning large models — mainstream deep learning frameworks like PyTorch and TensorFlow treat Python as a first-class citizen, and the reference implementations of most academic papers are in Python. But when the goal is building enterprise-grade AI applications, Java's strengths become apparent.

Enterprise systems typically need to meet high concurrency, high availability, and high performance requirements while supporting long-term maintenance and iteration. Java, as a strongly-typed language, offers more rigorous code standards, better IDE support, and higher system reliability. Python's dynamic typing, by contrast, can introduce hidden issues from type inconsistencies in large, long-lived projects and offers relatively weaker code maintainability in large-team collaboration. Additionally, Java's JVM ecosystem has accumulated far more mature solutions than Python for enterprise infrastructure like service governance (Spring Cloud), distributed transactions, and message queues (Kafka, RocketMQ).

An even more practical point: Java is already the most widely used language for enterprise systems. Since so many business systems run on Java, enhancing these systems with AI capabilities using Java is the most natural choice. For Java developers, this isn't a career change — it's layering AI capabilities on top of existing skills, gaining competitive advantages in both Java development and AI application development simultaneously.

Hands-On Project: Adding Intelligent Customer Service to an Airline Ticketing System

The hands-on case study in this tutorial involves AI-enabling a traditional airline ticketing system by adding an intelligent customer service module.

Traditional systems use an event-driven paradigm — users must click buttons or links to complete operations. After AI enhancement, users can complete all system operations directly through natural language conversation.

Role Preset: Customized Customer Service Identity

When a user sends "Hello," the system responds with "Welcome to [Airline Name]." This kind of branded, scenario-specific response isn't something a base large model can provide — if you ask DeepSeek directly, it would never claim to be a specific airline's customer service agent. This requires customized development through System Prompt (role preset), which is the first step in AI application deployment.

The System Prompt is injected by the developer before the conversation begins, declaring the model's identity, behavioral boundaries, and response style. It's the most direct and lowest-cost engineering method for controlling model output behavior. From an engineering perspective, the System Prompt is essentially a highest-priority "implicit instruction" — in a large model's attention mechanism, content at the front of the context has the highest influence weight on output. System Prompts leverage this characteristic by placing complete instructions at the very front of the context window in every conversation, globally constraining the model's role awareness, output format, safety boundaries, and other behaviors. In engineering practice, System Prompts typically contain four core elements: Identity declaration ("You are the intelligent customer service assistant for [Airline Name]"), Capability boundaries ("Only answer questions related to flight booking"), Response style ("Use polite, concise language"), and Safety guardrails ("Do not reveal internal system implementation details"). Compared to fine-tuning models, System Prompts cost virtually nothing to adjust, making them one of the core levers for rapid iteration of AI applications.

Tools/Function Calling: Letting the Large Model Drive System APIs

When a user requests a ticket cancellation, the system needs to call the business method cancelBooking, which takes a booking number and name as parameters. The core technology here is Function Calling / Tools.

Function Calling was officially introduced by OpenAI in June 2023 with GPT-4. It allows developers to describe a set of callable function signatures to the large model (including function name, parameter names, types, and descriptions). After understanding user intent, the model outputs which function to call and the corresponding parameters in structured JSON format, rather than returning natural language directly. The developer parses the JSON, executes the actual business logic, and returns the execution result to the model, which then generates the final natural language response. This mechanism essentially makes the large model an "intent parser," converting unstructured natural language into structured instructions that programs can precisely execute — a core bridging technology for connecting AI applications with existing business systems.

It's particularly worth emphasizing that the large model itself does not execute functions directly — it only "decides which function to call and what parameters to pass." Actual function execution authority always remains in the developer's hands. This design ensures that business logic security boundaries cannot be breached by the model: developers can add permission checks, parameter validation, operation auditing, and other security controls at the function execution layer without worrying about the model bypassing these safeguards. Function Calling's "model decides, code executes" division of labor is the fundamental reason it's more accepted than letting models generate executable code (Code Interpreter mode) in enterprise production environments.

Function Calling is also the foundational primitive for building AI Agents. When tool calling chains extend into multi-step autonomous decision-making, they evolve into an Agent architecture — the model autonomously plans, calls tools, processes results, and decides next steps in a "perceive-reason-act" loop. Security is the core challenge for Agent deployment, requiring tool permission hierarchies (read-only vs. read-write), operation audit logs, Human-in-the-Loop confirmation mechanisms, and defenses against Prompt Injection attacks — where malicious content infiltrates the context through tool return results to induce the model to execute unauthorized operations.

Demo flow: User provides booking number and name → Large model asks for confirmation → User replies "Confirm" → Order status changes from "Booked" to "Cancelled." The entire process is completed through conversation, with the large model serving as a bridge between natural language and system APIs.

RAG: Augmenting with an Enterprise Knowledge Base

Cancellation scenarios involve numerous business rules: refunds processed within 7 business days, must be completed a certain number of hours before flight departure, applicable cancellation fees, etc. A base large model has no knowledge of this enterprise-specific business information.

The solution is RAG (Retrieval-Augmented Generation). Proposed by the Meta AI research team in 2020, RAG's core idea is to retrieve relevant content from an external knowledge base before generating an answer, providing it as context to the large model so that responses are evidence-based rather than relying entirely on knowledge baked into model parameters during training. Compared to fine-tuning enterprise knowledge directly into model parameters, RAG offers real-time knowledge updates (just modify the document library — no retraining needed), source traceability (answers can cite their sources), and low cost (no GPU training resources required).

RAG's core workflow consists of two phases: the offline phase chunks enterprise documents and converts them into high-dimensional vectors via an Embedding model, storing them in a vector database; the online phase similarly vectorizes user queries, performs similarity search in the vector database, retrieves the most relevant document chunks, and concatenates them into the prompt before sending everything to the large model.

Embedding is the technology of mapping text into a high-dimensional continuous vector space — for example, mapping a passage of text to a 1536-dimensional floating-point vector. Semantically similar content has closer Euclidean distance or cosine similarity in vector space, which is the mathematical foundation of semantic search. Compared to traditional keyword search (BM25, etc.), vector search can understand queries that are semantically similar but worded differently — for example, "how to cancel an order" and "how to process a refund" would be mapped to nearby vector positions. Mainstream Embedding models include OpenAI's text-embedding-3 series (1536/3072 dimensions), Alibaba's text-embedding-v3, and open-source options like BGE and E5. Vector databases are optimized for storing high-dimensional vectors and Approximate Nearest Neighbor (ANN) search, with popular choices including Milvus (Chinese open-source, suitable for private deployment), Pinecone (cloud-native SaaS), and Chroma (lightweight local development). The "chunking strategy" in the RAG pipeline is equally critical — chunks that are too large introduce noise, while chunks that are too small lose context. In practice, hybrid strategies combining semantic boundaries, paragraph structure, and sliding windows are commonly used to optimize retrieval precision.

RAG also faces retrieval precision as a core bottleneck in real-world engineering. Sparse retrieval (BM25 keyword matching) and dense retrieval (Embedding vector search) each have their strengths — the former performs better with technical terms and exact matches, while the latter has stronger semantic generalization. Engineering practice therefore often employs Hybrid Search with a Reranker (re-ranking model) for secondary precision ranking of candidate results, combining the advantages of both exact matching and semantic generalization. A Reranker is essentially a Cross-Encoder that jointly encodes the query with each candidate document chunk to output more precise relevance scores, though its computational cost is higher than vector search, so it's typically used only for re-ranking Top-K candidate results. Additionally, HyDE (Hypothetical Document Embeddings) technology has the large model first generate a hypothetical answer, then uses that answer to search for real documents, which can significantly improve recall in certain scenarios — because the vector representation of a hypothetical answer is often closer to the vector space region where real answer documents reside than the original question.

With this approach, the large model has "source material" to reference when answering, rather than generating responses from nothing, effectively addressing the large model's "hallucination" problem. When a user sends a cancellation-related message, the system retrieves content matching "cancel booking," and the large model combines the retrieved results with conversation context to deliver accurate answers aligned with enterprise business rules — essentially giving the large model "an external private knowledge base."

MCP: Connecting External Third-Party Services

The tutorial also implements MCP (Model Context Protocol). MCP is a standardized protocol open-sourced by Anthropic in November 2024, designed to solve the pain point of fragmented integration between AI applications and external tools/data sources.

The best analogy for understanding MCP comes from software engineering's LSP (Language Server Protocol) — LSP is what allows editors like VS Code and IntelliJ to uniformly integrate intelligent suggestions, go-to-definition, refactoring, and other services for hundreds of programming languages, without each editor implementing separate integration logic for each language. MCP brings the same approach to AI tool integration: any tool service following the MCP protocol (MCP Server) can be directly called by any AI application supporting MCP (MCP Client), achieving "develop once, integrate everywhere."

At the technical specification level, MCP uses JSON-RPC 2.0 as its communication foundation and defines three core primitives: Resources (reading resources such as file contents and database records), Tools (executing actions like performing searches and sending requests), and Prompts (prompt templates for AI applications to reuse preset prompt fragments). MCP Servers can run as local processes (communicating via stdio) or as remote HTTP/SSE services — the former is suitable for local development tools (like file system access in Cursor), while the latter is suitable for cloud-deployed third-party services.

Before MCP existed, every AI application had to write custom integration code for each external service, resulting in extremely high maintenance costs. In early 2025, OpenAI announced native MCP support in its API and ChatGPT, marking the protocol's evolution from Anthropic's private ecosystem to a de facto industry standard. Hundreds of open-source MCP Servers now cover common scenarios including weather, maps, code execution, and database queries, with mainstream AI products like Claude, Cursor, and Windsurf all supporting the protocol natively. For AI application developers, MCP's strategic significance lies in the fact that tool capabilities can be developed, distributed, and reused independently of models and application frameworks, forming an emerging tool ecosystem marketplace similar to npm/PyPI — developers simply publish an MCP Server, and their tools can be directly called by all MCP-compatible AI applications, dramatically lowering the barrier to building the tool ecosystem.

In the demo, a user queries the real-time weather at the destination of a specific booking, and the system successfully returns weather information for Chengdu. This is achieved by calling an external third-party MCP service, extending the intelligent customer service's capabilities from internal systems to external services and greatly expanding the boundaries of what the AI application can do.

Conclusion: The AI Breakthrough Path for Java Developers

The value of this hands-on tutorial isn't in teaching you to train models from scratch. It's in demonstrating a pragmatic technical transition path: using Java as the foundation, and leveraging core technologies like System Prompts, Tools/Function Calling, RAG, and MCP to deploy mature large model capabilities into real business systems.

For developers with several years of Java experience who want to catch the AI wave, rather than agonizing over whether to switch to Python and compete for algorithm positions, it makes more sense to solidly master the AI application development skill stack — the job demand is greater, the entry barrier is more approachable, and it's more aligned with real enterprise deployment scenarios. The essence of this path is: you don't need to become a builder of large models, but rather an integrator and amplifier of large model capabilities — using engineering approaches to make already powerful models truly serve the creation of business value.