LangChain4j No AI Agent in Practice: Agent Architecture Without Accessing LLMs

From Pure Agents to Workflows: An Unavoidable Performance Problem

When building AI Agent applications, whether using a pure autonomous agent or workflow orchestration approach, there's a common, often overlooked pain point: high-frequency access to large language models.

An Autonomous Agent is an AI system with independent decision-making capabilities that can plan steps, select tools, and execute tasks on its own based on objectives. The entire process is driven by an LLM-powered decision chain, with typical examples being AutoGPT, BabyAGI, etc. Workflow orchestration, on the other hand, is a structured process with pre-defined execution paths and conditional branches, where developers explicitly control the execution order of each step. The core difference lies in who holds the decision-making power: the former delegates it to the LLM, while the latter keeps it in the developer's hands. In practice, autonomous agents offer more flexibility but less controllability, while workflow orchestration provides strong controllability but lacks adaptive flexibility. The industry has therefore been gravitating toward hybrid architectures—yet both patterns face the common challenge of high-frequency LLM calls.

Take a typical book borrowing scenario as an example: a user inputs "Zhang San borrows a copy of One Hundred Years of Solitude," and the system needs to complete a series of operations. Let's break down exactly how many times the LLM is accessed throughout this process:

• Parsing the user message: Extracting "book title" and "reader name" from natural language — 1 LLM call;
• Dispatching sub-agents: Book agent, reader agent, notification agent, recorder agent — each needs to access the LLM once, totaling 4 calls;
• Tool call decisions: The LLM decides which tool to invoke, and after tool execution, the LLM needs to interpret the result — 1 to 2 additional calls;
• Iterative calls: If multiple rounds of tool calls are involved, the count multiplies further.

It's worth explaining the underlying mechanism of Function Calling here. Function Calling is one of the core capabilities of modern LLMs, first introduced by OpenAI in June 2023. It works as follows: developers describe available tools' names, parameters, and functionality to the LLM; during inference, the LLM determines whether a tool call is needed; if so, it returns a structured call request (containing the function name and parameter JSON); the application layer executes the actual call and passes the result back to the LLM for the next reasoning step. This process requires at least two LLM calls—one to decide which tool to call, and another to interpret the tool's return value. If chain-calling multiple tools is involved, the count grows proportionally, making this the core source of the performance bottleneck.

All told, a single simple business operation can require nearly ten or even more than ten LLM calls.

The Triple Cost of High-Frequency LLM Calls

This design introduces three problems that cannot be ignored in production environments.

High Cost

Every LLM access incurs charges. LLM API billing is typically based on token count, where a token is the smallest unit after text tokenization—Chinese text averages about 1.5-2 characters per token. Taking GPT-4o as an example, input tokens cost approximately $2.5/million tokens, and output costs about $10/million tokens. When a simple workflow requires over a dozen calls, each including system prompts, historical context, tool descriptions, and other verbose input tokens, plus inference output tokens, the total token consumption per user request can reach thousands or even tens of thousands. In high-concurrency scenarios (e.g., hundreds of requests per second), monthly API costs can reach tens of thousands of dollars, directly driving up operational costs. For high-concurrency online services, this is nearly unbearable.

Slow Response

LLM inference requires waiting, and this synchronous call pattern does not support streaming output. Streaming output means the LLM returns partial results incrementally as it generates a response, without waiting for the complete content to be generated first—users see text appearing character by character, similar to ChatGPT's typewriter effect. This mechanism relies on Server-Sent Events (SSE) or WebSocket protocols. The reason it doesn't work here is that multi-step Agent calls don't return reactive types that Spring WebFlux can understand. Spring WebFlux is Spring's reactive programming module, implementing non-blocking IO based on Project Reactor, with core types being Mono (single value) and Flux (multi-value stream). To achieve streaming output, the LLM client needs to return a Flux<String> response stream, but in multi-step Agent call scenarios, synchronous waiting at intermediate steps breaks the streaming pipeline, forcing users to wait for all steps to complete before seeing the final result. If you want to implement streaming, you'd need to manually write imperative entry points and switch the chat model to a streaming chat model—a costly refactoring.

Fragmented Experience

Users have to wait at each step, and the accumulated latency severely degrades the interaction experience. This "wait—return—wait again" pattern is clearly intolerable.

Breaking Through: Agents That Work Without Accessing LLMs

Facing these problems, an elegant solution emerges: if an Agent's task doesn't truly require natural language understanding, why access an LLM at all?

The LangChain4j official documentation has a dedicated section that addresses exactly this need—No AI Agent. LangChain4j is the official implementation of the LangChain ecosystem on the Java/JVM platform, aiming to provide Java developers with AI application development capabilities equivalent to the Python version of LangChain. It offers unified LLM interface abstractions (supporting OpenAI, Anthropic, local models, etc.), Function Calling, RAG (Retrieval-Augmented Generation), memory management, Agent orchestration, and other core capabilities. Its design philosophy emphasizes deep integration with the Spring Boot ecosystem, lowering the development barrier through annotation-driven approaches. No AI Agent is a feature introduced in version 0.35+, allowing developers to create agent nodes that don't depend on LLMs, reflecting the framework's pragmatic consideration of engineering practices.

According to the official documentation: up until now, all agents have been AI agents, meaning they all rely on large language models to perform tasks requiring natural language understanding. However, LangChain4j also supports non-AI agents that can be used to perform tasks that "don't require natural language."

In other words, it's still an Agent, but it doesn't access an LLM, doesn't need to understand natural language, and can directly complete tool operations.

Implementing LangChain4j No AI Agent

Since there's no LLM access, the entire set of configurations originally built around the LLM can be simplified:

• No system prompts (System Message) needed;
• No ChatModel needed;
• No Tools collection needed.

Core Refactoring: Inlining Tool Methods as Plain Java Methods

The key refactoring is: taking the tool methods that were originally invoked by LLM decisions and inlining them as plain methods within the Agent class.

Tool execution was originally triggered by LLM decisions. Now that we're not accessing the LLM, we move the logic from Tools into the Agent class, making it a plain Java method that executes directly.

For example, in the book agent, you inject readerMap, wire dependencies via @Autowired, and have methods return business objects directly—this way the original readerTool can be completely replaced, eliminating the need for a separate tool definition.

Simplified Agent Creation and Injection

When creating this type of non-AI Agent, there's no need to pass in a chatModel or inject tools. The entire Agent becomes a pure Spring component that can be used directly via @Autowired injection, just like calling a regular Service.

@Autowired is one of Spring framework's core annotations, used to implement Dependency Injection (DI). DI is a concrete implementation of the Inversion of Control (IoC) design pattern, whose core idea is to delegate object creation and dependency relationship management to a container (Spring IoC Container) rather than having developers manage it manually. When a class is annotated with @Component, @Service, or @Bean, the Spring container automatically creates its instance and manages its lifecycle. Designing Non-AI Agents as plain Spring components means they can leverage the full power of the Spring ecosystem—such as transaction management, AOP aspects, lifecycle callbacks—while maintaining a programming model consistent with other business components, reducing the team's cognitive overhead.

In the code, bookAgent, readerAgent, notificationAgent, recorderAgent, and other components are all wired via @Autowired, with no need to manually new objects. Because these Agents neither access LLMs nor call tools, they complete all their work internally.

Test Results: Significantly Improved Response Speed

After completing the refactoring, real-world testing was conducted.

After running clean, compiling, and launching, you can observe that LLM access frequency drops dramatically—theoretically only one or two calls at necessary points (such as final result polishing), while intermediate business logic is handled entirely by local methods.

From actual runtime results, response speed is noticeably faster—responses return almost immediately after requests are sent, with wait times far shorter than the previous pure agent approach.

For business validation, an out-of-stock scenario was tested: when a particular book has no inventory, the system correctly detects "unable to borrow," and neither the reader status nor inventory data in the database undergoes incorrect changes—logic judgment is accurate. This demonstrates that even after removing the LLM, business correctness is still guaranteed.

Summary: A Hybrid Agent Architecture That Uses LLMs On Demand

LangChain4j's No AI Agent provides a pragmatic engineering perspective: not all agent nodes require natural language understanding capabilities.

For tasks with deterministic logic that don't need semantic parsing (such as inventory checks, status updates, data recording), plain methods can completely replace LLM calls. This "use LLMs on demand" hybrid architecture can significantly reduce costs and latency while maintaining complete functionality.

In actual Agent system design, identifying which nodes truly need an LLM and which can degrade to deterministic code is an important architectural decision. This decision is essentially a trade-off between "intelligence" and "determinism": LLMs excel at handling ambiguous input, semantic understanding, and open-ended reasoning, but for operations with clear inputs/outputs and fixed logic, deterministic code is not only faster and cheaper but also more reliable and testable. Properly leveraging non-AI agents enables the entire system to achieve a better balance between intelligence and efficiency.

Key Takeaways