I Questioned the AI Agent, and It Changed My System Design

In traditional development workflows, architectural decisions are usually made by humans. But when I actually brought an AI Agent into the design process, a simple follow-up question ended up shifting the entire approach. This isn't science fiction — it's something that really happened while I was using the built-in Agent Spaceship.

A design change triggered by a single challenge

While building a multi-model routing system, I originally planned to follow the existing Index Router pattern — mounting encoders from different vendors. This pattern had already been running smoothly within the team, so reusing it seemed like the natural choice.

But looking at the structure diagram on screen, I hesitated: since encoders can be mounted this way, why not do the same with Decision Models? With that question in mind, I directly raised the challenge in the built-in Agent Spaceship.

What I typed was straightforward: "Is this the correct option? In the index router, we mount encoders from different vendors — why not do the same for decision models?"

Index Router and encoder mounting: In multi-model systems, an index router is a pattern that distributes requests to different model endpoints according to preset rules. The phrase "mounting encoders from different vendors" refers to registering each vendor's embedding models (encoders) as pluggable components under a unified router, with the routing layer selecting which one to call based on index type or task characteristics. The advantage of this pattern is that it decouples the caller from the specific vendor — adding a new encoder only requires registration, with no changes to business code, which is why teams easily develop path dependency. But encoders and decision models are not equal in responsibility: encoders are typically stateless vectorization services, whereas decision models often involve conversation context, tool calls, and cost trade-offs. Directly copying the mounting approach may obscure the scheduling responsibilities the routing layer needs to bear.

Structural uncertainty: weighing two paths

Frankly, I didn't have a standard answer when I asked the question. I admitted my dilemma to the Agent: I could follow the encoder route and configure a separate decision route for each decision model; or I could take another path and mount decision models uniformly under the existing routing system.

"Honestly, for this structure, I'm not quite sure which is the correct answer." This candor actually made the conversation more valuable — the Agent didn't cater to my assumptions, but instead began analyzing the differences between the two approaches in scalability and maintenance cost.

Decision models and an independent decision routing layer: Decision models refer to the models in an Agent workflow responsible for planning, selecting tools, or generating final judgments. Unlike encoders that only perform vectorization, they typically require multi-turn interaction and state management. Introducing an independent decision routing layer for decision models means adding a layer beyond the original index router that specifically handles "which decision model to select and with what strategy to invoke it." This way, decision models from different sources can be registered and connected like plugins, while the routing layer centrally handles fallback, retries, and cost/latency trade-offs. The cost is that the system gains an extra layer of abstraction, and the boundary between the two routing layers must be clearly defined to avoid overlapping responsibilities that make debugging difficult.

The Agent's response and the design pivot

Based on the Agent's feedback, we ultimately adjusted the design: rather than simply copying the encoder mounting approach, we introduced an independent decision routing layer for decision models. This change allows decision models from different sources to be connected like plugins while preserving the flexibility of the routing logic.

This decision was recorded in the project documentation. Looking back, if I hadn't raised that question, the system would likely have followed inertia down a path that wasn't optimal. The value of an Agent isn't in making decisions for you — it's in helping you lay out your thinking process when you're uncertain, so that the truly reasonable approach can emerge.

Lessons for AI-assisted programming collaboration

This experience gave me a new understanding of what it means for an AI Agent to be a collaborator. It isn't a tool that merely executes instructions — it's a conversational partner that can be questioned and will respond to challenges. When you honestly lay out your uncertainty, an Agent can often provide structured analysis that exceeds your expectations.

For developers currently using AI programming assistants, my advice is: don't treat it only as a code generator. During the architecture design phase, proactively asking it "why not do it this way?" may lead to unexpected design optimizations.