LLM Tool Calls Failing? Deleting Half Your Tools Beats Writing a Massive Prompt

When an AI Agent misbehaves on tool calls, the first instinct for many teams is to add more tools and write more detailed prompts. But one developer's experience points to the opposite conclusion: delete half the tools, tighten the schema definitions, and the Agent suddenly gets "smarter." This counterintuitive fix reveals a widely overlooked issue in LLM tool-calling design—semantic collision in the tool interface.

Root Cause: Overlapping Tools Create Semantic Collisions

The developer's Agent was wired into Slack, Notion, HubSpot, and a few internal APIs—a typical automation workflow stack. But GPT-5.4 kept calling the wrong functions, and switching to Claude Opus didn't fix it either, just made it "a bit better behaved."

His initial fix was the classic mistake: add rules, add exceptions, add examples, add counterexamples, write more prompt text explaining when to use which tool. The result was an ever-longer prompt with no improvement in behavior. That itself was the key clue: the prompt was no longer the bottleneck—the tool interface was.

The essence of the problem is semantic collision. When you hand the model find_contact_hubspot, get_account_notes_notion, search_customer_records_internal, and search_slack_customer_thread, a human can tell them apart, but the model sees four partially overlapping "stories" and must semantically rank them before it can even begin solving the user's actual request. Tool-call reliability starts collapsing at exactly this step.

According to OpenAI's function-calling guidance, the recommended approach now is to use tool_search to lazily load infrequently used tools rather than stuffing every tool into a single request. Anthropic's tool-use docs also state clearly that Claude selects tools based on the user request and tool descriptions, recommending good descriptions and input_examples. On the surface this looks like a prompt-writing problem, but in substance it's mostly a boundary-design problem. If two tool descriptions overlap, the model can only guess which one you want.

Semantic collision can be understood as: the descriptions of multiple tools are close to each other in embedding space, making it hard for the model to reliably distinguish them from description alone. When an LLM decides which tool to call, it is essentially conditionally ranking candidate tools given the conversation context. When tool descriptions share a lot of vocabulary (like customer, account, search), their probability distributions overlap heavily, and the model has to rely on subtle differences or randomness to decide. This is similar to "query-document ambiguity" in information retrieval: the vaguer the query and the more similar the documents, the less stable the ranking. The engineering response is to proactively widen the semantic distance between tools rather than expecting the model to learn to tell them apart.

The More Tools, the Faster the Agent Gets Dumber

Once the tool count crosses a certain threshold, the model has to do a round of semantic ranking before it can even start the task. This is expensive in tokens, fragile in behavior, and hard to patch with prompt text.

OpenAI's newer Agents building blocks formalize this further, but most teams learn the lesson in production "the annoying way": runtime tool selection beats loading everything up front. The useful question is no longer "how do I make the prompt smarter" but "how do I shrink the decision surface."

The original article compares three approaches:

Approach	Actual Behavior
OpenAI function tools	JSON Schema defines functions, strict: true ensures schema compliance, tool_search lazily loads rarely used tools
Anthropic client tools	name, description, input_schema, optional input_examples, defer_loading, and disable_parallel_tool_use
Pure prompt disambiguation	No hard constraints, instructions only, degrades as tool count and overlap increase

The author's verdict is blunt: pure prompt disambiguation is fine for demos, not for real automation. If your n8n, Make, Zapier, or custom Agent flows touch Slack, HubSpot, and Notion in production, you need contracts, not vibes.

Deferred loading means not exposing all tool definitions in the first request, but instead letting the model first retrieve or declare the capabilities it needs through a meta-tool (like OpenAI's tool_search), after which the runtime dynamically injects the relevant tool schemas. This has two benefits: first, it reduces the number of candidate tools in a single request, directly shrinking the decision surface; second, it saves tokens, since tool definitions themselves consume a considerable amount of context. Anthropic's defer_loading pattern is similar in spirit but controls exposure timing through tool registration and loading strategy. Deferred loading isn't a silver bullet—it partly turns "choosing a tool" into "retrieving a tool," which still requires tool descriptions to be well retrievable.

Small Schemas Beat Clever Prompts

The boring weather-tool examples in the docs are actually great models. Why do they work? Because they're narrow. One required field, maybe an enum, clear intent, almost no room for improvisation.

Take the OpenAI-style weather tool: get_weather needs only a location parameter described as "city and state, e.g. San Francisco, CA", plus strict: true. The Anthropic style adds a unit enum field (celsius or fahrenheit), optional but clearly bounded.

That's the whole trick in miniature: narrow tools are easier to choose and harder to misuse.

The Five-Step Fix in Practice

The author lays out a reusable fix order, by priority:

1. Remove Overlapping Tools from the Same Turn

If multiple tools can answer essentially the same request, don't expose them together. Route first, then expose only the relevant subset. Use OpenAI's tool_search or an equivalent runtime-loading pattern; with Anthropic, rely on the defer_loading pattern and stricter tool registration.

2. Rename Tools So They Sound Mutually Exclusive

Tool names aren't decoration—they're part of the interface. Names like customer_lookup_v2, record_search, and account_tool offer zero differentiation. Better options are get_customer_billing_status, read_customer_support_notes, find_recent_slack_thread_for_customer—names that create separation.

3. Shorten Descriptions by Adding Contrast

Long descriptions are only useful when they define boundaries. A description like "retrieve customer info, account history, support context, messages, records, and related updates" is manufacturing confusion. A better version: "Use only to fetch billing status from HubSpot. Do not use for support notes, Slack history, or internal CRM records." The latter gives the model a cleaner decision.

4. Constrain Parameters Strictly

Use enums, use required fields sparingly, and avoid huge free-form objects. A schema with three free-form object fields—customer, options, mode—is a disaster; a schema with customer_id plus a source enum (hubspot or internal_crm) is the right answer.

5. Disable Parallel Tool Calls for Side-Effect Scenarios

If a workflow can send a Slack message, update HubSpot, and write to Notion, parallel tool calls aren't always a feature—sometimes they just create cleanup work faster. The Anthropic-style control is disable_parallel_tool_use: true. The author considers this setting underrated—one clean action always beats three half-right ones.

Parallel tool calls let the model fire multiple function calls in a single response turn, reducing round-trip latency. But in side-effect scenarios, parallelism introduces race conditions and consistency problems: for example, updating HubSpot and writing to Notion at the same time—if one fails, the system is left in a partially completed state requiring compensation logic. A subtler risk is the model making simultaneous decisions based on incomplete information, causing actions to conflict with each other. Disabling parallel calls effectively forces serialization, making each step's result the input to the next, trading throughput for predictability. For parallelizable read-only queries, parallelism can still be kept to optimize latency.

A Hidden Failure Mode: Better Descriptions Can Make Selection Worse

This is where many teams trip up. You improve each tool's description, add more detail and context, and feel good about it. Then tool selection is still terrible.

The reason: you made each description richer within the same vocabulary system. Now all five tools mention customer, account, support, history, messages, updates. The descriptions are better written, but the semantics are also more similar, which makes selection harder rather than easier.

The key to the fix is contrast. A good tool description should state four things: what the tool does, when to use it, when not to use it, and which neighboring tool to use instead. That last point matters more than most people realize.

Prompts Still Matter—Just Not for This

This isn't an argument against prompts; it's an argument against using prompts to paper over sloppy tool design. The author's current division of labor: prompts handle policy—things like "no refunds without manager approval," "don't contact customers after 9 PM," "confirm before deleting a Notion page"; schemas and tool boundaries handle behavior—things like enum values, required fields, mutually exclusive tools, and only one tool allowed per turn for high-risk actions.

Confuse the two and you'll write a 900-word prompt to solve a problem a 9-line schema could fix.

The Cost in Production

A three-tool demo can survive on vibes; real workflows can't. When a workflow touches Slack, Notion, HubSpot, Discord, internal APIs, and customer data, tool ambiguity becomes an operational problem. And if you pay per token, the cycles the model burns deciding among five overlapping functions get expensive fast.

The original author notes that for scenarios involving continuous Agent behavior iteration, tool-schema testing, and running automations, Standard Compute's flat-rate API access model is far easier than worrying about token consumption on every debug run. If your stack is already OpenAI API-compatible, the switching cost is low.

Key Takeaways

When LLM tool calls start failing, most people's reaction is to add: more prompts, more examples, more fallback logic, more tools. But the highest-leverage fix is usually subtraction: fewer active tools, fewer overlapping descriptions, fewer optional fields, fewer parallel side-effect opportunities.

If you find your Agent keeps picking the wrong tool, before writing another 500 tokens of prompt, try deleting half your toolbox. You may be solving an interface problem, not an intelligence problem.

One-line Summary

LLM tool-call instability is mostly about semantic overlap in the tool interface—trimming tools and tightening schemas beats writing long prompts.

Paragraph Summary

The article argues that AI Agent tool-call failures are often caused not by insufficient model intelligence but by tool-interface design. When multiple tools overlap in function, the model must first do semantic ranking, which causes selection reliability to collapse. The author significantly improved Agent performance with five fixes: removing overlapping tools, renaming to create mutual exclusivity, shortening descriptions via contrast, strictly constraining parameters, and disabling parallel calls in side-effect scenarios. The article also reveals a hidden failure mode: improving each tool's description can actually make semantics more similar and worsen confusion. The core takeaway is that prompts handle policy constraints while schemas and tool boundaries handle behavior constraints—subtraction has higher leverage than addition.