Silver Price Returns CPI: The Fatal Trap of MCP Tool Word Matching

AI Agents rarely pass precise IDs when querying data sources — they pass words: "10 year treasury yield," "core inflation," "silver price." So most tools that wrap data sources include a small resolver to convert words into IDs. A bug in that resolver may never throw an error — it just silently returns the wrong answer.

The author of the MCP tool canli-markets-mcp shared a particularly instructive engineering lesson: when a user asked for "silver price," the tool returned data for the US CPI (Consumer Price Index). Behind this seemingly absurd result lies a trap that any "natural language to ID" tool can fall into.

How the Bug Was Born

In canli-markets-mcp version 0.1.0, the economic_series function accepted both FRED series IDs and natural language. Its matching logic scored each of 47 common series based on "the proportion of words in the request that appear in that series' description," returning the top result if it scored at least 50%.

The problem was in how that threshold was calculated. A two-word request only needed one word to match to hit 50%. The CPI description is "cpi consumer price index inflation all items," so the single word "price" in "silver price" was enough to trigger a match — and silver got resolved to US CPI.

The author's comparison table is striking:

Request	0.1.0 Result	Now
silver price	CPIAUCSL (US CPI)	Rejected, directed to price_history
copper price	CPIAUCSL	Rejected
bitcoin price	CPIAUCSL	Rejected
japan inflation	CPIAUCSL	Rejected, directed to world_economy
uk unemployment rate	UNRATE (US unemployment)	Rejected
wage growth	Real GDP growth	Average hourly earnings

Every wrong answer returned a real series with real numbers. The series title was right there in the result — the error was in plain sight — but nothing forced the Agent to read it.

A clear pattern emerges: the words that determine the essence of a query ("silver," "japan," "uk") are exactly the ones that matched nothing; the words that did match ("price," "inflation," "rate") only indicate what kind of number you want, not which one.

---

FRED (Federal Reserve Economic Data) is a public economic database maintained by the Federal Reserve Bank of St. Louis, containing over 800,000 economic time series from hundreds of sources worldwide. Each series has a unique alphanumeric ID — for example, CPIAUCSL stands for "All Urban Consumers: All Items in U.S. City Average," and UNRATE represents the US unemployment rate. These IDs are machine-precise but completely unintuitive for humans — which is exactly why a natural language resolution layer is needed, and why errors in that layer are so hard to detect. In an AI Agent context, the Agent receives a JSON response with a series title and real numbers. If it doesn't actively verify that the title matches the question, the error silently flows into downstream reasoning.

---

The New Rules That Replace It

The fix centers on redefining "which words actually matter." A single unified function now serves three resolvers: FRED series, World Bank indicators, and OECD outlook indicators. Notably, both newer resolvers had already made the same mistake before release — "wage growth" returned GDP growth, "house prices" returned CPI, and OECD's "exchange rate" even returned unemployment.

The new rules classify every word in a request into one of three categories:

• Filler words: Ignored entirely — things like "what is," "show me," "since," "this year," and year numbers (e.g., 2020).
• Kind words: Indicate the type of number — rate, growth, index, level, daily, monthly. They contribute to best-match scoring but are never required.
• Specific words: Everything else — and every specific word must appear in the candidate entry.

This hard constraint — all specific words must match — is the key to the fix. The matchWords function first filters out filler words, then strips kind words to isolate specific words; if a candidate entry doesn't contain all specific words, it's skipped. Among qualifying entries, the best match is chosen by the number of shared words. The split function lowercases input, retains alphanumerics and &, and applies singular/plural folding ("wages" becomes "wage"). Ties are broken by list order, so more common series rank first.

Each resolver can also customize its own word lists. Since FRED series are all US data, "us" and "fred" are treated as filler words there, while "interest" is treated as a kind word.

When no entry qualifies, the tool explicitly says "not found" and points to what to use instead:

No common series matches "silver price". These are US series: for another country's economy use world_economy or economic_outlook, and for the price of a commodity, currency, crypto or stock use price_history. Or pass a FRED series id.

This distinction is crucial for Agents. An Agent that receives this rejection can proceed to call price_history; an Agent that receives CPI data has no clue it should do that.

Test Rejection First

The old tests all checked that "the right request finds the right series" — and they all passed. But not a single test asked: when the requested data doesn't exist, does the tool reject it?

The new test file starts by filling exactly this gap — asserting that requests like "silver price," "bitcoin price," "japan inflation," and "uk unemployment rate" must throw NotFound, with error messages that include redirections like price_history and world_economy. Only after that come 42 positive tests that must resolve to a single precise series, followed by equivalent tests for the World Bank and OECD resolvers.

One interesting design decision: "interest rates" is deliberately rejected. Because there are so many kinds of interest rates, forcing a single result is just the same bug with a nicer face.

---

This touches a classic blind spot in software testing: coverage measures code paths, not semantic correctness. A test suite containing only positive cases is essentially asking "can the system give correct answers to inputs it recognizes" — not "can the system recognize inputs it doesn't know." In traditional CRUD applications, the cost of this blind spot is relatively manageable — users see wrong results and report them. But in Agent pipelines, tool outputs are intermediate states that never pass through human eyes; a wrong answer may have already participated in several reasoning steps before being used to generate a final report. This makes "rejection capability" a first-order property of Agent tool reliability, not just defensive programming.

---

Reusable Engineering Lessons

The three takeaways the author summarizes apply to any developer building Agent tools or natural language retrieval:

• A word-to-ID resolver is essentially a search engine that returns exactly one result. It must be capable of returning zero results. A matcher without a rejection mechanism forces every input into some answer.
• Scoring should be based on words that distinguish entries, not on the proportion of words that match. Generic words are easiest to match but least meaningful — that's exactly why "price" matched CPI.
• Write rejection tests before positive tests. Positive tests pass on day one; the tests that actually expose problems are the ones for scenarios where you have no data but the system forces an answer anyway.

Open Source and Ready to Try

canli-mcp is free, MIT-licensed, runs locally, and requires no API key for FRED, World Bank, or OECD public data:

npx -y canli-mcp

# Claude Code
claude mcp add canli -- npx -y canli-mcp

The code is hosted on GitHub at arhancanli/canlicapital (includes each MCP server plus words.mjs and test files) and arhancanli/alphac (strategy research engine). The author also welcomes issues reporting parsing errors — those are exactly the bugs he wants to hear about.

For developers building Agent tools, the value of this case extends far beyond a single specific bug: it reminds us that a tool that silently returns wrong answers is far more dangerous than one that throws an error.