13 Search APIs Tested: The Hidden Costs You're Ignoring May Exceed the Listed Price

The Real Bill for Search APIs Is Far More Than the Listed Price

When building AI Agents or RAG (Retrieval-Augmented Generation) systems, developers often focus solely on the "per-request price" listed by search APIs. However, one developer's hands-on testing reveals a widely overlooked truth: The real cost of a search API is half-hidden where you can't see it.

RAG (Retrieval-Augmented Generation) is one of the most mainstream architecture patterns in AI applications today. Its core idea is to have the LLM retrieve from external knowledge bases before generating answers, thereby reducing "hallucinations" and providing verifiable responses. In this architecture, LLM billing is measured in tokens—tokens are the smallest processing units after text is segmented. In English, each word corresponds to roughly 1-1.5 tokens, while in Chinese, each character is about 1-2 tokens. Taking GPT-4o as an example, input tokens cost $2.5 per million and output tokens cost $10 per million. This means that if a search API returns a bloated payload containing 5,000 tokens, the cost of merely "reading" that content already exceeds the per-request fee of many search APIs. In real-world Agent operations, a single task might trigger dozens of search calls, and token costs scale up rapidly through a multiplier effect.

This developer (who is also the builder of SERPdive, which accounts for 3 of the 13 configurations tested) conducted a systematic benchmark of 13 pricing configurations. His core insight cuts straight to the point: every search provider publishes one price—the cost per request. But after the request returns, it sends a payload to your Agent, and your LLM charges you again when it "reads" that content.

"The second cost never appears on the pricing page, and it's usually the bigger one."

Why Search APIs Have a "Second Cost"

The key to understanding this issue lies in how search APIs work within the AI pipeline. An AI Agent is a system capable of autonomously planning, invoking tools, and iteratively executing tasks. Unlike traditional single-turn conversations, Agents often need to call external tools multiple times (such as search engines, databases, code executors) to complete complex tasks. In this multi-step architecture, every tool invocation incurs a cost, and the results returned by tools are injected into the model's context window for processing. The context window is the upper limit of all text a model can "see" at once—current mainstream models have context windows ranging from 128K to 200K tokens. When search results occupy a large portion of the context space, they not only generate direct token fees but may also crowd out other useful information, indirectly impacting answer quality.

A typical Agent retrieval workflow has two steps:

1. Search call: The Agent makes a request to the search API, and the provider charges per request;
2. Reading comprehension: The raw content returned by the search (web summaries, body text snippets, etc.) is stuffed into the LLM's context, and the model consumes tokens to process this content and generate an answer.

The problem lies in step two. Different search providers return vastly different payload sizes—some are lean and concise, while others are stuffed with redundant text. All of this content passes through the LLM's token billing. In other words, an API with a cheap listed price but bloated return content may end up costing you more than an API with a higher listed price but lean returns.

The Technical Root Cause of Payload Size Differences

The massive variation in payload sizes across different search APIs stems from each provider's data processing strategy. Some providers (such as traditional SERP APIs) return large amounts of raw webpage content, including residual HTML tags, sidebar text, ad copy, breadcrumb navigation, footer information, and other "noise." Others perform content extraction and denoising on the server side, returning only core paragraphs highly relevant to the query. This difference can be an order of magnitude—for the same query, one API might return 800 tokens of refined content, while another returns 8,000 tokens of unprocessed raw text. From an information theory perspective, the actual information density (signal-to-noise ratio) is the key metric for evaluating search API quality.

Testing Methodology: Minimizing Variables

To ensure fair and credible comparisons, this benchmark was designed with considerable rigor:

• Uniform question set: The same 100 questions were tested;
• Single search call: Only one search was triggered per question;
• Same reading model: All configurations used the same answering model to process returned content;
• Verbatim pass-through: Payloads were passed word-for-word to the reading model, with only each vendor's built-in synthesis features removed;
• Real token billing: Token counts came from real bills of the answering call, not estimates;
• Uniform pricing basis: All prices used list pay-as-you-go rates, including the author's own product.

The final results were sorted by "actual total cost per thousand queries." The value of this methodology is that it combines "search cost" and "reading cost" into a single calculation, reflecting the actual bill developers will receive.

How to Properly Interpret the Test Results

The author specifically reminded readers to "interpret the data with error margins in mind," reflecting a commendable level of rigor:

• Correctness column: With a sample size of n=100, the margin of error is approximately ±10 percentage points. This is based on binomial distribution confidence interval calculations. In statistics, when measuring a proportion (such as accuracy), the sample size n determines estimation precision. For accuracy rates near 70%-80%, at a 95% confidence level, n=100 corresponds to a margin of error of roughly ±8-10 percentage points. This means that if two APIs show accuracy rates of 78% and 73% respectively, we cannot conclude which is truly better—they are statistically indistinguishable. The top configurations on the leaderboard were not meaningfully separated in accuracy by this test.
• Cost column: The author explicitly stated this column is "not noisy," meaning the cost data is stable and reliable. This is because token counts and API billing are deterministic values unaffected by sampling fluctuation. This distinction in the reliability of different metrics is a hallmark of high-quality technical evaluation.

This distinction is crucial: When accuracy is a toss-up, cost becomes the decisive factor in selection. In other words, if several APIs tie in answer quality, whichever has the lowest total cost is the more rational choice.

Practical Implications for Developers

While this test carries the author's product bias (conflict of interest is clearly disclosed, and all payloads and logs are publicly available in a repository for verification), the methodological issues it reveals have universal value.

Evaluate Search APIs on "Full-Pipeline Cost"

The traditional comparison method of only looking at per-request unit price has become seriously misleading in AI Agent scenarios. What should truly be measured is: How much does the entire pipeline cost, from initiating the search to producing the final answer? A "cheap" API with bloated return content might become the "expensive" choice by blowing up your model's token bill.

Payload Size Is a Hidden Cost Lever

The conciseness of search return content directly determines downstream model costs. This suggests developers should consider during selection:

• Whether the provider offers content trimming and deduplication capabilities;
• Whether results contain large amounts of irrelevant HTML, navigation text, and other "noise";
• Whether the API allows controlling the granularity and length of returned content.

Beware of Vendors' Built-in Synthesis Features

Interestingly, this test deliberately "removed each vendor's built-in synthesis features." Synthesis (summarization) functionality is an important trend in the search API space in recent years. Represented by next-generation search services like Perplexity API, Exa, and Tavily, these services not only return retrieval results but also use built-in LLMs to summarize and reorganize results, directly providing a "pre-digested" answer. While this seems convenient, it introduces multiple layers of issues: first, the additional cost is opaque—the model invocation fees for synthesis are typically bundled into the request price, making it impossible for developers to account for them separately; second, controllability is reduced—you cannot choose the model, prompt strategy, or truncation logic used for synthesis; finally, debugging becomes difficult—when the final answer is wrong, it's hard to determine whether the issue lies in the retrieval step or the synthesis step.

For teams pursuing cost transparency and pipeline controllability, taking raw retrieval results and processing them uniformly with your own model actually makes it easier to track costs and locate and fix problems.

Conclusion: Making the Opaque Transparent

The greatest value of this benchmark may not be which API ranked where, but rather that it put the long-overlooked (whether deliberately or unintentionally by the industry) issue of "the second cost of search APIs" on the table. The author's practice of publishing all payloads and logs also sets a reproducible, verifiable standard for this type of evaluation.

For teams building Agents, RAG, or any retrieval-driven AI application, the next time you select a search API, ask yourself: Is the price I see the entire bill? The answer is very likely no. The truly rational approach is to do what this test did—run a full-pipeline cost analysis for your own use case and let real token bills speak for themselves.

Full test data and logs are open-sourced: github.com/edendalexis/search-api-cost-benchmark