How Local LLMs Can Hold More Web Search Data: Markdown Output in Practice

Title

How Local LLMs Can Hold More Web Search Data: Markdown Output in Practice

Meta Description

SerpApi Markdown output lets local LLMs fit more web search data into the same context window: 27% fewer tokens for the same fields, and full responses shrink by 4.6x. Includes measured latency for four local models and guidance on choosing output modes.

Article Content

Web search results usable by local large language models are always constrained by the context window. The SerpApi team previously built an open-source app that gives models running in LM Studio web search capabilities, but a single search response often exceeded the entire context window, so fields had to be trimmed server-side with a JSON Restrictor. SerpApi's new Markdown output changes this budget: the same data represented as Markdown uses fewer tokens, so the restrictor can retain more content, and most searches no longer need the restrictor at all.

How Many Tokens Does Markdown Output Save?

SerpApi's Markdown output is enabled by adding the output=md parameter to a request. It returns a Markdown document with headings, lists, and tables instead of JSON with keys, quotes, and braces. It covers over 100 APIs and costs the same as JSON searches.

According to benchmarks on the official SerpApi blog, using the same restrictor strings as the previous blog post and sending the same seven search categories (Search, Finance, News, Maps, Flights, Hotels, Shopping) as both JSON and Markdown, the JSON version totaled 4,668 tokens and the Markdown version 3,412 tokens—a 27% reduction.

The savings depend on how deeply nested the data is. Flight results contain legs, and legs contain airports and times, so JSON repeats keys, quotes, and brackets at every level; Markdown turns the result list into a table where column names are written only once. Shopping results are flat—each product has only a title, price, and store—so there is almost nothing to save (only −5%). Flights show the largest savings at −47%, while Finance and News are both −38%.

This number has direct implications for local deployments: if your 32K window previously only fit three search results, with Markdown the same budget can fit four or more.

Context windows and token budgets: A context window is the upper limit on the total number of tokens a model can "see" in a single inference pass, including the system prompt, conversation history, tool return values, and the model's own generated content. In local deployments this limit is usually set by the n_ctx configured when loading the model; content beyond it is truncated rather than raising an error. A token is the basic unit after a tokenizer splits text; in English it is roughly 0.75 words, while in Chinese it is often one token per character or more. So "saving tokens" is essentially about how much useful information fits in the same fixed budget: JSON's many keys, quotes, and brackets all count toward the budget but carry no semantics, whereas Markdown replaces repeated structure with headers and separators, freeing budget for the content itself.

tiktoken and tokenizer differences: tiktoken is OpenAI's open-source tokenizer library, and cl100k_base is the encoding used in its GPT-3.5/GPT-4 era, often treated as a general cross-model token estimation baseline. But every local model has its own tokenizer (e.g., different vocabularies based on SentencePiece or BPE), and the same text can yield token counts that differ by 10%–30% across models. This means the "Markdown saves 27%" figure calculated with tiktoken is a relative ratio between formats; when transferred to a specific model the absolute numbers will change, but the directional conclusion that "Markdown uses fewer tokens than JSON" generally holds.

One Restrictor Set, Shared by Both Outputs

Markdown does not have a separate restrictor mechanism. You simply include the original json_restrictor parameter alongside output=md, and SerpApi filters the fields first, then renders the remaining content as Markdown.

In the app, each tool maintains two sets of restrictor strings: the original set for JSON and a looser set for Markdown that keeps more results and more fields. For Maps, for example, the JSON restrictor keeps 5 places, while the Markdown restrictor keeps 6 places and additionally includes opening hours and place descriptions. The app defaults to the combination of Markdown plus restrictor.

Each tool spends this "saved budget" differently: Search goes from 5 web results to 5 results but with an answer box list, 8 AI Overview sections, and 3 top stories; News goes from 5 stories with 1 related article each to 7 stories with 2 each; Flights goes from 3+3 options to 4+4, including aircraft type, legroom, overnight layovers, and price insights.

These extra fields show up in answer quality. In testing, when asked about the best barbecue in Austin, Qwen 3.8 27B listed 5 restaurants with JSON and 6 with Markdown; the newly added Good BBQ Company was one of the two highest-rated. The Markdown answer also cited place descriptions that only that mode retains, describing Terry Black's as "fourth-generation pitmaster, old-school smoked meat."

What Happens When You Skip the Restrictor

The bigger change is full responses—what the app calls Raw mode. In the previous blog post, sending raw SerpApi responses to a local model was simply not feasible: seven raw JSON responses totaled 326,949 tokens, nearly 10 times the context window. The same seven responses converted to Markdown total 71,244 tokens—4.6 times smaller—and none individually exceeds a 32K window. This is a qualitative change brought by Markdown, not just a quantitative one.

Fitting is not the same as running fast. With Markdown Raw, the model reads two to three times as many tokens as in Markdown restrictor mode, and answer latency extends by up to 2.7x.

There is also an easily overlooked pitfall: putting two full pages into one question can still overflow the context window. In testing, one model comparing AAPL and TSLA had a prompt exceeding 32K tokens, and LM Studio silently truncated part of it, causing the model to miscalculate Apple's daily change. This kind of silent truncation is more dangerous than an explicit error because users have no way to notice it.

Why silent truncation is dangerous: When a prompt exceeds n_ctx, most inference engines default to dropping tokens from the beginning or end rather than throwing an error. In retrieval-augmented scenarios, what gets dropped is often earlier search results or the system prompt, yet the model still generates a seemingly coherent answer—one based on incomplete context. There is no exception signal for this kind of failure, and users can hardly detect it from the output; they can only infer it indirectly from discrepancies between the answer and the facts. Mitigation measures include estimating token counts at the application layer and proactively paginating, summarizing each search result individually before concatenating them, and instructing the model in the prompt to cite sources so results can be checked afterward.

What the restrictor does: JSON Restrictor is a mechanism that performs field-level trimming of API responses on the server side, using a whitelist-style string description to specify which paths to keep and which fields to discard, thereby reducing size before data enters the model. Its value lies in determinism—trimming happens outside the model, consumes no inference budget, and cannot miss key fields due to "misunderstanding" by the model. Markdown mode reuses the same parameters, showing that trimming logic is decoupled from rendering format: structured data is first filtered by path, then serialized to Markdown, so the two restrictor sets can be tightened or loosened independently.

How to Choose Among the Four Outputs

The author's recommendation is clear:

• Markdown (restrictor): The default option; use it whenever the model needs to read results.
• JSON (restrictor): Use it when your code needs to read results, e.g., sorting by price.
• Markdown (Raw): Use it when the answer genuinely needs full pages and you can accept up to three times the wait. Avoid for questions involving two large searches (e.g., comparing two stocks).
• JSON (Raw): Do not send it to local models.

If your hardware has enough memory, you can raise the 32K window limit via the app's LMSTUDIO_CONTEXT_LENGTH setting.

Side-by-Side Comparison of Four Local Models

Testing was conducted on a MacBook Pro M5 Pro with 48 GB of memory, connected via LM Link to two MacBooks. All four models ran with a 32K context window and completed all questions. Below is the average latency per search question in Markdown (restrictor) mode:

• Qwen 3.8 27B: About 42 seconds/question. It is the app's default model when installed and can correctly convert follow-ups like "around there" or "which one of those" into search terms during long conversations; it is the most stable overall.
• Meta Muse Glimmer 30B: About 62 seconds/question, the slowest. Its tokenization lets the same search results occupy less of the window—a hidden advantage.
• Prism Bonsai 27B: About 24 seconds/question. LM Studio lists its size at only about 4 GB, making it outstanding value.
• Gemma 4 12B QAT: About 16 seconds/question, the fastest; about 43 seconds after switching to Markdown Raw. But it has the most issues in long conversations, such as giving prices that do not exist in the search results.

The author also cautions that machines with less than 48 GB may load smaller windows and therefore fit less search data; and new open-source models appear almost weekly, so the optimal choice changes quickly. The staff picks in LM Studio's model search match your hardware and are a reasonable starting point.

When Search Results Conflict with Model Knowledge

All models' knowledge stops at their training cutoff. On September 1, 2026, John Ternus succeeded as Apple CEO and Tim Cook became executive chairman—something unknowable to models whose training data ends before that date.

When asked "Who is Apple's current CEO?", Qwen 3.8 27B's reasoning trace recorded its judgment: it knew Tim Cook had been CEO since 2011 but recognized this was a time-sensitive question and decided to search first to confirm. That step was correct. The harder part comes when search results conflict with its prior knowledge: in one run, the first result's AI Overview already stated John Ternus, yet Qwen did not accept it outright; it said "this is a major claim" and planned another news search to verify, lest it be a hallucination.

This reasoning is visible because the model runs locally—LM Studio writes each model's reasoning verbatim to the server log. Hosted reasoning models usually give you only summaries: OpenAI chose not to show raw chain-of-thought when releasing o1, and other labs have followed in recent years, partly out of competitive concerns about knowledge distillation—letting smaller models learn from larger models' outputs, for which complete reasoning traces are extremely valuable training material. Local deployment thus offers an extra benefit from a research perspective.

Practical Recommendations

The value of Markdown output for local LLM web search scenarios can be summed up in one operational path: add output=md to requests, keep one restrictor set per engine, and spend the tokens saved by Markdown on the fields the question actually needs. To try it, clone the serpapi/local-llm-web-search repository on GitHub and fill in an API key; free accounts include 250 searches per month.

Keep in mind that this benchmark measures two dimensions: payload size and end-to-end performance. The former is estimated with OpenAI's tiktoken (cl100k_base encoding), while each local model has its own tokenization, so these token counts should be treated as relative comparisons between formats rather than absolute values. In addition, the test sample is a combination of 7 searches and 13 questions—a small scale—so conclusions may need re-validation after changing hardware or models.

One-line Summary

SerpApi's Markdown output saves local LLMs about 27% of tokens on web searches, and most searches no longer need a restrictor.

Paragraph Summary

SerpApi added an output=md parameter for over 100 APIs, rendering the original JSON responses as Markdown with headings, lists, and tables. Benchmarks show tokens for the same seven search categories drop from 4,668 to 3,412—a 27% saving—with greater savings the deeper the nesting: Flights reaches 47%, while flat Shopping saves only 5%. Seven full responses converted to Markdown total 71,244 tokens, 4.6 times smaller than the raw JSON, all fitting into a 32K window. Markdown and JSON share the same json_restrictor parameters; the app maintains separate restrictors per format, and the Markdown version can keep more results and fields, improving answer quality. However, while Raw mode fits, it reads two to three times as many tokens as restrictor mode, extending latency by up to 2.7x, and two large searches in one question can still silently overflow the window and cause miscalculations. The author gives recommendations for choosing among the four outputs and compares the speed and stability of four local models.