[KongchangAI]
· 2 min read· 1,233 words

Running Language Models in the Browser: Catching Silent Errors in Orders

Running Language Models in the Browser: Catching Silent Errors in Orders

Browser-run 0.8B model demo shows how to catch silent errors in language model order parsing.

The article introduces a research project that compiles a fine-tuned 0.8B parameter language model into WebAssembly via llama.cpp, running entirely in a browser tab to parse natural-language orders into structured data. The project highlights "silent errors"—orders that are perfectly formatted but semantically wrong, which bypass automatic validation and are more dangerous than explicit format errors. The demo page uses a TypeScript matcher from the PIZZA benchmark to score orders, allowing users to find or construct silent errors. Across all 1,357 test orders, browser and server outputs matched 99.0%, both achieving an exact match score of 0.882, demonstrating the reliability of edge deployment. The article also discusses the advantages of edge AI in privacy protection and low latency, and calls on developers to prioritize semantic validation.

When language models are used to convert natural-language orders from users into structured data, they can make two kinds of mistakes: one is garbled or malformed output, which any system can easily intercept; the other is output that is perfectly formatted and plausible but describes an order that doesn't exist at all—we call this a "silent error," and it is the most dangerous kind. Recently, a research project ran a fine-tuned 0.8B parameter model entirely within a browser tab via a WebAssembly-compiled llama.cpp, letting users experience firsthand how to catch such silent errors.

Silent Errors: A More Insidious Failure Mode

In traditional software, errors are often accompanied by exceptions or crashes, which developers can quickly locate. But language model failures can be silent: the model outputs a perfectly valid structure that simply doesn't match the user's true intent. For example, the user says "I want a medium pizza," but the model generates "a large pizza with double cheese." The format is flawless, but the order content is wrong. Such errors are hard to detect in subsequent processes and can lead to delivery mistakes, inventory chaos, or financial loss.

The accompanying work for this project names this kind of error "silent errors" and emphasizes their danger. Unlike explicit format errors, silent errors do not trigger any automatic validation and can only rely on manual review or smarter verification mechanisms. Manual review is costly, especially in high-volume order scenarios.

A Browser-Based Order Parsing Demo

To let people intuitively experience silent errors, the research team built a web demo. The page integrates a fine-tuned 0.8B parameter language model, compiled to WebAssembly via llama.cpp, running entirely within the visitor's browser tab. This means the order text never leaves the user's device, and privacy is fully protected.

The demo is available at https://wrong.cuyor.com. Visitors can input order text, and the model parses it in real time to generate a structured order. The page uses the PIZZA benchmark's matcher (ported to TypeScript) to score each order, returning "?" for items outside the menu so the model can face the open web. Users can either actively look for silent errors or try to construct one, exploring the model's boundaries.

Demo screenshot

How llama.cpp and WebAssembly Work Together

llama.cpp is an efficient inference engine written in C/C++, designed specifically for running quantized large language models on CPUs. It supports low-precision quantization such as 4-bit and 5-bit, greatly reducing memory usage. WebAssembly (Wasm) is a binary instruction format that can execute safely and efficiently in modern browsers, allowing C/C++ code to be compiled into modules that run at near-native speed. By compiling llama.cpp to Wasm, model weights and inference logic can be delivered with the web page and all computation is completed within the browser sandbox, with no server round-trips. The key advantages of this combination are: model files can be cached by the browser, enabling offline use after the first load; and the Wasm sandbox ensures that code cannot access the user's file system beyond its permissions, balancing performance and security.

Model Performance and Latency Measurements

The research team ran all 1,357 PIZZA test orders in the browser and compared the deployed model's output with server-side evaluation results. The results showed that for 99.0% of orders, the browser-side output was completely identical to the server-side output, and it achieved the same exact match score (0.882). This result demonstrates that running such a small model in the browser behaves highly consistently with the server environment, with guaranteed reliability.

In addition, the team reported the model's latency on three consumer laptops. Although the abstract does not give specific numbers, these measurements provide a reference for real-world deployment. Considering the model has only 0.8B parameters and runs entirely locally, the latency is within an acceptable range and suitable for interactive applications.

The PIZZA Benchmark and Exact Match Score

PIZZA is a public benchmark for the "natural language to structured order" task, containing 1,357 human-annotated order texts and their corresponding structured representations (such as pizza size, toppings, quantity, etc.). During evaluation, the system compares the model's structured order with the ground truth field by field. The exact match score requires all fields to be completely identical to count as correct, so 0.882 means approximately 88.2% of orders were parsed completely and correctly. The benchmark also provides a matcher to handle synonymous expressions and out-of-menu items—when the model outputs a topping not on the menu, the matcher returns "?", thereby including cases of "correct format but out-of-bounds content" in the evaluation, which aligns closely with the focus on silent errors.

Why Silent Errors Deserve Attention

The danger of silent errors lies in their stealth. In scenarios such as order processing, medical records, and financial transactions, data that is correctly formatted but semantically wrong can trigger cascading problems. Traditional validation can only check format, not semantic correctness. Although language models can understand natural language, their output still requires careful handling.

Through the browser demo, this project lets developers and users experience the occurrence of silent errors firsthand, encouraging greater attention to verification when designing systems. For example, an additional semantic validation layer can be introduced, or users can be asked to confirm key information. At the same time, running the model in the browser also demonstrates the potential of edge AI: providing low-latency intelligent services while protecting privacy.

The Dual Advantages of Edge AI and Privacy Protection

Deploying language models in the browser not only eliminates the privacy risk of data transmission but also reduces dependence on server resources. User input is processed entirely locally, which is especially valuable for scenarios involving sensitive information such as orders and medical consultations. The combination of llama.cpp and WebAssembly makes it possible to run small language models in resource-constrained environments.

With advances in model compression and compilation technology, more AI capabilities will move to edge devices in the future. This demo project is not only a technical proof of concept but also provides developers with a reusable paradigm: use small models for specific tasks, achieve instant feedback in the browser, and catch silent errors through carefully designed verification mechanisms.

Edge AI and Model Offloading

Edge AI refers to the deployment paradigm of moving model inference from cloud data centers to user devices (browsers, phones, PCs, IoT devices). It is driven by three core factors: first, privacy—sensitive data does not need to leave the device; second, latency—local inference eliminates network round-trips and is suitable for interactive scenarios; third, cost—service providers do not need to pay for server compute for every inference. However, edge devices have limited memory and compute, and can typically only host small, quantized, compressed models (such as the 0.8B model in this project). Such models can approach the performance of large models on specific narrow tasks but have weaker generalization, so they require task-specific fine-tuning and strict output validation—which is exactly why the silent error problem deserves more vigilance in edge scenarios.

Conclusion

Silent errors are an easily overlooked but serious problem in language model applications. Through the demo of a 0.8B model running in the browser, the research team allows us to intuitively explore this phenomenon. The 99.0% consistency result and the 0.882 exact match score show that edge-deployed models can achieve reliability comparable to server environments. For developers, while pursuing model performance, they should pay more attention to designing robust validation processes so that silent errors have nowhere to hide.

#silent errors#browser language model#WebAssembly#llama.cpp#edge AI#order parsing#privacy
Share:

Related articles