TinyFish: The Full Pipeline from Post-Training to Production Deployment for Web-Native Models

What Is TinyFish? Why Web-Native Models Need Specialized Infrastructure

TinyFish is a company that builds web agent infrastructure for enterprises. Its core product includes capability modules such as Search, Fetch, and Browser, with the goal of enabling enterprise-grade AI agents to reliably complete search, scraping, and interaction tasks in real, dynamic web environments.

This is fundamentally different from traditional large language model applications. Most models can simply perform inference on static datasets, but web-native models must contend in real time with constantly changing page structures, anti-scraping mechanisms, dynamically rendered content, and unpredictable network latency. This means the pipeline from post-training to production deployment requires an entirely different engineering system. TinyFish chose to run its models on Databricks AI Runtime precisely to address this set of challenges.

From Post-Training to Production: TinyFish's Technical Pipeline

Special Requirements of the Post-Training Phase

Post-training for web-native models differs from fine-tuning a general-purpose LLM. The model needs to learn how to parse HTML structures, identify key information on a page, make decisions among multiple candidate actions, and handle various edge cases (such as page load failures, CAPTCHA blocks, and sudden content format changes).

Training these capabilities depends on a large volume of interaction data from real web environments. The post-training phase requires frequent iteration on model weights while maintaining inference behavior consistent with the production environment. If there is an environment gap between training and inference, the model will experience a performance cliff after deployment.

Post-training is a phase that stands in contrast to pre-training. Pre-training allows a model to acquire general language capabilities from massive amounts of unlabeled text, while post-training builds on that foundation—through supervised fine-tuning (SFT), reinforcement learning from human feedback (RLHF), or direct preference optimization (DPO)—to align the model with specific tasks, domains, or behavioral norms. For web-native models, the goal of post-training is not to make the model "better at chatting," but to teach it to complete sequences of operations in real web pages—parsing DOM structures, locating interactive elements, determining where to click or type next, and falling back and retrying when page states are abnormal. These capabilities are difficult to acquire from static corpora; they must rely on interaction trajectories with real browsing environments as training signals.

It is worth noting that the post-training phase also faces the risk of "catastrophic forgetting": when a model is repeatedly fine-tuned on web interaction data, it may lose the general reasoning and language understanding capabilities acquired during pre-training. Therefore, in practice, post-training for web-native models typically adopts a mixed-data strategy, training on a balanced ratio of general instruction data and web interaction data.

Running Production Workloads on Databricks AI Runtime

TinyFish deploys its models on Databricks AI Runtime, a choice that solves several key problems:

• Elastic compute scheduling: Web Agent request volumes fluctuate dramatically, and Databricks Runtime can scale inference resources on demand, avoiding the need to reserve GPUs long-term for peak traffic.
• Training-inference consistency: Completing post-training and production deployment on the same platform reduces the risk of inconsistency caused by environment switching.
• Observability: Databricks provides monitoring capabilities for model inference latency, throughput, and error rates, which is critical for agents that need to respond to web page changes in real time.

Training-inference parity is an often-underestimated but hugely impactful problem in machine learning engineering. It refers to fully aligning the input distribution, preprocessing logic, framework versions, and runtime environment that a model sees during training with those in the production inference phase. Once a discrepancy arises—for example, post-training used PyTorch 2.1 + CUDA 12, while the production inference environment is PyTorch 1.13 + CUDA 11—issues such as numerical precision differences, inconsistent operator behavior, or even different tokenizer results can occur, ultimately causing the model's performance on actual requests to diverge severely from offline evaluation results.

For applications like Web Agents that are highly sensitive to latency and robustness, the cost of training-inference inconsistency is especially high: a model that can correctly parse a page in offline tests may, after deployment, click the wrong button or lose critical information due to rendering differences. Running post-training and inference on the same platform (such as Databricks AI Runtime) essentially eliminates these hidden discrepancies at the engineering level, making model behavior predictable and reproducible.

The Industry Significance of Web Agent Infrastructure

The TinyFish case reflects a broader trend: AI Agents are moving from "being able to answer questions" to "being able to complete tasks." And the prerequisite for completing tasks is that the agent can browse and operate web pages like a human.

The technical difficulty in this direction is not how large the model itself is, but whether the entire engineering pipeline can support the model's stable performance in real environments. From data collection, post-training, and deployment to monitoring, every stage requires specialized optimization for web scenarios. TinyFish's choice of Databricks AI Runtime as its foundation shows that enterprise-grade Agent infrastructure is moving from "build it yourself" to "use a mature platform."

For enterprises building similar capabilities, TinyFish's path offers a reference: the key to productionizing web-native models lies in the consistency of training and inference environments, as well as the ability of elastic compute to support uncertain workloads.

Web Agent is a subcategory of AI Agent, referring to intelligent agents that can autonomously browse web pages, understand page content, and execute operations (such as clicking, filling out forms, scrolling, and extracting information). Its key difference from traditional crawlers lies in "decision-making": crawlers scrape according to preset rules, while Web Agents need to dynamically select actions based on real-time page states. This naturally exposes them to three types of challenges—dynamic rendering (many pages rely on JavaScript to generate content, and the target data is not present in the static HTML), anti-scraping and verification mechanisms (CAPTCHAs, fingerprinting, behavior detection), and long-horizon task planning (completing a purchase may require dozens of steps, each of which can go wrong).

From an engineering perspective, the bottleneck of Web Agents is often not the number of model parameters, but the stability of the entire pipeline: whether data collection covers the real distribution, whether post-training is consistent with the inference environment, whether inference resources can handle traffic spikes, and whether online performance can be continuously observed. This is also the fundamental reason why "Web Agent infrastructure" has emerged as an independent track.

Frequently Asked Questions

What is the difference between TinyFish's web-native models and ordinary LLMs?

Ordinary LLMs primarily train and perform inference on static text, while web-native models need to process dynamic web content in real time, including HTML parsing, page interaction, and exception handling. This requires the model to be exposed to interaction data from real web environments during the training phase.

Why choose Databricks AI Runtime instead of building a self-managed inference cluster?

Databricks AI Runtime provides elastic GPU scheduling and a unified training-inference environment. For Web Agent scenarios with highly variable request volumes, scaling on demand is more economical than reserving compute long-term, while also reducing the risk of environment discrepancies between training and inference.

One-line Summary

TinyFish builds Web Agent infrastructure on Databricks AI Runtime to solve training-inference consistency and elastic compute challenges.

Paragraph Summary

The article introduces TinyFish, a company that builds web agent infrastructure for enterprises, with core capabilities including modules such as Search, Fetch, and Browser. Unlike general-purpose LLMs that run on static datasets, web-native models must contend in real time with constantly changing page structures, anti-scraping mechanisms, and dynamically rendered content, which requires an entirely different engineering system across the entire pipeline from post-training to production deployment. TinyFish deploys its models on Databricks AI Runtime, gaining elastic compute scheduling, training-inference environment consistency, and observability of inference latency, throughput, and error rates. The article notes that AI Agents are moving from "being able to answer questions" to "being able to complete tasks," and the technical difficulty lies not in model scale but in the stability of the engineering pipeline. TinyFish's path offers a reference for enterprises building similar capabilities.