Why Serious AI Teams Will Eventually Build Their Own Data, Evals, Models — and Chips

From General to Specialized: The Evolution of AI Infrastructure

A take from swyx, host of the Latent Space podcast, has been making rounds in the industry: Every serious AI team will eventually build its own data pipelines, evaluation systems, models, and even chips. Simple as it sounds, this statement captures an increasingly clear trend in AI engineering — the future belongs to "domain-specific everything."

This conclusion is the inevitable result of general-purpose large models hitting their limits. When a company truly treats AI as a core competitive advantage rather than a nice-to-have feature, the constraints of relying on generic solutions become apparent fast. Whether it's the uniqueness of your data, the business alignment of your evaluation criteria, or the cost structure of model inference — all roads lead toward building your own.

The Four Layers of Building In-House

The four directions swyx outlines map neatly onto the AI system stack from top to bottom, and reflect where teams at different stages of maturity tend to focus their investments.

Data: The Starting Point of the Moat

Proprietary data is often the first thing AI teams realize they need to own. General-purpose datasets are available to everyone and can't create differentiation; only data accumulated through real business operations can help a model outperform competitors on specific tasks. Building your own data pipelines, annotation workflows, and feedback loops is the first step for virtually every serious AI team.

Evals: The Underrated Critical Layer

Evaluations (evals) are frequently overlooked, yet they're central to iteration speed. Public benchmarks can't accurately reflect real-world business performance — teams must design evaluation criteria tailored to their own use cases. Without reliable evals, there's no way to measure model improvements, and optimization easily devolves into subjective guesswork. This is a point swyx has repeatedly emphasized in the AI Engineer community.

Evals in large model engineering play a role similar to unit tests and integration tests in software engineering — they define what "correct" looks like. Industry-standard public benchmarks like MMLU, HumanEval, and HellaSwag measure model performance on standardized tasks, which often have a massive gap with real business scenarios. For example, a legal document assistant might need to be evaluated on citation accuracy, format compliance, and precision of language — metrics you won't find on any public leaderboard. Building high-quality business evals requires domain experts to design test sets, establish human-annotated golden sets, and track model iteration effects through continuous A/B comparisons. Many teams don't realize until they've been burned in production that a lack of reliable evals leads to an "optimization illusion" — where the numbers look good, but actual user experience hasn't improved, or has even regressed.

Models: From Fine-Tuning to Training From Scratch

As requirements deepen, teams often progress from calling APIs to fine-tuning open-source models, and eventually to training their own models in certain scenarios. This is not just about performance — it's also about cost control and data privacy. Building at the model layer represents a significant increase in a team's ownership of AI capabilities.

Chips: The Endpoint of Extreme Vertical Integration

"Even chips" is the most radical — and most symbolically significant — item on this list. Custom silicon only makes economic sense when scale and use cases are extreme enough; it represents a company's willingness to pay the deepest possible cost of vertical integration for a specific workload. This is largely a game for the top players, but it marks the theoretical ceiling of the specialization trend.

Building custom AI chips is not science fiction, but the bar is extremely high. Google's TPU (Tensor Processing Unit) is the most prominent example: purpose-built for matrix multiplication and deep learning inference, it offers significant energy efficiency and cost advantages over general-purpose GPUs for training and serving large Transformer models. Meta's MTIA, Amazon's Trainium/Inferentia, and Tesla's Dojo are all examples of internet giants doing silicon-level customization for their own specific workloads. The economic logic behind custom chips is: when inference or training costs represent a meaningful share of company revenue, even a 10–20% improvement in chip efficiency can translate to hundreds of millions of dollars in savings at scale. For the vast majority of teams, this layer remains something to "understand as a trend" rather than "put into practice" — but it reveals the ultimate form that specialization competition can take.

Why the Future Is "Domain-Specific Everything"

swyx's core argument is that there is a fundamental tension between generality and specialization. General-purpose models solve the question of "can it be done" — but in real commercial competition, "how well, how cheaply, and how reliably it can be done" is what determines the winner. And optimizing those metrics almost always points toward domain-specific customization.

This trend doesn't mean general-purpose large models will disappear. Rather, it means value creation in AI is shifting away from the model itself toward the complete system built around it — data, evaluations, engineering deployment, and even underlying hardware. In other words, the model is just the starting point; real differentiation happens in how a team refines the entire stack for their own context.

This trend has its own internal economic logic, which can be understood through the concept of "diminishing premiums on generality": when a technology matures and becomes widely adopted, the general-purpose version gradually commoditizes, margins compress toward zero, and competitive advantage necessarily migrates to the vertical customization layer. GPUs relative to CPUs, cloud databases relative to general-purpose relational databases — both have followed similar evolutionary paths historically. For AI, GPT-4-level general capability has already become — or is becoming — "infrastructure" rather than a "moat." What's truly scarce is the engineering capability to deeply integrate that capability with a specific domain: acquiring and cleaning domain data, defining evaluation criteria that match business logic, building deployment architectures optimized for latency and cost, and the organizational knowledge and engineering intuition accumulated along the way.

Implications for AI Practitioners

For teams building AI products, this perspective offers a clear roadmap: don't try to compete head-on with large players on general capabilities. Instead, invest early in building your own data and evaluation assets. These assets compound over time, forming barriers that are difficult to replicate.

Notably, the AI Engineer community that swyx is part of is hosting an in-person event in San Francisco on November 3rd to dive deep into these topics. For developers and teams who want to stay ahead of AI engineering trends, these conversations are becoming increasingly important.

As AI moves from the lab into production, "doing it yourself" may no longer be a choice — it may be the inevitable path for anyone who takes AI seriously.