NVIDIA's Open-Source AI Router PAIR Tested: A New Breakthrough in Multi-Device Collaborative Inference

NVIDIA's Open-Source Strategic Play

The past week has been dubbed an "AI mega-week" by industry insiders — Anthropic released new models, OpenAI launched its latest version, and countless discussions swirled around these cutting-edge closed-source systems. But amid all that noise, a more significant development came from NVIDIA: it is systematically staking out the "local AI" space.

NVIDIA has always been clear about its stance on open source. It doesn't just release open-weight models — it also publishes training papers, training recipes, and datasets, giving the community full transparency into how models are built and enabling independent fine-tuning. This strategy has allowed NVIDIA to surpass other corporate players and become the institution with the most contributed repositories on Hugging Face.

Even more telling: sources indicate that NVIDIA acquired Hugging Face for approximately $13 billion. While many worried that an acquisition would harm the open-source ecosystem, industry observers argue the move may have actually blocked other potential acquirers who might have closed it off — because keeping it open is fundamentally aligned with NVIDIA's core interests.

Local AI Reaches a Tipping Point

Why does NVIDIA care so much about Hugging Face? Because it's essentially the "hub of local AI," and local AI is approaching a mass-market tipping point.

A clear-eyed take: the tech community is not "the mainstream" — it's the early adopters running well ahead of the curve. Everyday users are gradually realizing that AI genuinely improves their productivity, while also — having lived through the data-abuse era of social media — being reluctant to hand over all their data. These two forces together are generating massive demand for local AI.

Data backs this up. The number of open models on Hugging Face has grown more than fourfold compared to the same period last year, with generative AI models showing dramatic overall growth. More importantly, open-source models are closing the gap on closed-source ones at an accelerating pace: scores that Claude Opus Max once held on human-analyzed intelligence benchmarks have since been surpassed by Qwen; in the video model space, Gemini's performance has been overtaken by MiniMax. These models still require substantial compute to run locally, but the mere fact that they can run locally is itself a signal of the trend.

PAIR: An AI Traffic Scheduler for the Multi-Device Era

This is the context in which NVIDIA introduced PAIR — the Personal AI Router.

The future scenario NVIDIA is betting on: homes will have multiple devices equipped with GPUs or AI accelerators. This prediction has a real foundation — RTX Spark laptops are on the way, and multiple hardware vendors have already announced Windows systems with built-in RTX. It's easy to imagine that within a few years, as people upgrade their devices, machines will routinely ship with some form of AI acceleration chip. What PAIR addresses is the question: "When you have multiple such devices, how do AI agents make full use of all of them?"

How PAIR Differs from Switchyard

PAIR is often compared to Switchyard. Switchyard operates at the API layer — whether cloud or self-hosted — selecting the most appropriate model based on quality and cost, handling decisions that may involve traffic leaving your network. PAIR, by contrast, targets local hardware you fully control: it's a traffic scheduler for local inference.

The core pain point it solves is this: agents like Hermes and OpenClaw are powerful precisely because a main agent can delegate research, coding, and other tasks to parallel sub-agents. But when running fully locally, those sub-agents often aren't truly parallel — they queue up waiting for the same GPU to become free. PAIR's job is to distribute those sub-agents across multiple devices on the network.

Think of PAIR as a "virtual inference traffic scheduler": when it detects that machine #1 is busy, it forwards the request to machine #2. The agent has no idea this is happening — it thinks it's just calling Ollama or LM Studio, making requests exactly as before. PAIR silently distributes the load in the middle.

Switchyard represents a class of "AI model routing" middleware whose core logic is to dynamically select the optimal option among multiple model APIs — weighing response quality, inference speed, and cost — transparently to the calling application. It typically runs as a proxy layer compatible with the OpenAI API format, supporting automatic switching or load balancing across backends like OpenAI, Anthropic, and local Ollama instances. The most fundamental difference from PAIR lies in the control boundary: Switchyard's decision scope can span local and cloud, whereas PAIR's design principle is to keep inference requests entirely within the user-controlled local network, with no involvement of any third-party cloud services — providing stronger privacy guarantees.

Important Clarification: This Is Not VRAM Pooling

To be clear: PAIR is not VRAM pooling technology. It does not merge multiple devices into a single virtual machine with more VRAM. Each request still runs on a single device. More nodes bring higher parallel request throughput, not acceleration of individual tasks — and it won't let you load larger models than a single device can handle.

VRAM Pooling is a more aggressive distributed inference technique whose goal is to merge the VRAM of multiple physical devices into a single unified memory space at the software level, allowing a single model's weights to be sharded and computed across devices. Representative approaches include NVLink-based multi-GPU interconnects and certain research-stage distributed inference frameworks. The advantage of VRAM pooling is the ability to run models whose parameter count exceeds what a single card's VRAM can hold (e.g., a model with hundreds of billions of parameters that can't fit on a single 24GB card). However, implementation complexity is extremely high, with stringent requirements on network bandwidth and latency — consumer-grade LAN environments present obvious bottlenecks. PAIR explicitly forgoes this direction, opting for the more pragmatic request-level scheduling, trading architectural simplicity for practical usability.

Open-Source License and Community Co-Development

PAIR's most exciting characteristic is that it's open-sourced under the Apache 2.0 license — anyone can fork, modify, and customize it. Currently it targets personal networks, but it's easy to imagine that with a capable coding model, it could be adapted to support cloud-based networks. There's even speculation that PAIR and Switchyard could eventually merge into a unified product.

The project is currently at version 0.1, with out-of-the-box support for Windows, Linux, and Mac. It works even when running Ollama on a Mac and isn't limited to NVIDIA GPUs. Right now it only supports Ollama and LM Studio, but the NVIDIA team has publicly stated it welcomes community contributions. Indeed, the community has already submitted pull requests adding support for the Anthropic-style API, llama.cpp, and more. Because it uses the Apache 2.0 license, even if NVIDIA over-restricts the project, the community can fork an independent version — and NVIDIA's choice of this license signals that they understand this.

Apache 2.0 is one of the most commercially friendly mainstream open-source licenses available. Unlike the GPL family, which requires derivative works to be open-sourced as well (the so-called "viral" clause), Apache 2.0 allows anyone to freely use, modify, distribute, and commercialize the software — even releasing modified versions as closed-source — as long as the original copyright notice is retained. This is highly attractive to both enterprise participants and individual developers. By choosing Apache 2.0 for PAIR, NVIDIA is essentially allowing third-party cloud providers, competitors, and community developers to build commercial products on top of it without paying licensing fees or opening their own source code. This choice legally guarantees "the community can independently fork," and serves as an important signal from NVIDIA to the open-source community.

Real-World Testing and Practical Performance

From a technical standpoint, PAIR essentially takes over the ports used by Ollama and LM Studio, acts as a traffic scheduler, and routes requests over the network to the appropriate engine on the appropriate device. The benefit is that all existing applications can continue working without any port configuration changes.

The testing process was fairly straightforward: after installing the Mac version, it immediately detected the locally installed Ollama (though it failed to detect LM Studio — likely because the current build only supports newer Apple Silicon). The interface displays available VRAM and RAM, and lists models already loaded in Ollama.

In the official demo, PAIR assigns a Qwen model to an RTX 5090, assigns a different version of the Qwen MoE model to a DGX Spark, and can even coordinate with Radeon cards — provided those models are already loaded and ready.

In practice, however, version 0.1 doesn't dramatically change existing workflows. Similar results could previously be achieved with tools like Tailscale by configuring dedicated models for sub-agents on different nodes. PAIR's biggest challenge is speed: when local Qwen models already run at roughly 300 tokens/second, averaging 380 tokens/second in batch processing, whether the latency introduced by network routing and scheduling impacts overall performance when delegating inference to other devices still requires more testing to validate.

The Future of Local AI

PAIR itself may still be in its early stages, but it clearly reveals how NVIDIA is thinking about the future of local AI: from acquiring Hugging Face and promoting open weights, to building out RTX Spark hardware, to launching a multi-device inference scheduler — this is a coherent, complete strategic chain.

For users who already own multiple GPU devices, PAIR is worth trying. And for the broader open-source community, as more routers and scheduling tools get integrated, the local agent experience stands to see qualitative improvements. The race around local AI has only just begun.