Together AI Partners with IBM Cloud and NVIDIA to Scale Enterprise Inference Compute

Enterprise Inference Enters the Dedicated Cluster Era

Together AI has announced a partnership with IBM Cloud and NVIDIA to launch a dedicated inference cluster powered by NVIDIA B300, enabling enterprises to run open-source models at production scale. This move directly addresses one of the core pain points enterprises face when deploying large language models — how to reliably bring open-source models into real business workflows while maintaining performance and cost control.

For a growing number of enterprises looking to break free from closed-source API lock-in and shift toward self-controlled open-source models, the availability and orchestration of inference compute is becoming a decisive factor. At its core, this three-way partnership is about laying the infrastructure groundwork for large-scale open-source model deployment.

What the B300 Dedicated Cluster Means

At the heart of this collaboration is a dedicated inference cluster built by Together AI, hosted on IBM Cloud, and powered by NVIDIA B300 hardware. Compared to shared resources, dedicated clusters offer enterprises more predictable performance and resource isolation — critical for production environments with strict SLA requirements.

The B300, as NVIDIA's accelerator platform designed for high-throughput inference workloads, can support real-time inference for models with significantly larger parameter counts. Organizing these into a dedicated cluster means enterprise customers no longer have to trade off response latency against cost in a competitive resource pool — instead, they get stable, predictable inference throughput.

Division of Responsibilities

Based on available information, each party plays a distinct role in this collaboration: Together AI is responsible for building and optimizing the inference stack, IBM Cloud provides the cloud infrastructure and enterprise-grade service capabilities, and NVIDIA supplies the underlying hardware acceleration platform. This division covers the full stack — from hardware and cloud platform to inference software.

NVIDIA B300 is a data center GPU based on the Blackwell architecture, part of NVIDIA's next-generation flagship accelerator lineup. Compared to the previous-generation H100/H200 (Hopper architecture), the Blackwell architecture introduces FP4 precision support and a second-generation Transformer Engine for inference workloads, dramatically improving performance-per-watt and throughput density without sacrificing model accuracy. The B300 variant further enhances memory capacity and bandwidth configurations, making it especially well-suited for running ultra-large models at the 700B parameter scale. For enterprise inference scenarios, higher memory bandwidth translates directly to lower time-to-first-token (TTFT) and higher tokens per second (TPS) — two metrics that directly impact user experience and system throughput costs.

The inference stack is the software layer between hardware and business applications in large model deployments, typically covering model loading and weight sharding, KV Cache management, continuous batching, quantized inference, and request scheduling. Together AI has built a deep optimization system in this domain, grounded in high-performance kernels such as FlashAttention and FlashDecoding. Its proprietary inference engine is designed to fully unlock the compute potential of underlying GPU hardware. This is Together AI's core value in this three-way partnership — IBM Cloud and NVIDIA respectively solve the questions of "where to run" and "what to run on," while Together AI answers the question of "how to run fast and reliably."

Accelerating Open-Source Model Deployment

Open-source models have rapidly closed the capability gap with closed-source alternatives in recent years, but enterprises looking to use them in production often hit a wall at the deployment and operations stage. The emergence of dedicated inference clusters lowers the barrier to building production-grade AI infrastructure, moving "running open-source models at production scale" from theory to operational reality.

This also reflects a broader trend: commercial services built around open-source models are shifting from simply providing models to delivering integrated "model + compute + operations" solutions. Together AI's alliance with IBM Cloud and NVIDIA is a prime example of this direction.

Conclusion

Given the limited details in the original disclosure, specifics such as cluster size, pricing models, and the list of supported models remain unclear. But strategically, this partnership strengthens the infrastructure supply for enterprise-grade open-source model inference and opens a practical path for more enterprises to adopt open-source models in production environments. As more details emerge, the real-world impact of this collaboration will be worth watching closely.