Together Launches Preemptible Compute: Same GPU Power at Half the Price

What Is Preemptible Compute

Together GPU Clusters recently announced support for preemptible compute, offering the same GPU capacity at a flat 50% discount off on-demand pricing. For AI teams long burdened by GPU costs, this is a signal worth paying attention to — it applies a well-established cloud pricing model to push the barrier to entry for training and inference even lower.

The core logic behind preemptible compute isn't complicated: cloud providers always have some idle or fluctuating compute resources on hand. Rather than letting those resources sit unused, they sell them at a lower price to users willing to accept the risk of interruption. When the platform needs to reclaim those resources, it notifies users in advance before terminating instances. The key parameter Together provides here is a five-minute drain window, meaning a task has five minutes to save its state and wrap up before being preempted.

The concept of preemptible compute was first introduced to the public cloud market by Google Cloud in 2015 under the name "preemptible VMs." AWS offers a similar mechanism called "Spot Instances," while Azure's equivalent is "Spot Virtual Machines." The shared principle: physical resource utilization in data centers exhibits clear peaks and valleys over time. Idle capacity during off-peak periods is sold at steep discounts, then reclaimed when higher-priority demand arrives — essentially a dynamic capacity auction mechanism. Compared to Reserved Instances and On-Demand instances, preemptible instances occupy the "lowest price, least stable" end of the spectrum, forming the third tier of cloud compute pricing. For GPU resources specifically, since individual cards cost far more than typical CPU instances, the same percentage discount translates into much larger absolute savings — which is why this model has drawn particular attention since being introduced to the AI compute market.

Why Half Price Is So Attractive

GPU compute is one of the most expensive cost items in AI development today, especially for model training, large-scale fine-tuning, and batch inference workloads, where compute bills often dominate the entire project budget. Together cuts the price directly to 50% of on-demand rates, while the compute specifications remain identical — the same GPU capacity, just with the possibility of being preempted.

This pricing model isn't Together's invention; it mirrors the Spot Instances and preemptible VM patterns that have long existed in the cloud computing industry. The real difference is that Together is targeting GPU clusters specifically — a high-value resource pool — aimed squarely at the AI training and inference workloads that are most price-sensitive. For research teams, startups, or anyone running large batches of experiments on a limited budget, this means running twice the workload for the same spend.

What the Five-Minute Drain Window Means

The biggest risk with preemptible compute is that tasks can be interrupted mid-run. Together's five-minute drain window directly mitigates this risk. Compared to platforms that give only a few seconds of notice — or none at all — five minutes is relatively generous: enough time to trigger a checkpoint save, gracefully shut down processes, or migrate a task.

This places clear requirements on workload design: tasks must be interruptible and resumable. In practice, this means implementing periodic checkpointing in the training pipeline, so that after a preemption, work can resume from the most recent saved state rather than starting over from scratch. For well-designed distributed training frameworks and inference services, this kind of fault tolerance is already standard — which means the engineering effort to adapt workloads for preemptible compute is relatively low.

The checkpoint mechanism is a core fault-tolerance technique in deep learning training. It involves periodically serializing and persisting the complete training state — model weights, optimizer state (such as Adam's momentum and variance), current training step, and so on — to a storage medium. If a task is interrupted, it can resume from the most recent checkpoint rather than restarting from step zero. Major training frameworks such as PyTorch's torch.save / torch.load, the Trainer class in Hugging Face Transformers, and distributed training frameworks like DeepSpeed and Megatron-LM all have built-in support for periodic checkpointing. In a preemptible setting, the right checkpoint frequency requires balancing storage overhead against potential recomputation cost — checkpointing too frequently consumes significant I/O bandwidth and storage space, while checkpointing too infrequently means more lost computation if preemption occurs. The five-minute drain window provides ample time to automatically trigger one final checkpoint save, which is its direct engineering value.

Which Scenarios Are a Good Fit

Preemptible compute isn't a universal solution — it's best suited for workloads with higher tolerance for interruption:

• Large-scale model training and fine-tuning: With checkpointing in place, tasks can resume seamlessly after preemption, trading half the cost for longer training runs.
• Batch offline inference: Batch processing jobs with no strict real-time requirements can simply be re-queued after an interruption.
• Hyperparameter search and experimentation: Large numbers of short parallel jobs where the impact of any single preemption is limited.
• Data preprocessing and feature engineering: Pipeline jobs that can be split and re-run as needed.

Conversely, latency-sensitive production inference services that require continuous uptime are better suited for on-demand or reserved instances. A common approach is hybrid deployment: stable instances for core online services, preemptible instances for elastic and batch workloads — striking a balance between cost and reliability.

Industry Significance

Together's move reflects a broader trend in the AI infrastructure market toward more granular resource pricing and tiered supply. As demand for large model training and inference continues to grow, compute providers need to serve two types of users simultaneously: those who prioritize maximum stability and are willing to pay full price, and those who are price-sensitive and willing to trade flexibility for discounts. Preemptible compute fills exactly this latter need.

From a broader perspective, this model also helps improve overall compute utilization — putting GPU resources that might otherwise sit idle to work, creating a win-win for both platforms and users. For teams evaluating GPU cloud services, Together's preemptible option is worth factoring into cost optimization decisions. The key is assessing your workload's tolerance for interruption and building in proper fault-tolerance design — starting with checkpointing.