Buy or Rent H200 GPUs? A Real Break-Even Analysis

In discussions about AI infrastructure, "buy GPUs or rent them" is a question almost every team has to wrestle with. Plenty of people in the community have strong opinions, but few actually run the numbers carefully. One Reddit user built a real break-even model for their team's procurement decision — and the conclusions didn't match what most people would intuitively expect. This article walks through that calculation framework and fills in a few hidden costs that are easy to overlook.

Hardware Cost and Rental Price Baselines

The calculation starts with two data points. An 8-GPU HGX H200 server runs roughly $320,000–$420,000 to purchase outright; a reasonable midpoint is around $370,000. That's the one-time hardware investment.

On the rental side, the author surveyed on-demand H200 pricing across 34 providers and found a median of roughly $4.40 per GPU-hour. An important caveat here: the $2–$3/hour prices you occasionally see are closer to spot (preemptible) pricing, not sustainable on-demand rates. Using spot prices for long-term planning will seriously underestimate your rental costs.

Converted to a full 8-GPU node, the equivalent rental cost is about $35.20 per hour — the basis for all the break-even math that follows.

HGX H200 is NVIDIA's flagship multi-GPU interconnect platform for data centers. In an 8-GPU configuration, eight H200 GPUs are connected via NVLink and NVSwitch for high-bandwidth communication, delivering a combined 1.1 TB of HBM3e memory. The H200's key upgrade over the H100 is memory capacity and bandwidth, making it especially valuable for large model training and inference.

On-demand vs. spot pricing is worth clarifying: on-demand instances are always available and won't be interrupted — suitable for training jobs that need stable uptime. Spot instances are cheaper (typically 30–70% less) but can be terminated by the cloud provider when resources are scarce, making them suitable only for short-term workloads that support checkpointing and can tolerate interruptions. Using spot prices to evaluate long-term total cost of ownership means using the best-case scenario as your baseline — which systematically understates actual rental spend.

The Hardware-Level Break-Even Point

Comparing the $370,000 purchase price against $35.20/hour in rental costs — looking at hardware alone — the payback period depends entirely on utilization:

100% utilization: ~14.4 months to break even
60% utilization: ~24 months to break even
40% utilization: ~36 months to break even

The counterintuitive part: at full load, breaking even in just over a year sounds like buying is a clear win. But the author points out that most small teams run a mix of bursty training and steady-state inference — sustaining 100% utilization long-term is nearly impossible. Once utilization drops to 40%, the payback window stretches to three years — by which point that hardware is already a generation old.

Four Hidden Costs That Are Easy to Underestimate

The author emphasizes that the analysis above is hardware-only. Any real decision needs to account for at least four additional cost categories:

Power and Cooling

The author's own experience: colocation cage pricing came in higher than budgeted. GPU servers have extreme power and cooling requirements, and this ongoing expense is routinely underestimated upfront.

Depreciation

The author's one-liner: "Whatever depreciation rate you assumed, cut it in half." The secondary market for previous-generation data-center hardware is thin, and residual values fall far below what you'd expect based on consumer hardware intuition. Owned equipment loses book value faster than most people anticipate.

Your Own Time

Building your own infrastructure means investing significant hidden labor in operations, troubleshooting, and procurement negotiations. For small teams, founder and engineer time is often the scarcest resource — a cost that's hard to quantify but very real.

Idle Hours

Once you've bought the hardware, costs accrue whether or not jobs are running. Idle time directly drags down effective utilization and delays break-even.

The Bottom Line: Utilization Is the Only Variable That Matters

The author's final verdict is clear:

If you can sustain ~60% utilization over two years, buying wins.
If utilization stays below 40%, renting is the better deal.

This framework reduces a complex decision to a single core variable — real, sustained utilization — not theoretical peak capacity.

The author also mentions one approach that can improve the economics of ownership: selling idle compute to offtake networks, using that revenue to offset hardware costs. This effectively converts "idle hours" from a pure negative into a partial revenue stream, lowering the effective break-even point.

Offtake networks are a category of compute marketplace that has emerged in recent years. The model: owners of purchased GPUs sell idle capacity at agreed rates to a platform or aggregator, which then distributes it to users with short-term compute needs. Notable players include GPU-focused cloud providers like CoreWeave and Crusoe, as well as P2P compute marketplaces like Vast.ai and RunPod. For GPU owners, participating in an offtake network is essentially a way to monetize idle time slots — effectively boosting your utilization rate and shortening the paper payback period. That said, these platforms typically have strict requirements around hardware specs, network bandwidth, and uptime, so integration costs and operational complexity shouldn't be ignored.

What This Means for Your Team

The greatest value of this framework isn't that it gives a universal answer — it's that it provides a reusable decision template. Any team can plug in their own purchase quote, local power and colocation costs, and most importantly, an honest utilization estimate, to arrive at their own conclusion.

Two common mistakes are worth watching out for: using spot pricing to evaluate rental costs, and underestimating depreciation and residual value loss. Avoiding those two pitfalls — and given how bursty most small-team workloads tend to be — renting is often more competitive than people assume.

The author closes with an open question: what utilization rates are people actually seeing in practice? That framing itself makes the point — without real utilization data, any buy-vs-rent debate is just speculation.