Guides
How to Find the Cheapest H100 Rental for AI Workloads
Renting an H100 is rarely as simple as picking the lowest number on a price page. The cheapest H100 rental depends on your workload, your tolerance for interruptions, and how you account for storage, egress, and engineering time. This guide walks through the factors that actually move the price, so you can compare providers without getting surprised by the final bill.
What the Cheapest H100 Rental Really Means
When engineers search for the cheapest H100 rental, they usually mean the lowest hourly rate for an H100 SXM or PCIe card. But the hourly rate is only one part of the equation. A provider might charge less per GPU-hour but bill in 60-minute increments, so a 10-minute job costs you a full hour. Another might have cheap compute but expensive egress or storage. The true cost is the total you pay to run your workload from start to finish, including failed runs and idle time.
Start by defining your workload. Is it a short inference job, a multi-day training run, or an interactive notebook? Each has a different sensitivity to billing granularity, preemption, and cold-start time. For long training runs, a slightly higher hourly rate from a reliable provider can be cheaper than a low rate that gets preempted every few hours. For short jobs, per-second billing is often the single biggest lever you can pull.
Where H100 Rental Prices Come From
H100 capacity is rented through three main channels: marketplaces, specialized GPU clouds, and hyperscalers. Each has a different cost structure, and the cheapest option for you depends on which trade-offs you can accept.
- Marketplaces like Vast.ai aggregate idle GPUs from many hosts. They often have the lowest headline rates because hosts compete on price. The trade-off is variable reliability, inconsistent networking, and less predictable availability. You may also need to handle your own container setup and data transfer.
- Specialized GPU clouds like RunPod, Lambda, and CoreWeave offer a more managed experience. Prices are usually higher than marketplaces but come with better tooling, support, and consistent performance. They often provide pre-built templates for popular frameworks and faster onboarding.
- Hyperscalers (AWS, GCP, Azure) provide H100 instances with enterprise-grade networking, security, and compliance. They are almost always the most expensive per GPU-hour, but they can be the cheapest option when you factor in existing commitments, reserved instances, or the cost of integrating with other cloud services. If your data already lives in a hyperscaler's object storage, egress fees can make other providers more expensive.
If you are comparing marketplaces and specialized clouds, a side-by-side breakdown like our Vast.ai vs RunPod comparison can help you weigh the trade-offs.
H100 Variants and Their Impact on Price
Not all H100s are the same. The two most common variants for rental are H100 SXM and H100 PCIe. SXM cards use a different form factor and interconnect, which typically makes them faster for multi-GPU training. PCIe cards are easier to fit into standard servers and are often used for inference or single-GPU workloads. Because SXM is in higher demand for large-scale training, its rental price is often higher. However, for inference or fine-tuning smaller models, a PCIe card may be more than enough and could save you money. You can check current H100 SXM rental prices on our site to see how they compare to other variants.
There is also the H100 NVL, which is less common in rental fleets. If you find one, verify its memory and interconnect before assuming it will perform like an SXM. Some providers also offer H100 MIG instances, which partition a single GPU into smaller slices. MIG can be a cost-effective way to run multiple small workloads, but it limits your memory and compute per slice.
Billing Models: On-Demand, Spot, and Reserved
Most providers offer several billing models. On-demand gives you immediate access and no commitment, but you pay the highest rate. Spot instances (or interruptible instances) are significantly cheaper, sometimes 50% or more off the on-demand price, but they can be reclaimed with little notice. Reserved instances require a commitment of weeks or months and offer a discount in exchange for predictability.
For short experiments, spot is often the cheapest H100 rental option if your code can checkpoint and resume. For production inference with strict latency requirements, on-demand or reserved capacity is usually worth the premium. To estimate your monthly spend under different models, use our hourly and monthly cost estimator.
Pay attention to billing granularity. Providers that bill per second are ideal for short jobs, while per-minute or per-hour billing can add up quickly if you run many small experiments. Some providers also charge for stopped instances, so remember to terminate rather than stop when you are done.
Comparing Provider Types Without Guessing Numbers
Prices change constantly, so instead of listing specific rates, here is a generic comparison of what each provider type optimizes for. Use this to decide where to look first, then check live prices.
| Provider type | Billing granularity | Best for | Main trade-off |
|---|---|---|---|
| Marketplace | Often per second or per minute | Cost-sensitive experiments, batch jobs | Variable reliability, less support |
| Specialized GPU cloud | Per minute or per hour | Managed training, inference endpoints | Higher headline rate than marketplaces |
| Hyperscaler | Per second (with minimums) | Enterprise workloads, hybrid cloud | Highest per-GPU cost, complex pricing |
Notice that billing granularity can matter more than the headline rate. A marketplace charging per second at a slightly higher rate can be cheaper than a provider charging per hour if your jobs are short. Similarly, a specialized cloud with included storage and fast networking can beat a marketplace with cheap GPUs but slow interconnects.
How to Find the Cheapest H100 Rental for Your Workload
Follow these steps to avoid overpaying:
- Define your job length and interruption tolerance. If your job runs for less than an hour, prioritize per-second billing. If it runs for days, prioritize reliability over the lowest rate.
- Check multiple provider types. Don't assume marketplaces are always cheapest. During high demand, specialized clouds sometimes have better availability at similar prices.
- Calculate total cost. Include storage, egress, and any minimum billing increments. A cheap GPU with expensive storage can cost more than a premium GPU with included storage.
- Test with a small job. Before committing to a long run, launch a short job to measure actual throughput and startup time. A provider with slower networking might cost you more in wall-clock time.
- Monitor spot prices. If you can tolerate interruptions, spot instances are usually the cheapest H100 rental option. Set up checkpointing and automate restarts.
- Use price comparison tools. Our GPU model list shows live prices across providers, so you can spot outliers quickly.
Common Pitfalls That Inflate Your Bill
Even experienced engineers get caught by these:
- Idle time. Leaving an instance running while you debug or sleep is the fastest way to burn money. Use auto-shutdown policies.
- Egress fees. Moving large datasets or model checkpoints out of a provider can cost more than the GPU time itself. Check egress pricing before you start.
- Minimum billing increments. A provider that bills in 1-hour blocks will charge you for a full hour even if your job finishes in 5 minutes.
- Preemption without checkpointing. Spot instances are cheap until you lose 6 hours of training because you didn't save state.
- Over-speccing. Not every workload needs an H100. For smaller models, an A100 or even an RTX 4090 may be cheaper and fast enough. Check our full list of GPU models to compare options.
Why the Cheapest H100 Rental Isn't Always the Best
There is a real cost to unreliability. If a provider's instance crashes mid-training, you lose the compute time and the engineering time to restart. If networking is slow, your distributed training job may spend more time communicating than computing. If support is unresponsive, you may wait hours to resolve a simple issue. In these cases, paying 20% more per hour for a reliable provider can actually reduce your total cost.
Also consider compliance and security. If you handle sensitive data, a marketplace with anonymous hosts may not meet your requirements. Hyperscalers and specialized clouds often provide certifications and private networking that marketplaces lack. The cheapest H100 rental for a hobby project is not the same as the cheapest for a production workload with regulatory constraints.
Final Thoughts
Finding the cheapest H100 rental is a balancing act between headline rate, billing granularity, reliability, and hidden costs. Start by understanding your workload, then compare providers across those dimensions. Don't chase the absolute lowest number without checking what you give up. With a bit of planning, you can cut your GPU bill significantly without sacrificing results.
Ready to see live prices? Check live H100 rental prices on Vast.ai and compare against other providers in our list.