Guides
NVIDIA H100 Cloud Price per Hour: A Working Engineer's Review
I rent cloud GPUs every week — some weeks it's a single-GPU fine-tune, other weeks it's a multi-node run that eats a whole weekend. The question teams ask me most isn't which provider to pick; it's what the NVIDIA H100 cloud price per hour actually buys them, and whether a cheaper card would finish the job just as well. After a year of weekly rentals for both training and inference, here's my honest review: what the hourly rate gets you, where it's wasted money, and the billing details that decide whether you're paying for compute or for waste.
What You're Renting: The Hardware Behind the Hourly Rate
Start with the silicon, because not every H100 listing is the same chip. The SXM5 flavor carries 80GB of HBM3 with memory bandwidth north of 3 TB/s, 900 GB/s NVLink for multi-GPU runs, and a 700W power envelope. The PCIe version runs at a lower power limit with noticeably less memory bandwidth, and most cloud VMs don't give you NVLink on it at all. Both support FP8 with the Transformer Engine, which is where much of the throughput magic lives for transformer workloads, and both can be split with MIG into up to seven isolated slices. When I review a listing, the first thing I check is which variant I'm getting, because the hourly rate should reflect that gap — and sometimes it doesn't.
What the NVIDIA H100 Cloud Price per Hour Includes — and What It Doesn't
An hourly rate covers the GPU and usually a fixed slice of vCPUs and system RAM attached to it. Everything around the GPU is often metered separately, and this is where reviews of 'cheap' providers fall apart. Object storage for datasets and checkpoints, egress traffic when you move results off the box, snapshot storage, sometimes even the boot disk — all of it can show up on the final invoice. A rate that looks aggressive can lose its edge the moment you push two terabytes of checkpoints home. My rule: before comparing any nvidia h100 cloud gpu price per hour, I price the whole run — data in, data out, storage while idle — because the sticker rate is only part of the real story.
Where the Rate Pays Off — and Where It's Money Burned
This is the part of the review that matters most. For throughput-bound work — pretraining, full fine-tunes of dense models, large-batch evaluation — the H100 earns its rate. FP8 plus the big memory pool means fewer hours on the meter, and on multi-GPU nodes the NVLink fabric keeps gradient all-reduce from strangling your scaling. When I weigh an h100 cloud gpu for llm training cost against an older flagship, I do the math in tokens per dollar, not dollars per hour; a faster card at a higher rate frequently wins because the job simply ends sooner.
A reality check from last quarter
Last quarter we moved a 13B fine-tune from an eight-GPU A100 node to a four-GPU H100 node with FP8 enabled. Same dataset, same schedule, half the billable GPU-hours. That's the entire argument for the card in one sentence — and the same math is why we don't rent it for the 3B LoRA jobs that run fine on a 24GB card overnight.
Where the rental stops making sense is small-scale work. Low-traffic inference doesn't need 80GB of HBM3 sitting mostly idle, and a quantized LoRA run rarely does either. For a short nvidia h100 per hour rental to test a pipeline, fine — but don't park one on routine jobs out of habit.
How I Evaluate an NVIDIA H100 Cloud Price per Hour Before Booking
Any honest h100 gpu cloud pricing comparison comes down to rate structure more than the rate itself. Here's the grid I run through before I book.
| Rate type | What it usually means | Watch out for |
|---|---|---|
| On-demand | Pick a machine, pay while it runs, kill it when done | Idle billing between runs; minimum billing increments |
| Spot / interruptible | Discounted capacity the provider can reclaim | Checkpoint frequency; preemption with little warning on some platforms |
| Reserved / committed | Discount for promising weeks or months of usage | You pay even if the project slips; hard to resell |
| Bare metal | Whole physical node, no hypervisor neighbors | Pricier; overkill unless you need the full node |
One more filter: uptime history. A cheap GPU that's unavailable when you need it isn't cheap. And when a provider offers a trial or an hourly minimum, run your own micro-benchmark — a fixed prompt set for inference, a fixed step count for training. Two minutes of measurement beats any marketing page.
Billing Traps That Inflate Your H100 Cloud Rental Cost
Nearly every h100 cloud rental cost overrun I've seen traced back to one of these. First, idle time: the meter runs while you debug dataloaders, grab lunch, or leave a notebook open overnight. Second, orphaned storage: you terminate the instance but forget the 500GB volume and the snapshots, which bill forever. Third, egress: pulling a trained checkpoint out of one cloud and into another can cost more than an hour of GPU time. Fourth, rounding: some platforms bill per second, others round every session up to the next hour, which stings when you're spinning up machines to test things. And fifth, my favorite: auto-scaling groups that keep a spare GPU warm 'just in case.' Set a hard budget alert on day one.
Verdict: Who Should Rent One, and Who Shouldn't
After a year of weekly rentals, my verdict is straightforward. Rent an H100 when wall-clock time is the bottleneck: big fine-tunes, pretraining experiments, dense batch inference where you're paying for throughput anyway. Rent something smaller when the job fits — and be honest about what fits. Chasing the cheapest h100 cloud gpu listing is usually a mistake if it comes with thin uptime, no spot guarantees, or egress fees that bite on the way out; the cheapest listing is rarely the cheapest finished run. Judge every rental in tokens per dollar and finished experiments per week, not in the hourly number alone. On that measure, the H100 is still the card I reach for first when the clock matters — and the first one I shut off when it doesn't.