I’ve been renting GPUs for training runs since the A100 days, and I’ve probably burned through $40k in cloud compute across Lambda, RunPod, Vast, and even a sketchy dedicated box in Finland. If you’re trying to decide between H200 and H100 in 2026, h

The price gap is real, but not where you think

Let’s cut to the numbers. In early 2026, I’m seeing H100 SXM (80GB) rental prices hover between $1.85 and $2.40 per hour on spot-ish providers, and $2.60–$3.10/hr for guaranteed on-demand. H200 SXM (141GB) is running $2.90–$3.80/hr on the same platforms. That’s roughly a 40–55% premium for H200, not the 2x gap I saw in late 2024. On Vast.ai specifically, I grabbed an H200 for $2.75/hr last week (a 4x A100 80GB node, 3.5TB NVMe), while the cheapest H100 on the same marketplace was $1.95. A100 80GB? You can still find those at $1.10–$1.40/hr, which makes them the budget king for fine-tuning, not pre-training.

But here’s the trap: the hourly rate isn’t the real cost. H200’s 141GB memory vs H100’s 80GB means you can often skip tensor parallelism entirely for a 70B model. I ran a 70B LoRA on a single H200 with batch size 8, whereas the same run on H100 required 2 nodes (16 GPUs) or heavy offloading. My actual cost per epoch: H200 = $11.20 (4 hours at $2.80), H100 = $19.50 (5 hours at $1.95 but with 2x slower per-step due to memory swapping). So the H200 won on total cost despite the higher hourly rate.

What actually changed in 2026: memory bandwidth and idle pricing

The biggest surprise wasn’t the raw price – it was the idle-time discounts. On Vast and RunPod, I’ve seen H200s drop to $1.90/hr during US off-peak (midnight to 8am PST), which is cheaper than some H100s during daytime. H100s, being older, have more stable pricing but less elasticity. If you can schedule jobs at night, H200 is a no-brainer. If you need 24/7 availability, the H100’s lower baseline wins.

Another factor: H200’s HBM3e runs at 4.8TB/s vs H100’s 3.35TB/s. For my use case (long-context LLM inference, 128k tokens), that’s a 30–35% speedup on decode tokens. I benchmarked a 7B model with vLLM – H200 served 2,900 tokens/s vs H100’s 2,150 tokens/s on identical settings. At the same price per token, H200 is cheaper per million tokens. But for batch training with short sequences (<2k tokens), the difference shrinks to 10–15%, and then the H100’s lower hourly rate wins.

• H100 best for: 24/7 batch training, multi-node jobs, budget-conscious fine-tuning
• H200 best for: single-node 70B+ inference, long-context, overnight jobs with spot pricing
• A100 best for: prototyping, small models, or when you need 80GB but don’t care about speed

L40S is the wildcard nobody talks about

I see L40S (48GB) renting at $0.85–$1.20/hr, and it’s criminally underrated for inference. It has the same Ada Lovelace architecture as the RTX 6000, but with 48GB and FP8 support. For a 32B model quantized to 4-bit, I get 1,400 tokens/s on L40S – that’s 65% of H100’s speed at 40% of the cost. The catch? No NVLink, so multi-GPU scaling is a pain. But for single-node serving or batch jobs that fit in 48GB, L40S is my go-to. I ran a 3-day RLHF pipeline on 4x L40S for $280 total, versus $620 on 4x H100 for the same job (just slower wall-clock, but I didn’t care).

For 2026, my rule of thumb: if you need >100GB memory, go H200. If you need <50GB and can tolerate slower interconnects, L40S. Only pick H100 when you need NVLink or the price drops below $1.70/hr. A100? Only for legacy code or if you find a steal under $1.00/hr.

How to actually save money on rentals (my 2026 playbook)

I’m not a sponsored shill, but I’ve learned these tricks the hard way:

1. Bid on spot, but with a hard stop. On Vast, I set a max bid of $2.20/hr for H200. I win about 60% of the time, and the average winning price is $1.95. If I lose, I just wait 20 minutes – prices cycle.

2. Check the “interruptible” flag. Some H100s on Vast are listed at $1.50/hr but can be killed at any moment. For checkpointed training, that’s fine. For a 2-hour inference benchmark, don’t risk it.

3. Use the memory-to-price ratio. Calculate $/GB/hour. H200 at $2.80/141GB = $0.0199/GB/hr. H100 at $2.00/80GB = $0.025/GB/hr. L40S at $1.00/48GB = $0.0208/GB/hr. The H200 wins on this metric almost always.

4. Don’t ignore CPU and RAM specs. I rented an “H200” for $2.60/hr that turned out to have a 16-core Xeon and 128GB RAM – the data loader bottlenecked everything. Now I filter for at least 32 cores and 256GB RAM.

One more thing: always check the GPU age. I’ve seen H100s listed that are actually H800s (China export version) with reduced NVLink. That’s a 20% performance hit. Ask the provider for nvidia-smi -q | grep "Product Name" before you pay.

My final verdict for 2026

If you’re a solo dev or small team, rent H200 for anything that touches 70B models or long context. The 141GB memory saves you so much engineering time that the hourly premium pays for itself. For high-throughput, short-sequence fine-tuning, stick with H100 if you can get it under $2.10/hr. And for pure inference on 7B–13B models, L40S is the quiet value king.

I’ve been using Vast.ai more and more because their pricing is transparent – you see the exact GPU, CPU, RAM, and disk before you rent, and the spot market is genuinely cheaper than the big clouds. I’m not affiliated, but I’ll leave my referral link here because it gives me $10 in credits and you get $10 too: Vast.ai. That’s basically 4 hours of H100 time for free. Try the H200 spot filter, set a bid of $2.00, and see what you get. Your wallet will thank you.