Guides
GPU Cloud Pricing Comparison: A Practical Guide for AI Teams
A GPU cloud pricing comparison looks like a simple exercise: find the hourly rate, multiply by the hours you need, pick the cheapest. Anyone who has actually run training or inference in production knows it does not work that way. The same accelerator can be sold under a dozen names, billed in different increments, bundled with different amounts of CPU, RAM and storage, and priced very differently depending on whether you commit, whether you can be interrupted, and how much data you move. This guide walks through the parts that actually change the bill, and how to compare providers without getting fooled by the headline number.
Why a GPU cloud pricing comparison is harder than it looks
Start with the hardware. An A100 comes in 40GB and 80GB versions, in SXM and PCIe form factors, and with or without high-speed interconnect between cards. H100 has several memory and interconnect configurations too. Providers rarely make it easy to tell which one you are renting, and the price difference between variants is real. If you compare two listings that both say "A100" without checking memory and interconnect, you are not comparing the same thing.
Then there is billing granularity. Some providers bill per second, some per minute, some round up to the hour. On a short job that starts and stops often, that difference can matter more than the rate itself. Minimum runtimes, per-instance startup charges and idle time on a running instance all feed into the final number.
The billing models behind any GPU cloud pricing per hour comparison
Before comparing rates, decide which purchasing model you are comparing. They are not interchangeable.
| Model | How you pay | What to watch |
|---|---|---|
| On-demand | Per unit of time, no commitment | Billing rounding, idle time, availability |
| Spot / preemptible | Discounted rate, instance can be reclaimed | Checkpointing overhead, restart cost, wasted work |
| Reserved / committed | Discount in exchange for a term commitment | Forecast risk, lock-in, unused capacity |
| Serverless / per-token | Per request or per token processed | Cold starts, less control, harder to benchmark |
A per-hour rate on spot capacity and a per-hour rate on a reserved instance are different products. Comparing them directly is the most common mistake in a cloud GPU provider pricing comparison.
What a cloud GPU provider pricing comparison usually leaves out
The GPU rate is one line item. The others:
- Storage. Model weights, datasets and checkpoints are large. Fast storage is priced separately and often costs more than people expect.
- Egress. Moving data or responses out of a region or provider is usually charged. For inference serving external users, this can be a meaningful share of the bill.
- Idle time. A GPU that is allocated but waiting for data, a queue or a human is still billed.
- Orchestration and networking. Load balancers, NAT gateways, cross-zone traffic and managed Kubernetes control planes add up.
- Support and quota. Higher support tiers cost money, and capacity quotas can delay a launch even when the price looks fine.
The cheapest GPU is rarely the cheapest workload. Model the whole bill, not the rate.
Training vs inference: where the costs diverge
GPU cloud pricing comparison for training
Training jobs run for hours or days, often across many GPUs, and the interconnect between them matters as much as the chips. Cost is best measured per training run or per experiment, not per hour: a slower, cheaper GPU that takes twice as long is not cheaper. Utilization is usually high and predictable, which makes reserved capacity and spot with checkpointing both viable. Watch for storage throughput on data loading, and for the cost of restarts when preemptible instances get reclaimed.
GPU cloud pricing comparison for inference
Inference is spiky. Traffic arrives in bursts, and the GPU sits idle between them unless you scale down. That makes billing granularity and scale-to-zero behaviour more important than the raw hourly rate. Memory bandwidth and batching efficiency determine how many requests one GPU can serve, so the useful metric is cost per request or cost per million tokens, not cost per hour. A provider with a higher hourly rate but better batching can be cheaper overall.
Cloud GPU pricing comparison for LLM workloads
LLM work straddles both categories. Fine-tuning and continued pretraining behave like training: long jobs, high memory, checkpointing. Serving a model behaves like inference: latency-sensitive, bursty, and sensitive to how well the runtime batches requests. Model size and quantization decide which GPUs you can use at all, which narrows the comparison before price even enters the picture. If a model needs more VRAM than a single card offers, you are comparing multi-GPU configurations and interconnect, and the per-hour rate stops being the main variable.
GPU cloud pricing comparison: AWS, Azure and Google vs specialist clouds
The three large clouds sell GPUs as one product inside a much larger platform. You get mature identity, networking, storage and managed services, plus committed-use discount programmes and enterprise agreements. In exchange, pricing is more complex, capacity for the newest accelerators often requires quota requests, and the surrounding services are where a lot of the spend ends up.
Specialist GPU clouds sell mostly the GPU. Their catalogues are narrower and their pricing pages are shorter. Some bill per second, some offer simpler reservation terms, and many are quicker to make new hardware available. The trade-off is fewer managed services, smaller regions and less enterprise tooling, so you may end up building more yourself.
| Dimension | Large clouds | GPU specialists |
|---|---|---|
| Service breadth | Broad, integrated | Narrow, GPU-focused |
| Pricing complexity | Higher, many line items | Usually simpler |
| Discount structure | Commitments and enterprise agreements | Shorter terms, smaller commitments |
| Operational overhead | Lower if you already use the platform | Higher, but less to configure |
Neither column is automatically cheaper. It depends on how much of the surrounding platform you actually use.
GPU cloud pricing comparison for startups
Early-stage teams usually optimise for flexibility, not for the lowest possible rate. A few things help:
- Start on on-demand or spot. Do not sign a long reservation before you know your steady-state utilization.
- Use provider credits and startup programmes, but do not let them dictate your architecture.
- Keep a second provider configured. Capacity shortages are real, and being able to fail over is worth a small price premium.
- Watch storage and egress from day one. They are the costs that surprise teams at the end of the month.
- Benchmark your own workload on two or three instance types rather than trusting published comparisons.
Running your own cloud GPU rental price comparison
Published comparisons age quickly. The reliable approach is to build a small internal benchmark:
- Pick the two or three instance types that fit your memory and interconnect requirements.
- Run the same job on each and record wall-clock time, not just the hourly rate.
- Measure actual GPU utilization. If it is low, the problem is your pipeline, not the price.
- Add storage, egress and orchestration costs to get a real monthly figure.
- Test cold start and scale-down behaviour if you serve traffic.
- Check quota and availability before you commit, not after.
Do that once per workload and you will have a more accurate picture than any static table, including this one.