GPUs
Hardware for AI inference and training - VRAM, power, throughput, pricing, and availability.
| GPU | Vendor | VRAM | TDP | FP16 TFLOPs | INT8 TOPs | Price | Category | Status |
|---|---|---|---|---|---|---|---|---|
| H100 SXM5 | NVIDIA | 80GB | 700W | 1979 | 3958 | $30,000 | Data Center | Available |
| H200 SXM | NVIDIA | 141GB | 700W | 1979 | 3958 | $40,000 | Data Center | Available |
| A100 SXM4 80GB | NVIDIA | 80GB | 400W | 312 | 624 | $10,000 | Data Center | Available |
| RTX 4090 | NVIDIA | 24GB | 450W | 82.6 | 165 | $1,599 | Consumer | Available |
| RTX 5090 | NVIDIA | 32GB | 575W | 209 | 838 | $1,999 | Consumer | Available |
| MI300X | AMD | 192GB | 750W | 1307 | 2614 | $15,000 | Data Center | Available |
| Gaudi 3 | Intel | 128GB | 600W | 1835 | 3670 | $12,000 | Data Center | Available |
| TPU v5p | 96GB | 450W | 459 | 918 | Cloud | Cloud | Cloud Only | |
| Trainium2 | Amazon | 96GB | 700W | 380 | 760 | Cloud | Cloud | Cloud Only |
| RX 7900 XTX | AMD | 24GB | 355W | 61.4 | 123 | $999 | Consumer | Available |