Liquid-cooled PCIe GPU servers with up to 8 GPUs per node. From RTX 5090 to H200 NVL. Latest models: Llama 4, Qwen 3.5, Gemma 4, Mistral Small 4, DeepSeek R2. Every precision format. Real benchmarks.
All TFLOPS numbers shown as Dense / With Sparsity. Precision support verified from NVIDIA official datasheets. Each GPU available in 1x to 8x with Comino liquid cooling.
All numbers from NVIDIA official datasheets. TFLOPS shown as Dense / With Sparsity.
| Specification | RTX 5090 | RTX PRO 6000 | RTX 6000 Ada | L40S | H200 NVL | A100 PCIe |
|---|---|---|---|---|---|---|
| Architecture | Blackwell | Blackwell | Ada Lovelace | Ada Lovelace | Hopper | Ampere |
| Die | GB202 | GB202 | AD102 | AD102 | GH100 | GA100 |
| Tensor Core Gen | 5th | 5th | 4th | 4th | 4th | 3rd |
| CUDA Cores | 21,760 | 24,064 | 18,176 | 18,176 | 16,896 | 6,912 |
| Tensor Cores | 680 | 752 | 568 | 568 | 528 | 432 |
| VRAM | 32 GB GDDR7 | 96 GB GDDR7 | 48 GB GDDR6 | 48 GB GDDR6 | 141 GB HBM3e | 80 GB HBM2e |
| ECC Memory | No | Yes | Yes | Yes | Yes | Yes |
| Memory BW | 1,792 GB/s | 1,792 GB/s | 960 GB/s | 864 GB/s | 4,800 GB/s | 1,935 GB/s |
| TDP | 575W | 600W | 300W | 350W | 600W | 300W |
| PCIe Gen | 5.0 | 5.0 | 4.0 | 4.0 | 5.0 | 4.0 |
| NVLink | No | No | No | No | 900 GB/s | 600 GB/s |
| MIG | No | 4 inst | No | No | 7 inst | 7 inst |
| FP4 Native | NVFP4 | NVFP4 | — | — | — | — |
| FP6 Native | — | — | — | — | ||
| FP8 Native | 2nd gen | 2nd gen | 1st gen | 1st gen | 1st gen | — |
| FP32 TFLOPS | 104.8 | 125 | 91.1 | 91.6 | 60 | 19.5 |
| FP16/BF16 Tensor | 209 / 419 | ~500 / ~1,000 | 364 / 728 | 362 / 733 | 836 / 1,671 | 312 / 624 |
| FP8 Tensor | 419 / 838 | ~1,000 / ~2,000 | 729 / 1,457 | 733 / 1,466 | 1,570 / 3,341 | N/A |
| FP4 Tensor | ~1,676 / ~3,352 | ~2,000 / ~4,000 | N/A | N/A | N/A | N/A |
Real tok/s from Ollama, llama.cpp, vLLM, and TensorRT-LLM. Batch 1 = interactive, High batch = production API serving.
Measured with vLLM. Target <200ms for interactive UX.
StorageReview independently tested the Comino Grando with 8x NVIDIA RTX PRO 6000 GPUs (768 GB VRAM) in their lab. Real vLLM benchmarks, real thermal data, real noise measurements.
Measured by StorageReview on 8x RTX PRO 6000 Comino Grando. vLLM, batch size 256.
Measured by StorageReview. 230B parameter model, all 768 GB VRAM utilized. 8 users simultaneously coding with near-real-time response.
3x 140mm radiators, direct-to-chip liquid cooling on all 8 GPUs + CPU. 100% boost clock sustained under full load.
Select a GPU, choose count (1-8x), see which latest models (April 2026) fit at every quantization level.
| Model (April 2026) | Params | FP16/BF16 | FP8 | INT4 (AWQ) | GGUF Q4_K_M | GGUF Q8_0 | Status |
|---|---|---|---|---|---|---|---|
| Llama 4 ScoutMoE | 109B MoE Apr 2025 | 220 GB | 120 GB | 65 GB | 61 GB | 115 GB— | More VRAM |
| Llama 4 MaverickMoE | 400B MoE Apr 2025 | 800 GB | 420 GB— | 224 GB | 230 GB | 420 GB | — More VRAM |
| Llama 3.3 70B | 70B Dec 2024 | 148 GB— | 80 GB— | 42 GB— | 43 GB— | 75 GB— | — More VRAM |
| Llama 3.1 405B | 405B Jul 2024 | 810 GB | 486 GB | 243 GB | 250 GB | 430 GB | — More VRAM |
| Qwen 3.5 397BNEWMoE | 397B MoE Feb 2026 | 794 GB | 420 GB— | 230 GB— | 260 GB— | 420 GB— | — More VRAM |
| Qwen 3.5 32BNEW | 32B Feb 2026 | 68 GB— | 36 GB | 20 GB | 20 GB | 36 GB | Q4_K_M |
| Qwen 3 8B | 8B Apr 2025 | 18 GB | 10 GB | 6 GB | 5.5 GB | 9.5 GB | FP16 Full |
| Qwen 3 72B | 72B Apr 2025 | 152 GB | 80 GB | 44 GB | 45 GB | 78 GB | More VRAM |
| Gemma 4 31BNEW | 31B Apr 2026 | 66 GB | 35 GB | 19 GB | 20 GB | 35 GB | Q4_K_M |
| Gemma 4 26B MoENEWMoE | 26B MoE Apr 2026 | 55 GB | 30 GB | 16 GB “ | 17 GB “ | 28 GB | “ FP8 |
| Gemma 3 27B | 27B Mar 2025 | 58 GB | 32 GB | 17 GB | 17 GB | 30 GB | “ FP8 |
| Mistral Small 4NEWMoE | 119B MoE Mar 2026 | 238 GB | 130 GB | 70 GB | 78 GB | 130 GB | More VRAM |
| Mistral Large 3MoE | 675B MoE Dec 2025 | 1350 GB | 700 GB | 390 GB | 400 GB | 700 GB | More VRAM |
| DeepSeek R2NEWMoE | ~1.2T MoE Mar 2026 | 2400 GB | 1250 GB | 700 GB | 700 GB | 1250 GB | More VRAM |
| DeepSeek V3.2MoE | 685B MoE Late 2025 | 1370 GB | 700 GB | 400 GB | 400 GB | 710 GB | More VRAM |
| Phi-4 Reasoning 15BNEW | 15B Mar 2026 | 32 GB | 18 GB “ | 10 GB “ | 10 GB “ | 17 GB “ | “ FP16 Full |
| Command A | 111B Mar 2025 | 222 GB | 120 GB | 64 GB | 66 GB | 118 GB | More VRAM |
| Mixtral 8x22BMoE | 176B MoE Apr 2024 | 264 GB | 180 GB | 66 GB | 80 GB | 185 GB | More VRAM |
Quantization compresses model weights from 16-bit to 4-bit. Blackwell GPUs (RTX 5090, PRO 6000) unlock native FP4 – a precision unavailable on Ada/Hopper/Ampere.
| Precision | RTX 5090 | PRO 6000 | 6000 Ada | L40S | H200 NVL | A100 |
|---|---|---|---|---|---|---|
| FP4 (NVFP4) | ✓ Native | ✓ Native | ✗ | ✗ | ✗ | ✗ |
| FP6 | ✓ Native | ✓ Native | ✗ | ✗ | ✗ | ✗ |
| FP8 | ✓ 2nd gen | ✓ 2nd gen | ✓ 1st gen | ✓ 1st gen | ✓ 1st gen | ✗ |
| FP16 / BF16 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| INT8 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| INT4 | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ |
| GPTQ / AWQ (software) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| GGUF (llama.cpp) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Post-training calibration-based quantization. Groups weights into blocks.
Activation-Aware. Protects important weights. Best INT4 quality.
Universal format. CPU offloading. Q2_K through Q8_0. Runs on any hardware.
Variable bits per layer. Fastest interactive inference on GPU.
NVIDIA native FP4 via TensorRT-LLM. Hardware-accelerated on 5th gen Tensor Cores.
Zero calibration data needed. Quantize in minutes.
Self-hosted OpenAI-compatible APIs with vLLM or TGI. 40-200x lower cost than cloud APIs at 30M+ tokens/day.
QLoRA makes 70B fine-tuning accessible on a single GPU. PCIe Gen5 bandwidth is adequate.
Embedding + reranker + LLM on one server. Vector DB on CPU/RAM.
SDXL, Flux, video models. L40S has 3x NVENC hardware encoders.
Whisper + LLM + TTS on a single GPU.
Self-hosted Copilot with Qwen 3.5, Mistral Small 4, DeepSeek Coder. 8 concurrent Claude Code sessions on 230B model (StorageReview verified).
Chat + code + image + speech simultaneously. One 8-GPU server replaces 5+ cloud subscriptions.
5,000-10,000 docs/sec per L40S with BGE-large.
8x 600W GPUs = 4,800W in one server. Air cooling can't handle it. Comino direct-to-chip liquid cooling (6.5 kW capacity) keeps 100% boost clock 24/7.
Air-cooled: 60-80% sustained †’ Comino liquid: 100% = 25-40% more actual throughput from the same GPUs.
| Scenario | Best Config | VRAM | Why |
|---|---|---|---|
| Startup, 7-32B | 2x RTX 5090 | 64 GB | Best price/perf, native FP4, ~150 tok/s per GPU |
| 70B comfortably | 2x PRO 6000 | 192 GB | 70B FP16 + room for context. ECC, MIG |
| Dense rack inference | 4-8x L40S | 192-384 GB | Passive cooling, data center grade |
| Max single-GPU | 1x H200 NVL | 141 GB | 70B FP16 on 1 card, 4.8 TB/s, 21ms TTFT (8B) |
| DeepSeek R2 / 405B | 6-8x H200 NVL | 846-1,128 GB | ~400 tok/s on 405B (TRT-LLM FP8) |
| QLoRA fine-tuning 70B | 1x A100 80GB | 80 GB | Proven, 300W, MIG for multi-tenant |
| Full AI stack | 8x PRO 6000 | 768 GB | Chat + Code + Image + Speech. 12,109 tok/s on 8B (StorageReview verified) |
| Power-constrained | 8x 6000 Ada | 384 GB | Only 2,400W total. Lowest W/GB ratio |
| 8-dev code team (230B) | 8x PRO 6000 | 768 GB | 38.7 tok/s per session, 8 concurrent (StorageReview) |
MoE models load ALL expert weights into VRAM. Only active params compute per token → fast inference, but huge VRAM footprint.
Mistral Small 4: 119B total / 6.5B active → ~70 GB INT4 | Llama 4 Scout: 109B / 17B active → ~65 GB INT4 | Qwen 3.5 397B: 397B / 17B active → ~230 GB Q4 | DeepSeek R2: ~1.2T / ~78B active → ~700 GB Q4
Tell us your workload, models, and budget. Our engineers design the optimal Grando configuration – GPU, quantization strategy, and cooling included. Up to 8 GPUs per server, verified by StorageReview.