Know the model you want to run? Start there and we size the hardware from its published architecture. Or browse the GPUs: liquid-cooled PCIe servers and workstations with up to 8 GPUs per node, built around the RTX PRO 6000 Blackwell, with independently measured throughput.
Pick what you want to run, the context you will actually use and how many people or agents hit it at once. We size the memory from the model's published architecture and show the Grando systems that fit, with measured speeds where we have them.
Speeds are reference figures from independent reviews of one Comino system. The theoretical maximum differs and is not shown until we measure it.
All TFLOPS numbers shown as Dense / With Sparsity. Precision support verified from NVIDIA official datasheets. Each GPU available in 1x to 8x with Comino liquid cooling.
All numbers from NVIDIA official datasheets. TFLOPS shown as Dense / With Sparsity.
| Specification | RTX 5090 | RTX PRO 6000 | RTX 6000 Ada | L40S | H200 NVL | A100 PCIe |
|---|---|---|---|---|---|---|
| Architecture | Blackwell | Blackwell | Ada Lovelace | Ada Lovelace | Hopper | Ampere |
| Die | GB202 | GB202 | AD102 | AD102 | GH100 | GA100 |
| Tensor Core Gen | 5th | 5th | 4th | 4th | 4th | 3rd |
| CUDA Cores | 21,760 | 24,064 | 18,176 | 18,176 | 16,896 | 6,912 |
| Tensor Cores | 680 | 752 | 568 | 568 | 528 | 432 |
| VRAM | 32 GB GDDR7 | 96 GB GDDR7 | 48 GB GDDR6 | 48 GB GDDR6 | 141 GB HBM3e | 80 GB HBM2e |
| ECC Memory | No | Yes | Yes | Yes | Yes | Yes |
| Memory BW | 1,792 GB/s | 1,792 GB/s | 960 GB/s | 864 GB/s | 4,800 GB/s | 1,935 GB/s |
| TDP | 575W | 600W | 300W | 350W | 600W | 300W |
| PCIe Gen | 5.0 | 5.0 | 4.0 | 4.0 | 5.0 | 4.0 |
| NVLink | No | No | No | No | 900 GB/s | 600 GB/s |
| MIG | No | 4 inst | No | No | 7 inst | 7 inst |
| FP4 Native | NVFP4 | NVFP4 | — | — | — | — |
| FP6 Native | — | — | — | — | ||
| FP8 Native | 2nd gen | 2nd gen | 1st gen | 1st gen | 1st gen | — |
| FP32 TFLOPS | 104.8 | 125 | 91.1 | 91.6 | 60 | 19.5 |
| FP16/BF16 Tensor | 209 / 419 | ~500 / ~1,000 | 364 / 728 | 362 / 733 | 836 / 1,671 | 312 / 624 |
| FP8 Tensor | 419 / 838 | ~1,000 / ~2,000 | 729 / 1,457 | 733 / 1,466 | 1,671 / 3,341 | N/A |
| FP4 Tensor | ~1,676 / ~3,352 | ~2,000 / ~4,000 | N/A | N/A | N/A | N/A |
Real tok/s from Ollama, llama.cpp, vLLM, and TensorRT-LLM. Batch 1 = interactive, High batch = production API serving.
Measured with vLLM. Target <200ms for interactive UX.
StorageReview independently tested the Comino Grando with 8x NVIDIA RTX PRO 6000 GPUs (768 GB VRAM) in their lab. Real vLLM benchmarks, real thermal data, real noise measurements.
Measured by StorageReview on 8x RTX PRO 6000 Comino Grando. vLLM, batch size 256.
Measured by StorageReview. 230B parameter model, all 768 GB VRAM utilized. 8 users simultaneously coding with near-real-time response.
3x 140mm radiators, direct-to-chip liquid cooling on all 8 GPUs + CPU. 100% boost clock sustained under full load.
Select a GPU and count. Same calculation as the module above: published weights + KV cache for an 8k context, one user. Green = fits with headroom, amber = tight.
| Model | Params | BF16 | FP8 | 4-bit (NVFP4 / native) | Status |
|---|
Quantization compresses model weights from 16-bit to 4-bit. Blackwell GPUs (RTX 5090, PRO 6000) unlock native FP4 – a precision unavailable on Ada/Hopper/Ampere.
| Precision | RTX 5090 | PRO 6000 | 6000 Ada | L40S | H200 NVL | A100 |
|---|---|---|---|---|---|---|
| FP4 (NVFP4) | ✓ Native | ✓ Native | ✗ | ✗ | ✗ | ✗ |
| FP6 | ✓ Native | ✓ Native | ✗ | ✗ | ✗ | ✗ |
| FP8 | ✓ 2nd gen | ✓ 2nd gen | ✓ 1st gen | ✓ 1st gen | ✓ 1st gen | ✗ |
| FP16 / BF16 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| INT8 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| INT4 | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ |
| GPTQ / AWQ (software) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| GGUF (llama.cpp) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Post-training calibration-based quantization. Groups weights into blocks.
Activation-Aware. Protects important weights. Best INT4 quality.
Universal format. CPU offloading. Q2_K through Q8_0. Runs on any hardware.
Variable bits per layer. Fastest interactive inference on GPU.
NVIDIA native FP4 via TensorRT-LLM. Hardware-accelerated on 5th gen Tensor Cores.
Zero calibration data needed. Quantize in minutes.
Self-hosted OpenAI-compatible APIs with vLLM or TGI. 40-200x lower cost than cloud APIs at 30M+ tokens/day.
QLoRA makes 70B fine-tuning accessible on a single GPU. PCIe Gen5 bandwidth is adequate.
Embedding + reranker + LLM on one server. Vector DB on CPU/RAM.
SDXL, Flux, video models. L40S has 3x NVENC hardware encoders.
Whisper + LLM + TTS on a single GPU.
Self-hosted Copilot with Qwen 3.5, Mistral Small 4, DeepSeek Coder. 8 concurrent Claude Code sessions on 230B model (StorageReview verified).
Chat + code + image + speech simultaneously. One 8-GPU server replaces 5+ cloud subscriptions.
5,000-10,000 docs/sec per L40S with BGE-large.
8x 600W GPUs = 4,800W in one server. Air cooling can't handle it. Comino direct-to-chip liquid cooling (6.5 kW capacity) keeps 100% boost clock 24/7.
Air-cooled: 60-80% sustained → Comino liquid: 100% = 25-40% more actual throughput from the same GPUs.
| Scenario | Best Config | VRAM | Why |
|---|---|---|---|
| Startup, 7-32B | 2x RTX 5090 | 64 GB | Best price/perf, native FP4, ~150 tok/s per GPU |
| 70B comfortably | 2x PRO 6000 | 192 GB | 70B FP16 + room for context. ECC, MIG |
| Dense rack inference | 4-8x L40S | 192-384 GB | Passive cooling, data center grade |
| Max single-GPU | 1x H200 NVL | 141 GB | 70B FP16 on 1 card, 4.8 TB/s, 21ms TTFT (8B) |
| DeepSeek R2 / 405B | 6-8x H200 NVL | 846-1,128 GB | ~400 tok/s on 405B (TRT-LLM FP8) |
| QLoRA fine-tuning 70B | 1x A100 80GB | 80 GB | Proven, 300W, MIG for multi-tenant |
| Full AI stack | 8x PRO 6000 | 768 GB | Chat + Code + Image + Speech. 12,109 tok/s on 8B (StorageReview verified) |
| Power-constrained | 8x 6000 Ada | 384 GB | Only 2,400W total. Lowest W/GB ratio |
| 8-dev code team (230B) | 8x PRO 6000 | 768 GB | 38.7 tok/s per session, 8 concurrent (StorageReview) |
MoE models load ALL expert weights into VRAM. Only active params compute per token → fast inference, but huge VRAM footprint.
Mistral Small 4: 119B total / 6.5B active → ~70 GB INT4 | Llama 4 Scout: 109B / 17B active → ~65 GB INT4 | Qwen 3.5 397B: 397B / 17B active → ~230 GB Q4 | DeepSeek R2: ~1.2T / ~78B active → ~700 GB Q4
Tell us your workload, models, and budget. Our engineers design the optimal Grando configuration – GPU, quantization strategy, and cooling included. Up to 8 GPUs per server, verified by StorageReview.