Blackwell Bench

Snapshot

Sources: Thunder Compute, GPU Poet, NVIDIA RTX PRO Blackwell Architecture, Spheron B300 Guide, Tech Insider Blackwell Pricing, HMC RTX 5090 specs, StorageReview.

Workload advisor

Pick the constraints that actually bind — PHI, model class, and the circuit on the wall. The pick is the same decision tree as the notes below, not a marketing ranker.

Full spec comparison

Click a header to sort. Toggle column groups — the table is wide on purpose.

RTX PRO 6000 Workstation pricing reflects the Aug 2026 market ($13,998 Newegg – $16,000 NVIDIA Marketplace, midpoint $15,000; 87% above the $8,565 launch). Max-Q / Server prices carried from July — the Aug 12 hike applied to the Workstation Edition only. RTX 5090 street pricing normalized to ~$2,400 (FE MSRP $1,999). HGX B300 uses the mid-point of documented per-GPU costs ($37.5K–$53K) from a full 8-GPU HGX system ($300K–$500K). FP4 dense for RTX Pro 6000 and RTX 5090 derived from NVIDIA whitepaper (peak FP4 with FP32 accumulate); FP4 sparse doubles that. B300 FP4 dense = 15 PFLOPS from NVIDIA / Spheron reference.

MoE inference sensitivity

Mixtral-style 141B MoE (8 experts × 22B, 2 active → ~39B active params). Batch 1, prompt 512, decode 1024. Memory-bandwidth-bound decode plus compute-bound prefill, derated for confidential computing. Engineering estimates — validate on vLLM or TensorRT-LLM before you buy.

Model footprint = params × bytes/param + ~15% KV/activation reserve. Fits-on-device requires the full weight set resident in unified GPU memory (no CPU offload). For "does not fit", the system must page from host RAM through PCIe — throughput is capped by PCIe 5.0 bandwidth (~64 GB/s bidirectional) which crushes decoding speed by 10–50×, so those rows are shown as OOM rather than a synthetic number. Confidential Computing modes: Off = normal driver; CC-On (GPU-only) = Hopper/Blackwell CC with encrypted PCIe bounce buffers (~3–5% on Blackwell RTX Pro 6000); TEE-I/O = B300 with SR-IOV TEE-I/O extensions (~1–3% derate).

Workstation ECC ROI

Where workstation ECC and Blackwell pay off

1. ECC on GDDR7 is a real reliability lever for long-running local inference.

   The RTX 5090 has no ECC on its 32 GB GDDR7; the Pro 6000 and B300 both do. For a single 30-day fine-tuning run at 96 GB working set, uncorrected soft errors materially degrade training convergence and produce silent inference drift — the classic argument for the workstation tier over the consumer tier.

2. FP4 is the Blackwell moat.

   Neither H100 nor RTX 40-series supports native FP4 tensor ops. On Pro 6000 you get ~2.0 PFLOPS dense FP4 (4.0 PFLOPS sparse); on B300 you get ~15 PFLOPS dense — a 7.5× uplift per GPU. This is the single biggest reason to buy Blackwell today for local LLM inference.

3. Confidential Computing changes the calculus for PHI workloads.

   RTX Pro 6000 supports CC ("Confidential Compute: Supported" in the datasheet) with attested boot and encrypted CPU↔GPU DMA. That is the difference between a HIPAA-defensible local inference workstation and a compliance liability, and it is not available on the RTX 5090.

4. 2× RTX Pro 6000 is the sweet spot for 70B–120B local inference.

   192 GB combined VRAM fits Llama-3 70B at FP16 or a 141B MoE at FP4/INT4 with room for a 32K–64K KV cache. Full price tag ≈ $30K (either Max-Q or full-power) vs. ~$400K for an 8-GPU B300 chassis — roughly 13× cheaper for the tier of models most psychiatry / clinical NLP workloads actually run. The cheapest confirmed-purchasable Max-Q lane (Aug 30 check) is the PNY variant at $14,499.99/card, ≈ $29K per pair.

5. 4× RTX 5090 ($9,600) is a compelling FP4 workhorse — but only for non-PHI research.

   128 GB combined VRAM fits the 141B MoE at FP4, and per $1K spent it is the second-cheapest platform in this comparison. But no ECC, no MIG, no Confidential Compute, no NVLink, and gaming-tier driver / BIOS lifecycle make it disqualifying for any clinical workload touching PHI. Treat it as an FP4 dev-box, not production infrastructure.

6. Max-Q wins the 2-GPU head-to-head against full-power Pro 6000 — on power, not price.

   Same 192 GB VRAM, same 1.79 TB/s bandwidth, identical modeled decode throughput (memory-bandwidth-bound), 2× Max-Q draws 900 W peak / 3,071 BTU/hr vs 1,500 W / 5,118 BTU/hr for the full-power pair — a 40% power reduction and 40% cooling reduction. Street price has converged: NVIDIA raised the Max-Q Marketplace listing to $16,000 the same week as the Workstation Edition (Aug 10–12, 2026), so a pair of either edition now runs ≈ $30,000. The Max-Q case rests on power, thermals, and 4-GPU workstation feasibility, not price. The bulk OEM discount lane (SKU 900-5G153-2200-000-01, briefly ≈ $11,900) closed by Aug 30 — re-priced to $14,299.99 and marked Unavailable Online at Central Computer; the purchasable floor is now PNY at $14,499.99.

7. Max-Q also unlocks 4-GPU dense workstations.

   A 4× Max-Q build stays at 1,600 W peak, single 2000 W PSU, no server chassis, no colo, no NVLink. Same ECC, same 96 GB, same 4,000 AI TOPS FP4 sparse per card as the full-power variant.

8. 3× Max-Q is physically viable but rarely improves throughput.

   288 GB combined VRAM (matches a single B300), 1,250 W peak (fits a 20 A / 240 V circuit), $45,000 street at $15K/card midpoint. But tensor parallelism requires GPU count to divide the model's attention-head count — 32, 64, or 128 heads do not divide evenly by 3. Real deployments either (a) run TP=2 with the third GPU idle, (b) use pipeline parallelism at the cost of latency, or (c) reserve the third GPU for KV cache / expert offload / a second concurrent model. Sensitivity throughput here is modeled at TP=2 utilization (identical decode tok/s to 2× Max-Q), which is the honest answer. Choose 3-way only if you have a specific parallelism plan; otherwise jump straight from 2× to 4×.

9. The B300 is only rational when you need >300 B params or NVLink-5 collective ops.

   Below that threshold, you are paying a ~15× premium per token for interconnect bandwidth you will not use for single-user or small-batch inference. Above it, nothing else in the market fits.

10. Power and cooling flip the ROI calculus at the 4-GPU tier.

    A 4× RTX Pro 6000 Max-Q build peaks at ~1.6 kW (5,459 BTU/hr) — well within a single 2000 W PSU and a normal office HVAC envelope (roughly the heat of a 1.5-ton mini-split at part load). A 4× RTX 5090 build peaks at ~2.7 kW (9,213 BTU/hr) and needs a redundant PSU pair plus a dedicated 20 A/240 V circuit; a 1× B300 peaks at ~2.6 kW (8,872 BTU/hr) with the same demands. The 8× B300 (42,309 BTU/hr, ~12.4 kW) requires rack-scale liquid cooling and 3-phase 208 V — not a workstation deployment. This is why Max-Q is the actual winning form factor for on-prem multi-GPU clinical inference.

Methodology and assumptions

• Model

  141B-total-parameter MoE, top-2 routing, ~39B active params per token. This matches Mixtral 8×22B; substitute DeepSeek-V3 (671B, 37B active) or Llama-4-Maverick (400B, 17B active) as needed by rescaling the model-footprint column linearly with total params × bytes/param.

• Decoding phase

  Memory-bandwidth-bound: tok/s ≈ (bandwidth × utilization) / (active_params × bytes/param). Utilization: 70% for a single GPU, 55% for tensor-parallel 2-way, 45% for 4-way (NVLink absent on Pro 6000; PCIe P2P only), 65% for B300 with NVLink 5. Confidential-compute derates applied multiplicatively.

• Prefill phase

  Compute-bound; not shown separately because a 512-prompt / 1024-decode workload is dominated by decoding.

• KV cache

  Assumed FP8 across all rows (~4 KB per token per layer for a 141B MoE with GQA). This is a favorable assumption; real deployments running FP16 KV will see ~1.5× the memory footprint.

• Pricing

  Street/reseller pricing as of August 2026 — Pro 6000 midpoint $15,000 (Newegg $13,998 – NVIDIA Marketplace $16,000); RTX 5090 AIB midpoint $2,400; B300 per-GPU midpoint from full-system list price. Max-Q bulk OEM lane closed Aug 30.

• What this dashboard is not

  A substitute for a real benchmark. Before spending >$20K, replicate these numbers with vLLM or TensorRT-LLM on the exact quantization scheme and KV-cache dtype you intend to ship.

Data sources cited inline. Specifications verified against NVIDIA product datasheets and PNY / BIZON / Central Computers listings. Decode throughput is an engineering model, not a substitute for a vLLM or TensorRT-LLM bench on the exact quant you will ship.