Bare Metal vs VM vs Container on a DGX Spark — and Why “Bespoke” Is the Fourth Option
TL;DR: On a DGX Spark, the bare metal vs VM vs container debate is the wrong frame. One 128 GB unified memory pool and no BMC matter more than abstract hypervisor tax. Layers still matter for isolation — but the real choice is how much of the ops stack you build yourself.
This post is for anyone who owns, operates, or rents a DGX Spark and has to decide what runs directly on it: the solo researcher with one box, the team sharing it, and the operator letting strangers SSH into it.
SSH to the Spark is dead. nvidia-smi was fine an hour ago. The TensorRT-LLM build ate the box. There is no IPMI button, no Redfish power cycle, no Wake-on-LAN. NVIDIA’s own forum answer is blunt: DGX Spark Founders Edition does not support a BMC or out-of-band management. You are holding a smart plug, a car key, or a prayer.
That hang is a better introduction to this hardware than another “bare metal wins GPUs” chart.
In this post
- Bare metal vs VM vs container: the three layers
- What DGX Spark actually is: GB10, unified memory, no BMC
- GPU virtualization overhead by layer: VM, container, vGPU, MIG
- The homelab argument, applied to Spark
- The fourth option: a complete bespoke stack
- Decision guide: which layer for which DGX Spark workload
- Frequently asked questions
- Conclusion
- Glossary
Bare metal vs VM vs container: the three layers
TechMormo still teaches the ladder cleanly. NetActuate sells it as a purchasing guide. Same three nouns:
| Layer | What it is |
|---|---|
| Bare metal | OS and apps on the hardware. You own the machine. The blast radius is the machine. |
| Virtual machine | Hypervisor + guest kernel. Stronger boundary; snapshots; slower to boot. |
| Container | Shared host kernel; namespaces and cgroups. Fast, portable, weaker isolation. |
Figure: containers are not a third kind of computer — they run on bare metal or inside a VM, and on Spark every path shares the same 128 GB unified memory pool.
Homelab operators figured this out years ago: Proxmox on the metal, then VMs or LXC, then Docker where it fits. Purity is for slide decks.
NetActuate’s matrix still steers GPU AI/ML training to bare metal and quotes a 5–15% VM tax. Hold that number. We will come back to it with a lab that disagrees.
What DGX Spark actually is: GB10, unified memory, no BMC
DGX Spark is a GB10 Grace Blackwell superchip on a desk: 20-core Arm CPU, integrated Blackwell GPU (sm_121), 128 GB coherent LPDDR5x at roughly 273 GB/s (NVIDIA hardware overview). CPU and GPU share one pool via unified memory architecture (UMA). There is no discrete VRAM carve-out. nvidia-smi memory fields come back N/A — there is nothing separate to report (known issues).
Callout: UMA = one pool. A greedy vllm serve with --gpu-memory-utilization 0.9 can starve SSH the same way it starves the next CUDA malloc. Docker/Kubernetes memory limits do not fence GPU allocations on this topology — a NIM container has been observed occupying 120+ GB straight through its cgroup limits. MIG is not available. Watch free -h / MemAvailable, not NVML VRAM gauges borrowed from an H100 mental model.
Callout: no BMC. Enterprise manageability for Spark is in-band SSH tooling. Useful while the OS answers. Useless when it does not. Community recovery is a smart PDU plus BIOS “restore on AC,” an external IP-KVM you wired before you left town, or physical access with a USB stick.
Wrong mental model: dual-socket Xeon, discrete PCIe GPU, BMC on a dedicated NIC. Right mental model: a dense personal appliance where memory pressure and remote recovery are first-class design constraints. (For where GB10 sits in NVIDIA’s lineup, see our visual timeline of NVIDIA AI GPUs.)
GPU virtualization overhead by layer: VM, container, vGPU, MIG
Passthrough and containers are usually noise for GPU compute on discrete cards. Sharing is where you pay. Numbers below are lab ranges — mostly A100/H100/RTX PRO — not a Spark BM-vs-VM bake-off (that study is not in this research set).
| Layer | Typical Δ vs bare metal | Hardware | When it hurts | Source |
|---|---|---|---|---|
| VFIO / KVM passthrough (1 GPU : 1 VM) | 0–5% compute (often 0–2%; IBM Vela A100 training ~5%) | Discrete | Untuned virt NIC / topology; GPU path is fine | IBM Vela, Pextra |
| Docker + NVIDIA Container Toolkit | <1–3% steady-state | Discrete | Cold start, OverlayFS, driver mismatch | GigaGPU |
| Licensed NVIDIA vGPU (dedicated profile) | ~94–105% of BM (MLPerf-class H100/L40S) | Discrete | Dedicated ≠ time-sliced | VMware |
| Time-sliced vGPU under contention | ~5–25% (bandwidth-bound worse) | Discrete | Latency tails; memory BW first | Colfax |
| MIG partitions | ~3–8% vs full GPU | Discrete only | N/A on GB10 | NVIDIA MIG guide |
| Software GPU sharing (e.g. HAMi hooks) | ~9–19% microbench; often ~1–3% on tuned vLLM | Discrete | Hook tax ≠ production gap | GPU-Virt-Bench |
| Container-in-VM vs container-on-BM (vLLM + AIPerf) | ≈ equal or VM slightly better (qualitative) | RTX PRO 6000 | No published % | n0derunner |
| Native vs Docker on GB10 UMA | ~0% tok/s; +20–31 GB phantom RAM; 40–63% less KV cache | Spark | Capacity/accounting via cgroups — not FLOPS | benchmark-spark, Geraci |
On July 20, 2026, Nutanix’s n0derunner lab ran AIPerf against vLLM serving Llama 3.1 8B on an RTX PRO 6000 Blackwell. Same container, AHV VM vs bare-metal NKP. Result: same performance or the VM slightly ahead. No charts, no percentages — but enough to retire the slogan “GPU ⇒ must be bare metal” for tuned passthrough inference.
What does not transfer to Spark: discrete GDDR/HBM, x86 NUMA recipes, sm_100 datacenter wheels, MIG. Spark’s ceiling is LPDDR5x bandwidth and sm_121 software readiness — missing kernels can cost far more than a 2% passthrough tax. Do not quote n0derunner tok/s for GB10. Re-benchmark if the claim matters.
The Spark punchline in the last row: Docker can match native tokens/sec and still shrink usable KV cache because cgroups double-count unified memory. Same word “overhead,” different failure mode.
The homelab argument, applied to Spark
Ask a Proxmox Discord whether bare metal “wins” and you get a fork, not a winner. Snapshots argue for VMs. Density argues for LXC. Simplicity puts Docker on the host. Isolation puts Docker in a VM. Routers and low-RAM appliances stay bare metal because the abstraction is the problem.
Same instincts apply on Spark — with sharper teeth. Shared kernel means shared NVIDIA driver surface (CVE-2025-23266 was a CVSS 9.0 container-toolkit escape). One hung CUDA job is a host event. Guest restart helps CPU-side messes; it does not invent a second memory pool.
The fourth option: a complete bespoke stack
DIY on Spark is NVIDIA’s appliance story: DGX OS, drivers, Docker, Dashboard, Sync/Tailscale playbooks, USB recovery. Enough for a single owner who can walk to the desk.
A complete bespoke stack is everything you add when the box is not just yours — when someone else needs SSH to a GPU tenant over the public internet:
| Component | DIY appliance | Complete bespoke |
|---|---|---|
| Isolation | Docker --gpus on the host | LXD system containers (VM-like tenants); GPU VFIO VMs are not the Spark path NVIDIA supports |
| GPU policy | Owner sees the GPU | One active tenant per host on a single GB10 |
| Memory / OOM | Dashboard + free -h | Swap off or carefully tuned; earlyoom; headroom under --gpu-memory-utilization |
| Network / SSH | Tailscale, mDNS, manual tunnels | Relay, outbound WireGuard, per-instance DNS, L4 :22 mapping |
| Monitoring | htop, Dashboard | Host + tenant health; UMA-aware alerts (no VRAM gauges) |
| Image | NGC containers | Curated base: sshd, injected driver libs, smoke test |
| Recovery | USB + physical access | Same + smart PDU / IP-KVM; re-push relay maps after reset |
| Lifecycle | Manual | Create / start / stop / delete, queue, billing gates |
That table is the product, whether you build it or rent it. “Bespoke” here means ops completeness, not a brand adjective. It is also, roughly, what Enverge Cloud runs under each rented Spark.
Decision guide: which layer for which DGX Spark workload
| You are… | Lean toward… | Because… |
|---|---|---|
| Solo researcher on your own Spark | DIY BM + Docker / NGC | Shortest path; you own OOM and the smart plug |
| Many trusted services on one box | Containers on BM or LXD system containers | Blast-radius control without a full guest tax |
| Snapshots / mixed OS (non-GPU or CPU-side) | VM / LXC on a hypervisor (Proxmox pattern) | Rollback beats reinstall — not GB10 GPU passthrough tenants |
| Inference on discrete cloud GPUs | Tuned VM passthrough may ≈ BM | n0derunner + passthrough literature |
| Multi-node NCCL training | Bare metal or carefully tuned passthrough | Untuned virt networking hurts first |
| Renting Spark to strangers | Complete bespoke or sequential whole-GPU time-share | UMA + no MIG + no BMC + shared driver |
| Concurrent adversarial multi-tenant on one GB10 | Don’t | One tenant; drain; reboot or reload drivers between sessions |
| Bandwidth-bound LLM on Spark | Any layer — fix memory and wheels first | 273 GB/s and sm_121 readiness dominate layer choice |
Spine view of the same trade-offs:
| Axis | Bare metal DIY | Container on BM | LXD guest | Complete bespoke |
|---|---|---|---|---|
| Perf (inference) | Ceiling | Near-native; UMA cgroup risk | Near-native | Isolation quality, not magic FLOPS |
| Isolation | Host = blast radius | Shared kernel | Stronger user-space boundary | Tenant + platform ops |
| Memory (UMA) | You tune everything | cgroups help CPU side only | Same pool; one tenant | Pre-tuned floors, earlyoom |
| Recovery (no BMC) | You + smart plug | Same host fate | Guest restart ≠ host power | Runbooks + external power / KVM |
| Time to SSH | Days of setup | Hours if host ready | Hours–days | Minutes if rented; days if you DIY the platform |
| Who owns the hang | You | You | You (host) | Split: tenant vs operator |
Frequently asked questions
Does DGX Spark support virtualization or GPU passthrough to a VM?
Not officially. NVIDIA states that GPU virtualization is not supported on DGX Spark, and the GB10’s IOMMU firmware maps the GPU 1:1, so VFIO passthrough of the integrated GPU into a KVM guest does not work out of the box. CPU-only VMs run fine; GPU tenants on Spark are done with containers or LXD system containers instead.
Does DGX Spark have a BMC, IPMI, or Redfish?
No. DGX Spark Founders Edition has no baseboard management controller and no out-of-band management. NVIDIA’s enterprise manageability tooling works over in-band SSH only. If the OS hangs, recovery means a smart PDU with “restore on AC power,” an external IP-KVM, or physical access to the USB recovery port.
Does DGX Spark support MIG?
No. Multi-Instance GPU is a feature of discrete datacenter GPUs (A100, H100, B200). The GB10 integrated GPU cannot be hardware-partitioned, and licensed vGPU profiles are not available either. Sharing a Spark between users means time-slicing one whole GPU, one active tenant at a time.
Is Docker slower than bare metal on DGX Spark?
For throughput, no — community benchmarks show identical tokens per second native vs Docker. The cost is capacity: cgroups double-count unified memory, so a container can appear to use 20–31 GB more RAM and end up with 40–63% less KV cache than the same vLLM process on the host.
How much performance does a VM cost for GPU workloads?
On discrete GPUs with tuned VFIO/KVM passthrough, 0–5% — IBM’s Vela cluster measured ~5% on A100 training, and Nutanix’s n0derunner lab found a VM equal to or slightly faster than bare metal for vLLM inference. Losses grow with time-sliced vGPU (5–25%) or untuned virtual networking, not with the GPU path itself.
Conclusion
Bare metal, VMs, and containers remain complementary on a DGX Spark. Pick them for isolation and operability, not because a blog said GPUs hate hypervisors. Watch the unified pool. Plan the reboot without a BMC. Decide whether you are assembling that stack — or using one that already exists.
If you want a Spark without building the relay and OOM runbooks yourself, see DGX Spark pricing and the Enverge Cloud docs. If you are wiring your own box, leave headroom in the 128 GB, keep a smart plug on the PDU, and treat MemAvailable as gospel.
Glossary
| Abbr | Expansion |
|---|---|
| AIPerf | AIPerf — inference-endpoint benchmark used in the n0derunner study (vLLM path) |
| BM | Bare metal |
| BMC | Baseboard Management Controller |
| IPMI | Intelligent Platform Management Interface |
| IP-KVM | Network keyboard/video/mouse appliance (external console) |
| KVM | Kernel-based Virtual Machine (hypervisor) — not the same as IP-KVM |
| LXC / LXD | Linux containers / Canonical’s LXD (system containers and VMs) |
| MIG | Multi-Instance GPU |
| NCCL | NVIDIA Collective Communications Library |
| NGC | NVIDIA GPU Cloud (container registry / catalogs) |
| NKP | Nutanix Kubernetes Platform (n0derunner bare-metal path) |
| OOM | Out of memory |
| PDU | Power distribution unit |
| Redfish | DMTF (Distributed Management Task Force) hardware management API |
| UMA | Unified Memory Architecture |
| VFIO | Virtual Function I/O (PCI passthrough) |
| vGPU | NVIDIA virtual GPU (licensed sharing) |
| vLLM | High-throughput LLM inference engine |
| WoL | Wake-on-LAN |