2× DGX Spark cluster#
Note
Already SSH’d in?
After login, the cluster banner points to /etc/enverge/cluster.md on your instance: rail IPs, worker SSH, and a sample torchrun command with this VM’s addresses filled in (you still bring your own script). This page is the generic reference, that file is instance-specific.
A 2× DGX Spark instance is a two-node GPU cluster behind a single SSH endpoint. You land on node0 (the head). A second Spark (node1, the worker) is attached over a ~200 Gb/s RoCE fabric. You do not SSH to the worker directly, you launch a distributed job from node0 and it uses both GPUs.
Warning
nvidia-smi shows one GPU
nvidia-smi on node0 reports one GB10 (~128 GB unified). The second GPU lives on the worker, reachable only through a sharded multi-node job (tensor or pipeline parallel, FSDP, DeepSpeed, vLLM multi-node, Ray). A plain python train.py uses only node0 and leaves the worker idle.
Memory model#
The tier advertises 256 GB unified memory, which is aggregate across two 128 GB pools joined by the fabric, not one flat address space. No single process or tensor can exceed ~128 GB. Use model parallelism or sharding to spread work across both nodes.
Pre-configured on your instance#
NCCL over RoCE, both ConnectX-7 rails are wired for RDMA (
/etc/nccl.conf). NCCL auto-discovers the fabric, no manualNCCL_IB_*exports needed.VLLM_HOST_IP, preset to this node’s rail IP (from
/etc/profile.d/enverge-cluster.sh) so vLLM’s multi-node control plane advertises the correct address instead of the default-route interface.Worker SSH, passwordless SSH from this container to the worker’s container over the rail is already set up for launchers that spawn remote ranks. Check
/etc/enverge/cluster.mdon the instance for this VM’s rail IPs and worker address.
Running a multi-node job#
Launch rank 0 here and rank 1 on the worker over the rail. Any NCCL-based stack works once the workload is sharded. The shape is:
torchrun --nnodes=2 --nproc_per_node=1 --node_rank=0 \
--rdzv_endpoint=<node0-rail-ip>:29400 your_job.py
your_job.py is your distributed script, nothing is pre-installed to run for you. Set --node_rank=1 on the worker, or let Ray or mpirun spawn the remote rank via the pre-configured rail SSH. Replace <node0-rail-ip> with this instance’s rail-2 address (see /etc/enverge/cluster.md for the filled-in template, or ip -4 addr show enP2p1s0f0np0).
Unified memory on GB10#
Warning
Cap GPU memory utilization
On GB10, “GPU memory” is unified with system RAM. Settings that grab all GPU memory, for example vLLM gpu_memory_utilization (default 0.9), can reach for ~110 GB and get killed by the host memory guard. Use a conservative cap (~0.5, never 0.9) on memory-heavy jobs.