Documentation

Instances

An instance is a containerized GPU sandbox running on dedicated hardware. Each instance has a name, one attached SSH key, and a selected GPU type.

GPU types

• NVIDIA DGX Spark GB10, single node, 128 GB unified memory.
• 2× NVIDIA DGX Spark GB10, dual node interconnect, 256 GB unified memory.
• NVIDIA RTX Pro 6000, 96 GB VRAM, 256 GB DDR5 ECC, Ryzen Threadripper 7960X.

If a type has no free capacity, your launch falls back to the queue instead of failing. Current hourly rates for every type live on the pricing page.

Lifecycle

Once launched, an instance stays available until you delete it. Dashboard status badges reflect live container state (running, exited, unknown).

• Restart cycles the container without ending the billing session.
• Delete removes the container and ends billing for that session.

SSH access

Running instances expose an SSH snippet in the format ssh user@<hostname>.ssh.enverge.dev. The first connect after a create or restart can take up to a minute while relay state settles.

2× DGX Spark cluster

A 2× DGX Spark instance is a two-node GPU cluster behind a single SSH endpoint. You land on node0 (the head). A second Spark (node1, the worker) is attached over a ~200 Gb/s RoCE fabric. You do not SSH to the worker directly, you launch a distributed job from node0 and it uses both GPUs.

Memory model

The tier advertises 256 GB unified memory, which is aggregate across two 128 GB pools joined by the fabric, not one flat address space. No single process or tensor can exceed ~128 GB. Use model parallelism or sharding to spread work across both nodes.

Pre-configured on your instance

• NCCL over RoCE, both ConnectX-7 rails are wired for RDMA (/etc/nccl.conf). NCCL auto-discovers the fabric, no manual NCCL_IB_* exports needed.
• VLLM_HOST_IP, preset to this node's rail IP (from /etc/profile.d/enverge-cluster.sh) so vLLM's multi-node control plane advertises the correct address instead of the default-route interface.
• Worker SSH, passwordless SSH from this container to the worker's container over the rail is already set up for launchers that spawn remote ranks. Check /etc/enverge/cluster.md on the instance for this VM's rail IPs and worker address.

Running a multi-node job

Launch rank 0 here and rank 1 on the worker over the rail. Any NCCL-based stack works once the workload is sharded. The shape is:

torchrun --nnodes=2 --nproc_per_node=1 --node_rank=0 \
         --rdzv_endpoint=<node0-rail-ip>:29400 your_job.py

your_job.py is your distributed script, nothing is pre-installed to run for you. Set --node_rank=1 on the worker, or let Ray or mpirun spawn the remote rank via the pre-configured rail SSH. Replace <node0-rail-ip> with this instance's rail-2 address (see /etc/enverge/cluster.md for the filled-in template, or ip -4 addr show enP2p1s0f0np0).

Unified memory on GB10

Shared filesystem

There is no cluster-wide shared filesystem yet. Model weights, datasets, and code must be present on each node, or downloaded separately. Plan for N× download size when pulling large checkpoints.

Queue

When capacity is unavailable, a launch request enters a queue instead of failing. Queue order is first in, first out per GPU type.

• You can have one active queue entry per GPU type.
• Queue entries can be cancelled at any time, at no charge.
• When capacity opens, the oldest matching entry launches automatically.
• We email you as soon as your instance is up, so you can close the tab.

Reserve

A reservation books a dedicated Spark for a fixed block of days. Where the queue trades certainty for a free wait, a reservation is the opposite trade: you pay up front and the hardware is guaranteed to be yours for the window you picked.

How it works

• Pick a start date and a length of 1 to 5 consecutive days.
• Pay up front through Stripe Checkout for the days you reserved, charged at a flat daily rate rather than by the minute.
• Your dedicated Spark launches automatically at the start of the window. No queue, no capacity check on the day.

Dates and windows

• A reserved day runs 00:00 to 00:00 UTC, so a one day reservation is a full UTC day, not 24 hours from checkout.
• The earliest start date is tomorrow, and you can book roughly a month ahead.
• Reserved days have to be consecutive. Book twice for two separate blocks.

Availability and changes

• Reservations are offered on the single NVIDIA DGX Spark GB10 today.
• A reservation that has not completed payment can be cancelled from the reserve page before its start date.
• Dates already taken by another reservation are unselectable in the calendar, so the window you book is the window you get.

Reserving is a separate flow from requesting datacenter capacity (H100, H200, B300, RTX Pro 6000), which is arranged with us directly rather than booked self-serve.

Startup script

A startup script is optional free-text shell that runs once inside a fresh instance after it is created. Use it so a box launched from the queue, often while you are offline, arrives already warming the GPU or serving a model instead of sitting idle until you SSH in.

The field starts empty. Nothing runs unless you put something there. What runs is exactly the text in the editor: templates only pre-populate it, nothing stores which template you picked.

Where to set it

In the create or queue panel, open + startup script (optional). Start from a template or write your own. Setup minutes are billed like any other minute on a live instance.

While an entry is queued it has not launched yet, so the script is still yours to change: use add / edit startup script on the queued card. Whatever is saved when capacity opens is what runs, and clearing the box means nothing runs.

Templates

• Warm up + benchmark, checks the GPU and leaves a benchmark figure for the login banner. Runs offline (the CUDA toolkit is already in the image), about 2 minutes.
• Download a model, pulls the weights into the instance's Hugging Face cache using vLLM's image, then stops. Nothing is served, no port is opened and no GPU is requested, downloading is network and disk. Defaults to Qwen3-8B (~16 GB), which finishes well inside the runner's one-hour cap even on a slow link. A later vllm serve then starts against a warm cache, and the script prints that command for you. Model and image are variables at the top.

Nested Docker needs the nvidia runtime and NVIDIA_VISIBLE_DEVICES, not --gpus all. The serve command the download template prints already uses the correct form.

When it runs

• Handed over only after SSH is answering and your key is written, then started in the background. Create, and the ready email, never wait for the script to finish. You can SSH in while it is still running, or because it failed.
• Runs once as user (a login shell, so CUDA is on PATH). Capped at one hour, a hung script is stopped and marked timed out.
• Does not re-run on a host reboot or when you restart the instance. A done marker blocks a second pass.
• Hard limit: 16 KB of raw script text.

Status, log, and stop

The runner reports what it observed, not that your workload is “ready”. A script that backgrounds a server can exit 0 while the server is still loading. The log is the authority.

• Instance card, a startup line: script running, script finished, script timed out, or script stopped.
• SSH login banner, the same state, plus the watch and stop commands when a script was supplied.
• Ready email (queue path), notes that a script was started, never that it succeeded.

# watch
tail -f /var/log/enverge/startup.log

# stop the oneshot unit (frees its GPU work)
sudo systemctl stop enverge-startup

# if your script left a docker container running, use its stop hint instead
# (shown on the banner while the script is running), e.g.:
docker stop my-container

SSH keys

SSH keys are managed during instance creation through a named key picker.

• Names are unique per account.
• Only paste public keys, never private keys.
• Keys attached to active instances cannot be deleted.
• Removing a key from your account does not retroactively remove it from existing instances.

Billing

Billing is metered by usage and settled through Stripe. A session is billed from create to delete. Rates per GPU type are on the pricing page.

Until a card is on file, launch and restart are gated. Add one from the dashboard banner or the billing section in your user menu.

On a team, only the owner adds or manages the card. Members see read only “billing managed by {team}” copy instead.

Queueing itself is free, and payment method checks happen when an instance launches. Reservations are the exception: they are charged up front at checkout, not metered per minute.

Teams

A team lets several people share one payment method. Everyone launches their own instances, usage is tracked per person and billed to the team. Open team from the user menu in the app.

Roles

• Owner, creates the team, manages billing, sends invites, sees each member's usage, and can remove members or revoke pending invites.
• Member, launches instances billed to the team. Cannot add a personal payment method. Sees their own usage in view usage, while the owner sees everyone's totals on the team page.

Create a team

Any signed-in user without an active team can create one from team in the user menu. You become the owner. If you already had solo billing, it moves to the team, you are not billed twice.

Invite a member

• The owner enters an email on the team page and clicks send invite.
• The invitee receives an email with an accept link.
• New users sign up first (the invite authorizes that email), then accept. Existing users sign in, then accept.
• Once active, the member can launch instances as soon as the team has a payment method on file.

Limits

• One active team per user.
• One payment method per team.
• Removing a member, or revoking a pending invite, takes effect immediately. Removed members fall back to solo billing once they set up their own card.

Usage history

Usage pages show both runtime sessions and queue history. Team members see only their own sessions and queue history, while the team owner sees per-member totals on the team page.

• Sessions table: one row per create-to-delete run.
• Queue history: wait duration and final outcome (launched or cancelled).
• Both views are paginated and refreshable from the UI.

Account

• Sign in with email and password, or with a magic link.
• A password reset flow is available from sign-in.
• Sign out is available from the user menu in the app.