Mac Studio#

Note

Already SSH’d in?

After login, the banner points to /usr/local/share/enverge/mac.md on your instance: what is preinstalled, how to install packages without sudo, Docker via Colima, and Metal GPU notes. This page is the same material with a little more room.

A Mac Studio M5 Ultra instance is a standard macOS account (user) on the machine, with the whole Apple GPU and all of its 256 GB unified memory. It is not a Linux container and not a macOS VM: you get bare-metal Metal performance. There is no CUDA on a Mac.

What is installed#

The Xcode command-line tools are already there (clang, swift, git, make, xcrun), and Homebrew’s /opt/homebrew/bin is on your PATH with:

  • uv, fast Python environments and package installs.

  • pixi, conda-forge packages (Python and many non-Python tools).

  • macmon, live CPU / GPU / memory readout.

  • docker, colima, and lima, Docker CLI plus a Linux VM to run containers without root.

For GPU work on macOS itself, prefer MLX (mlx, mlx-lm) for the fastest path, or PyTorch on the mps device. Metal lets one process use up to about 240 GiB on the GPU, and any one buffer up to about 180 GiB: split anything larger into several arrays.

Homebrew#

Note

We can install almost anything you need

Email info@enverge.ai with the formula name and what you need it for, and we add it to the machine’s Homebrew, ready on your PATH. We install it on the host for every Mac Studio instance; it is not a per-tenant install.

Good candidates are native tools and build deps that ML stacks expect on PATH or via pkg-config, for example ffmpeg (video/audio pipelines), git-lfs (large weight repos), cmake / ninja / pkg-config (compiling llama.cpp or other native kernels), protobuf, libomp, libsndfile, or redis (local job queues).

You have no sudo, and Homebrew in /opt/homebrew belongs to the machine: you can use it, but not change it.

What works

  • Every tool in /opt/homebrew/bin.

  • brew list shows what is installed. brew search <name> and brew info <formula> find the formula name to ask us for.

What does not

  • brew install, upgrade, uninstall, tap and update stop with Error: /opt/homebrew/Cellar is not writable. Nothing is wrong with your instance: email us instead.

  • A Homebrew of your own needs an administrator to install the official way. A copy unpacked into your home is possible but unsupported: many packages build from source there, which takes minutes each.

  • brew services start needs a desktop session, and an SSH login has none. Start the service yourself, or from your startup script; it keeps running after you log out:

    nohup redis-server > ~/redis.log 2>&1 &

  • GUI apps (casks) have no desktop to run in.

Without asking us

  • Python: uv venv, uv pip install …, or uv run --with mlx python ….

  • conda-forge packages: pixi, for Python and many non-Python tools.

  • Anything else: drop a release binary in ~/bin and add that directory to your PATH.

Docker without sudo#

Start Colima once, then use the CLI as usual. Containers run in a Linux VM, so they have no GPU: use them for services and tooling, and run GPU work on macOS itself. colima stop gives the VM’s cores and memory back.

colima start
docker run --rm hello-world
colima stop

These hosts ship the Docker CLI and Colima, not Docker Desktop, so docker model is not available. The Desktop-free equivalent is dmr (standalone Docker Model Runner): same pull/run/serve workflow, no Docker.app. Put the binary in ~/bin, or ask us to add the Homebrew formula (see above). Metal inference still runs on the macOS host, not inside the Colima VM — Model Runner extracts a Python env under your home and starts a native process.

dmr serve
dmr pull ai/smollm2
dmr run ai/smollm2 "Hello"

Running vLLM (vllm-metal)#

On Apple Silicon, vLLM ships as vllm-metal. Do not use the Homebrew formula here (you cannot brew install). Use the official install script, which writes a Python environment under ~/.venv-vllm-metal. Or use dmr above if you want the Docker Model Runner workflow that drives vllm-metal on the host:

curl -fsSL https://raw.githubusercontent.com/vllm-project/vllm-metal/main/install.sh \
  | bash -s -- --stable
source ~/.venv-vllm-metal/bin/activate
vllm serve mlx-community/Qwen2.5-0.5B-Instruct-4bit

Prefer models from mlx-community on Hugging Face (often already quantized for MLX). After install, activate the venv in each new shell before running vllm. Full options are in the vLLM Metal docs.

Note

Want more examples?

Serving larger models, OpenAI-compatible clients, MLX vs vLLM tradeoffs, or a startup script that brings vLLM up on create: tell us what you are building at info@enverge.ai and we will add concrete recipes.