Setup

Four steps: a server, a private network, the machines, and the CLI on whatever submits work. Only the first needs a public address.

1. The server

A small VPS is enough, the server schedules and does no computing itself. 4 GB of memory runs Postgres, dstack and Caddy with room to spare. The repository's deploy/ directory holds the whole stack as one compose file.

scp deploy/bootstrap.sh root@<vps>:/tmp/
ssh root@<vps> "bash /tmp/bootstrap.sh"

The bootstrap installs Docker, adds swap, and writes /opt/falllow/.env with a generated database password and admin token. Point DNS at the server, then publish a release, or run the Deploy workflow by hand: it builds the site and the manager, syncs the stack and waits until the server answers.

The admin token in the .env is what you log in with, in the manager and the CLI. It is only read when the server first starts against an empty database; after that the database holds it.

2. The network

The server reaches machines over SSH, so it has to be able to reach them. A rented box has a public address, a workstation at home does not. So every machine opens a WireGuard tunnel to the server and keeps it alive, and the server reaches the machine through it. Nothing forwards a port, nothing needs an account, and there is no third party in the path: WireGuard is in the Linux kernel.

bash deploy/wg/server.sh

That is the hub, 10.7.0.1. Machines get 10.7.0.2 and up as they join, and keep their address wherever they are.

3. The machines

A machine needs Linux and a root shell. A dual boot workstation joins while it is booted into Linux. Everything else, WireGuard, Docker, the pool's SSH key, is put in place by one script. On the server, register the machine and get that script:

cp ~/.ssh/dstack_fleet.pub /opt/falllow/dstack_fleet.pub   # once
bash deploy/wg/add-peer.sh workstation

workstation is 10.7.0.2. Run this on the machine as root:

bash -s <<'JOIN'
...
JOIN

Then add it to deploy/pool.yml as address: 10.7.0.2 and run: falllow pool apply

Paste the printed script on the machine as root. It reports back once the tunnel is up and the hub answers. The key it installs is the pool's own, generated for dstack rather than anybody's personal one; the CLI uploads the private half when the fleet is applied, and the server uses it to install its agent and start jobs.

Then list the machine in deploy/pool.yml, which is the one place a machine is written down: the fleet is generated from it and the supervisor reads the same file. blocks is how finely the machine is shared, not a limit on one job: auto is one block per core, and a job takes as many blocks as what it asked for needs. A machine that can be switched off says how, and how long it may idle first.

fleet: pool

ssh:
  user: root
  identity_file: ~/.ssh/dstack_fleet

machines:
  - name: workstation
    address: 10.7.0.2
    blocks: auto

  - name: rented
    address: 10.7.0.3
    blocks: auto
    power:
      provider: command
      status: hcloud server describe rented -o format='{{.Status}}'
      running_when: running
      start: hcloud server poweron rented
      stop: hcloud server poweroff rented
      stop_after_idle: 2h

falllow pool apply
falllow devices

Applying again after an edit adds and removes machines. Changes dstack cannot make in place, a changed address or block count, need falllow pool apply --replace, which rebuilds the fleet while the machines keep running. A machine that stops answering is shown as unreachable and gets no new work until it is back.

4. The CLI

On every machine that submits work: your laptop, a CI runner, an agent's sandbox. It needs Python 3.10 or newer and brings dstack's client with it.

uv tool install "git+https://github.com/milanofthe/falllow#subdirectory=cli"
falllow login --server https://app.falllow.com --token <token>
falllow run --no-workspace -- nproc

The login is stored in ~/.dstack/config.yml, so the dstack command that comes with it talks to the same server. That is all a machine needs: the working directory is shipped from it, nothing is cloned.