Model Seeding

Dubnium keeps model weights out of Git and out of the Nix store. Nix owns the runtime policy and vLLM service definition; model bytes are runtime data under /var/lib/dubnium/models.

The workstation configuration selects a vLLM model, but the USB seed format does not depend on one specific model. Use the configured model’s local bundle name where the examples say selected-model-bundle.

The installed workstation serves a local model bundle from:

/var/lib/dubnium/models/selected-model-bundle

This avoids depending on the Hugging Face hub cache layout at runtime. The USB seed carries a normal directory of model files plus a checksum manifest.

Runtime Model Store

Dubnium creates:

/var/lib/dubnium/models

The workstation vLLM service passes this local path to vllm serve:

/var/lib/dubnium/models/selected-model-bundle

Do not commit model weights to the Dubnium repo. Do not put model weights inside the Nix store or custom ISO payload.

USB Seed Layout

Use a stable USB layout so the same seed can be used during fresh install, recovery, or rebuild:

DUB-SEED/
└── models/
    └── selected-model-bundle/
        ├── config.json
        ├── generation_config.json
        ├── model-00001-of-000NN.safetensors
        ├── model-00002-of-000NN.safetensors
        ├── model.safetensors.index.json
        ├── tokenizer.json
        ├── tokenizer_config.json
        ├── vocab.json
        ├── merges.txt
        ├── LICENSE
        ├── README.md
        └── SHA256SUMS

The exact file set may vary by model revision, but the directory must be a materialized model snapshot, not a Hugging Face refs / blobs / snapshots cache tree.

Create A Local Bundle

If the source already exists as a normal model directory, copy it directly to the seed partition:

mkdir -p /run/media/$USER/DUB-SEED/models
rsync -a --info=progress2 \
  /path/to/selected-model-bundle/ \
  /run/media/$USER/DUB-SEED/models/selected-model-bundle/

Preferred source: a materialized model directory from a trusted local store or previously prepared artifact. Do not make the fresh install depend on Hugging Face availability.

Legacy fallback: if the only available source is an existing Hugging Face cache on the build machine, materialize the current snapshot once by following symlinks. This is a build-machine preparation step, not an install-time dependency:

MODEL_CACHE=/var/lib/vllm/.cache/huggingface/hub/models--OWNER--MODEL
REVISION="$(cat "$MODEL_CACHE/refs/main")"

mkdir -p /run/media/$USER/DUB-SEED/models/selected-model-bundle
rsync -aL --info=progress2 \
  "$MODEL_CACHE/snapshots/$REVISION/" \
  /run/media/$USER/DUB-SEED/models/selected-model-bundle/

Then create the checksum manifest:

cd /run/media/$USER/DUB-SEED/models/selected-model-bundle
find . -type f ! -name SHA256SUMS -print0 \
  | sort -z \
  | xargs -0 sha256sum \
  > SHA256SUMS

Seed From USB

After the workstation has booted into NixOS and the USB is mounted, copy the bundle into the Dubnium model store:

sudo mkdir -p /var/lib/dubnium/models
sudo rsync -a --info=progress2 \
  /run/media/$USER/DUB-SEED/models/selected-model-bundle/ \
  /var/lib/dubnium/models/selected-model-bundle/
sudo chown -R root:root /var/lib/dubnium/models/selected-model-bundle

Adjust the mount path if the USB is mounted somewhere else.

Verify the checksum manifest:

cd /var/lib/dubnium/models/selected-model-bundle
sudo sha256sum -c SHA256SUMS

Then verify the local model path exists:

test -f /var/lib/dubnium/models/selected-model-bundle/config.json
test -f /var/lib/dubnium/models/selected-model-bundle/model.safetensors.index.json

Acceptance Check

After seeding, switch to compute only when normal bring-up preconditions are satisfied:

sudo mode request compute
systemctl status vllm.service
journalctl -u vllm.service -b

The first start should load the local model path. If vLLM tries to fetch model files from the network, the model argument or bundle location is wrong.