The Oath manual

How the system works today: shipped behavior, not plans. It is built from docs/manual in the repository, and a change to the system changes these pages in the same commit. On a running machine, start with oath, which prints /oath/INDEX.md.

Contents
  1. 1Using oath
    1. 1.1Start here
    2. 1.2Verbs
    3. 1.3Safety
    4. 1.4Disks
    5. 1.5Kernel module options
    6. 1.6A device's own protocol
    7. 1.7Who you are
    8. 1.8Agents
  2. 2Catalog
    1. 2.1Kinds
    2. 2.2Where objects come from
    3. 2.3Devices
  3. 3Services
    1. 3.1What ships
    2. 3.2Boot and converge
    3. 3.3State, restarts and logs
    4. 3.4Add and remove a service
    5. 3.5Account, environment, low ports
    6. 3.6Reboot and power off
    7. 3.7Off-box backup
    8. 3.8Scheduled services
    9. 3.9If PID 1 panics
    10. 3.10A new PID 1 without a reboot
  4. 4Packages
    1. 4.1What ships
    2. 4.2Install / remove
    3. 4.3Fields in practice
    4. 4.4A tree built here
    5. 4.5Publishing a package on its own (R72)
    6. 4.6Origins
    7. 4.7Plans
    8. 4.8Recipes
    9. 4.9Security audit
    10. 4.10Host store cache and publishing
  5. 5Network and SSH
    1. 5.1The link
    2. 5.2Names
    3. 5.3Time
    4. 5.4SSH
  6. 6Generations
    1. 6.1Layout
    2. 6.2Apply and undo
    3. 6.3Boot menu
    4. 6.4What the boot screen says
  7. 7Build and QEMU
    1. 7.1The toolchain and packages
    2. 7.2The kernel
    3. 7.3The image
    4. 7.4Test it
    5. 7.5Run it in QEMU
    6. 7.6Leave the VM
    7. 7.7Telemetry
  8. 8Install
    1. 8.1What it does
    2. 8.2The overlay
    3. 8.3Check the installed machine
    4. 8.4Rehearse in QEMU
    5. 8.5Onto a USB disk plugged into this host
    6. 8.6From a stick, on the machine: oath install
    7. 8.7USB installer stick
    8. 8.8Update an installed machine

1Using oath§

oath is the only admin surface. Humans and agents use the same verbs. Do not edit /etc or hunt random files.

1.1Start here§

oath                 the INDEX (generated, live; also /oath/INDEX.md)

--json prints the same facts as JSON. Failures include a hint pointing at the next step (oath ls, oath schema <kind>, INDEX safety).

The catalog root is /oath. --root / OATH_ROOT is for tests and for build and install tooling on the build host.

1.2Verbs§

VerbDoes
oath lsList object ids. --kind host (etc.) to filter
oath schema [kind]List kinds, or one kind's prose + JSON Schema
oath get <id>Desired + actual + status. --desired / --actual to clip
oath set <id> k=vWrite desired only. Does not converge. --from-json '<obj>' sets several fields. The result must fit the kind's type. oath set plan:<name> --from-file <path> files a plan (canonical, oath fmt -w); --from-file - files the canonical form of a plan read from stdin
oath new <id> k=vCreate a svc or pkg (--from-json too). Runs after apply
oath rm <id>Mark a svc or pkg for removal. Removed at apply
oath diff [id]Drift between desired and actual (or last applied, for svc)
oath apply [id...]Check, snapshot, then converge the listed ids or all drift
oath keepKeep an apply that is on probation. Run it from a new SSH session
oath undoPut back what the last apply changed (repeat to go further back)
oath verify [pkg:…]Hash linked package trees again against their pins; exit 1 if one changed on disk (-q: only damaged ones and the count)
oath audit [pkg:…]The daily security audit's findings for what is here, when a key this machine trusts signed them; --json is the report. From the next release
oath reapWhat holds each version in the store and what would go; sudo oath reap --confirm removes it, and each boot does. From the next release
oath logApply log (JSON lines): ok, error, refused, undo:N, keep:N. -n N for the last N
oath logs [id]A service's output (svc:<name>), the DHCP client's (net:net0), the last build of a plan (plan:<name>), or PID 1's (no id). -n N lines, default 50; --json gives {id, path, lines}
oath fmt <path>Canonical plan.plan to stdout (-w in place)
oath build plan:<name>Isolated, netless build of a pinned plan. Does not link /bin. Creates the product's pkg object if it is missing; --no-pkg does not, for a product that is only a build input to other plans
oath build --file <path>Same, ad-hoc (does not store the plan)
oath mcpThese verbs as an MCP server on stdin/stdout, for an agent (Agents)

set, new and rm are staging. Nothing is live until apply, and PID 1 never runs a spec that was not applied. set refuses a key the kind does not have, or a value of the wrong type (oath set svc:time restart=sometimes names the field and fails).

Writes to the catalog need root: over SSH, use sudo oath set, sudo oath apply, and so on. Reads (ls, get, diff, log) do not. Writers take a lock on /oath/.lock, so two oath commands never interleave; the second waits up to 30 s.

Ids are kind:name — host:local, svc:sshd, snap:current, pkg:curl, net:net0, ssh:local, dev:net0, plan:hello. Names are letters, digits, ., _ and - (at most 64, not starting with .).

1.3Safety§

apply checks every object it will touch before it changes anything. A refused apply (bad value, wants cycle, broken package needs, missing --confirm) changes nothing and takes no generation. Then it takes a filesystem snapshot and converges.

  • mutate — hostname, service enable, package present. Snapshot and go.
  • confirm — halt, reboot, picking an old boot generation (not undo), and anything that can cut this machine off from SSH: disabling, removing or changing svc:sshd, dropping a key from ssh:local, changing net:net0. oath apply without --confirm exits 3 and explains. Do not pass --confirm unless the owner asked for that class of change.
oath set host:local hostname=atlas
oath diff
oath apply

oath set host:local power=reboot
oath apply              # refused (exit 3)
oath apply --confirm    # reboots

After a reboot apply, desired power is set back to run so the box does not loop. The apply asks PID 1 for the reboot; PID 1 stops the services, syncs and remounts / read-only first. A power change on its own takes no generation (power is not configuration); plain sudo reboot and sudo poweroff do the same.

A change that can cut off SSH goes on probation once applied:

sudo oath set net:net0 ipv4=192.168.1.20/24 gateway=192.168.1.1
sudo oath apply --confirm              # on probation for 180 s
ssh home@192.168.1.20 sudo oath keep   # a new session: kept

If no new session runs sudo oath keep in time (--probation <secs>, 10–3600, default 180), PID 1 undoes the apply and logs undo:N by probation. A reboot before keep undoes it too. The session you applied from cannot keep it: only a login made after the apply proves the machine is still reachable. While a probation is open nothing else applies; sudo oath undo takes it back at once.

oath undo puts back the spec of the objects the newest apply changed (an apply that failed counts), converges them, and logs undo:N. Run it again to undo the apply before that. It never rewinds the log, the package store, or the SSH host keys.

1.4Disks§

Another disk mounts through host:local.mounts: a path under /mnt or /home, the filesystem by label or UUID, and mount options.

sudo oath set host:local mounts='{"/mnt/warehouse": {"source": "LABEL=warehouse", "options": "noatime,compress=zstd:1"}}'
sudo oath apply host:local

Apply mounts it now if the disk is plugged in, or prints that it waits for it. PID 1 mounts it at every boot and whenever the disk appears, and unmounts it at shutdown. set replaces the whole map: mounts='{}' and an apply unmount it, and oath undo takes a change back. oath get host:local --actual lists what is mounted now. Make the filesystem first (sfdisk, mkfs.btrfs -L <label>; blkid shows labels and UUIDs). A device name such as /dev/sdb1 is refused: it changes with the port and the boot. A machine's mounts belong in its install overlay (hosts/<name>.json).

Home itself can live on another volume, a btrfs subvolume of another disk, say:

sudo oath set host:local mounts='{"/home": {"source": "UUID=580d861c-4b96-4017-856c-0b55ff4e58ce", "options": "subvol=@oath-home,noatime,compress=zstd:1"}}'
sudo oath apply host:local

The apply changes nothing yet: it prints /home is … from the next boot. At the next boot PID 1 mounts it before anything reads /home, in place of the install's @home, and writes sshd's keys onto it. Copy what /home holds into the new place first (cp -a): the old home stays where it was, not mounted. If the disk is missing at boot, the machine comes up on the old home, keys written, and logs it (oath logs | grep 'mount: /home').

1.5Kernel module options§

A machine's hardware workarounds are module parameters in host:local.module_options, by module name. PID 1 passes them each time it loads the module; one already loaded takes a change at the next boot.

sudo oath set host:local module_options='{"nvidia": "NVreg_EnableGpuFirmware=0"}'
sudo oath apply host:local

They come after the parameters Oath always gives a module (amdgpu's si_support=1, nvidia_drm's modeset=1) and before module.key=value on the kernel command line, so a later one wins. Like mounts, they belong in the machine's overlay, never in a profile: another machine with the same driver gets none of them.

1.6A device's own protocol§

A program on the desk (as home) may need to talk to a device in its own protocol: a wireless mouse's battery, say, which the kernel does not report. host:local.seat_hid lists such devices by vendor:product. grep HID_ID /sys/class/hidraw/*/device/uevent shows them: 0003:00003554:0000F510 is 3554:f510.

sudo oath set host:local seat_hid='["3554:f510"]'
sudo oath apply host:local

PID 1 makes the device's vendor interfaces (/dev/hidraw*) root:home 0660, at once and whenever it is plugged in, whatever number it gets. An interface that carries a keyboard, a mouse or touch stays root's, so no program reads keys or motion; oath apply prints the nodes it gave. The list belongs to the machine, never a profile.

1.7Who you are§

Serial login is root (break-glass) when svc:serial is enabled. The screen and keyboard are home when svc:screen is enabled. SSH is home (uid 1, HOME=/home, login shell zsh). sudo (no password) becomes root. The groups are root, home, work and void. The apply log records uid and tty. host:local.env and TZ (from host:local.timezone) are injected by PID 1 into services and written to root-owned /etc/profile for logins, and to sshd's SetEnv in /etc/ssh/sshd_config, so ssh host 'command' has them too, as of sshd's last start. oath apply does not write $HOME/.profile. work (uid 2, home /work, R55) is for servers and workloads: nobody logs in as it, sudo refuses it, and /home is closed to it. sudo -u work <command> runs a command as it from home, and it cannot type into home's terminal, since the kernel's TIOCSTI needs root (R60); a service names it in user. /work is its own subvolume, @work, which PID 1 makes on the first boot that lacks it, so no generation or boot snapshot holds its data. void (uid 65534, home /var/empty) is the fourth account: nobody logs in as it; sshd, dhcpcd and chronyd drop to it.

A program can trace or read the memory of its own descendants only (Yama, R64, from the next release), so strace -p or gdb -p on a program already running takes sudo.

1.7.1The password and the seat§

home has no password until you set one (R42, login.md):

oath password                 # asks the current one (if set), then the new one twice
oath password --status        # password: set|none, autologin: on|off, seat: desk|console
printf 'old\nnew\n' | oath password   # off a terminal: the current one (if set), then the new

An empty new password removes it. svc:login keeps it, hashed, in /oath/login/password (root's alone); a wrong one is answered after a delay that doubles each time, up to 30 s. It is asked at the console when host:local.autologin is false, and by the desk's lock screen (Sola's); SSH (keys only) and sudo never ask, so it guards the screen and keyboard, not the account: login.md says exactly what it does and does not protect. Auto-login is on by default; Sola's Settings turn it off with the password, and root can with sudo oath set host:local autologin=false and sudo oath apply host:local.

On a desk, the screen and keyboard (tty1) are the seat, and either the desk or the console has it. Exit Sola ends the desk on purpose (oath desk stop does the same): its services are held (state: "held") and the console, a shell as home (or the password prompt), takes tty1. oath desk, from that shell or over SSH, gives the seat back and the desk's services start again. Every boot starts with the desk.

A machine installed before R42 gets the programs with oath update (its pkg:oath moved first, install.md) but not the objects. Reboot into the update first: an older PID 1 knows no seat and would start the console on tty1 under the running desk. Then make them once:

sudo oath new svc:login --from-json '{"exec":["/lib/oath/login"],"wants":[],"restart":"always","enabled":true}'
sudo oath set svc:screen enabled=true seat=console
for s in rivulet sola-river sola-shell sola-session sola-kvm; do sudo oath set svc:$s seat=desk; done
sudo oath apply

1.7.2The shell§

home's login shell is zsh (pkg:zsh, not removable); /bin/sh is bash, and root's shell. A login zsh reads /etc/profile through /etc/zprofile. /etc/zshrc sets the prompt, history in ~/.zsh_history, emacs keys and completion:

14:02:07 home@canto Workspace/sola main § 

The time, user@host, the last two directories of the working directory, the git branch, and §. user@host and the branch take a color from a hash of the hostname, so each machine keeps its own and a new hostname gets a new one. oath writes /etc/zprofile and /etc/zshrc with /etc/profile, when host:local converges and at boot; put your own settings in ~/.zshrc. ssh home@host 'command' runs the command through zsh, with host:local env from sshd.

1.8Agents§

An agent reaches the machine the way the owner does: over SSH as home, with a key in ssh:local. oath mcp speaks the Model Context Protocol on stdin and stdout, so an MCP client gets the verbs as tools and the INDEX as its instructions. Nothing listens on a port; the server lives as long as the SSH connection.

claude mcp add oath -- ssh -T -o BatchMode=yes home@<host> oath mcp
ssh -T home@<host> oath mcp --read-only    # only the tools that look

Each tool runs one oath --json command and returns its JSON: index, ls, schema, get, diff, log, logs and verify look; set_host, set_svc, set_pkg, set_net, set_ssh, new_svc, new_pkg, rm, apply, undo, keep, update (what the newest output would move, or with confirm move to it), file_plan (a plan from its text) and build run through sudo, as the owner's would. The same reads are resources too: oath://index, oath://schema/<kind> and oath://object/<kind>/<name>. A client that sends a progressToken hears every 5 s that a long build or apply is still running. A tool's fields are the kind's schema, so a field added to a kind reaches the tools with nothing else to do. A failed command is a tool error carrying the CLI's hint; exit 3 says the apply needs confirm, which an agent passes only when the owner asked for that class of change.

oath log shows what came through MCP: such entries have actor.via: "mcp:<client>", and an apply given confirm has confirm: true. A change that can cut off SSH goes on probation as usual, and the connection that applied it cannot keep it: a new ssh … oath mcp (or ssh home@<host> sudo oath keep) can. The server speaks both the current MCP revision (2026-07-28) and the initialize-based ones before it.

2Catalog§

The catalog is the source of truth. It lives at /oath.

/oath/
  INDEX.md                 generated. read this first
  schema/<kind>.json       JSON Schema 2020-12
  schema/<kind>.md         prompt-sized prose
  objects/<kind>/<name>/   spec (snapshots hold it; undo reads it)
    desired.json           what you asked for
    applied.json           the spec as last applied (PID 1 runs a svc's)
    meta.json              id, safety; `fixed` on a seed-fixed package
    new / remove           markers for a pending `oath new` / `oath rm`
  status/<kind>/<name>.json  what the machine observed: tmpfs, rebuilt each boot
  log/apply.jsonl          every apply (ok, error, refused), undo and keep
  probation.json           an apply that can cut off SSH, until `oath keep`
  log/init.log             PID 1's messages
  log/svc/<name>.log       each service's output (`oath logs svc:<name>`)
  log/net/net0.log         the DHCP client
  .lock                    held by whoever is writing the catalog
  store/pkg/<name>/<hash>/ package trees (apply links /bin)
  store/plan/<name>/<hash>/plan.plan
  ssh/                     host keys
  run/                     sockets and mounts, not desired state
    init.sock              `oath apply` asks PID 1 to converge here
    fs/                    btrfs top level (subvolid=0)

Desired and actual are separate files. You may cat any of these; oath get is the supported way. If a kind is not under /oath/schema, it does not exist. Every file is replaced atomically (temp file, fsync, rename), so a reader never sees half of one.

2.1Kinds§

KindIds in the base profileRole
hosthost:localHostname, power, env, timezone, hosts, origins, mounts
svcsvc:serial, svc:sshd, svc:timePID 1's only config
snapsnap:current, then snap:NGenerations
pkgglibc, bash, zsh, coreutils, btrfs, oath, openssh, curl, … (39 in base)Store + /bin and /lib symlink farms
netnet:net0The link: dhcp or a static address
sshssh:localKeys that may log in as home
devdev:net0, dev:disk-pci-… (discovered)Hardware inventory
plannone until you file oneHashed plan.plan; oath build writes a pkg tree

Drift is desired against applied.json, the spec as last applied, for every kind; actual (/oath/status/…) is only what the machine observed (svc actual is runtime: state, pid, restarts).

oath schema <kind> on the box gives fields, safety, and examples. That text is the same as crates/oath-core/schema/ in the repo.

2.2Where objects come from§

On the box, oath new <id> creates a svc or pkg and oath rm <id> removes one; both take effect at oath apply, and oath undo of that apply reverses them. host, net, ssh, snap and dev objects, and the base packages, cannot be created or removed. Before the box first boots, objects come from:

  • Seed profiles — crates/oath-core/seed/<profile>.json, data that pkg:oath ships as /lib/oath/seed/ and no binary carries (R72), so a profile's pins never change PID 1. base is a headless machine you reach over SSH; desk extends it with the graphical session. oath seed --profile <name> reads /lib/oath/seed (OATH_SEED_DIR for another directory; cargo make sets it to the tree's). cargo make build --profile <name> seeds the image from one; a profile lists the first desired document of each object, the packages that can never be removed (fixed), and any first actual that differs from the default.
  • Overlays — {"objects": {"kind:name": {…}}}. Each object is a JSON merge patch (RFC 7386) on the current desired: objects merge key by key, null deletes a key, anything else replaces. A new id creates the object. Each patched document must pass the same checks as oath set. An overlay counts as applied: PID 1 boots with it. cargo make install --overlay <file.json> merges one into the installed catalog, after the hostname and the SSH keys the installer writes.
  • Plans — oath set plan:<name> --from-file files a plan, and oath build creates the product's pkg object if it is missing (--no-pkg does not, for a product only other plans build with).

oath seed is the build- and install-time tool behind the first two. It is hidden from oath --help:

oath --root ./stage/oath seed --profile base
oath --root /mnt/oath seed --overlay overlay.json

A fresh root gets the profile (default base). An existing catalog only takes --overlay; --profile on it is refused.

2.3Devices§

dev:* is the hardware inventory, discovered by PID 1 at boot and again whenever a disk, NIC or serial port comes or goes: disks are dev:disk-<bus path> (dev:disk-pci-0000.00.1f.2-ata-1), the NIC is dev:net0 (net:net0 is its link) and other NICs dev:net1, …, a real UART keeps its kernel name (dev:ttyS0). Actual has the node, bus path, driver, and for a disk its size, model and removable. A device that goes away loses its object. /dev is nodes from devtmpfs, not the admin UI. There is no udevd; PID 1 loads the driver a hot-plugged device asks for (devices spec).

PID 1 also mounts, outside the catalog: /proc, /sys, devtmpfs /dev with /dev/pts, tmpfs /tmp, /dev/shm and /run, and cgroup2 at /sys/fs/cgroup. /etc/hosts has localhost, the hostname and host:local.hosts (Network).

3Services§

PID 1 is oath-init (/lib/oath/init, also /sbin/init). It is not a catalog object. Every supervised process is a svc:* object. There is no unit file, no systemd, no /etc/init.d.

3.1What ships§

The base profile:

IdExecDefaultNotes
svc:serial/lib/oath/serial-loginenabled, restart=alwaysRoot shell on the serial console (ttyS0 / hvc0). With no UART it idles and logs no serial console. Do not disable it unless you have another way in.
svc:screen/lib/oath/screen-login, user: homeenabled, restart=alwaysLogin shell as home on the screen and keyboard (tty1); sudo for root. The screen shows it once the display driver loads, just before services start. A desk turns it off.
svc:sshd/bin/sshd -D -e (OpenSSH)enabled, restart=always, keep_childrenKeys in ssh:local; /etc/ssh/sshd_config comes from host:local. Passwords and root login off. Restarting it keeps open sessions.
svc:time/bin/chronyd -d against 0–3.pool.ntp.orgenabled, restart=alwaysKeeps the clock (see Network); runs as void after start-up.

The desk profile adds the desk's own (rivulet, Sola's, PipeWire's) and two for Bluetooth: svc:dbus-system, the system bus (/run/dbus/system_bus_socket, as void), and svc:bluetooth, BlueZ's bluetoothd. Pair a device with bluetoothctl: power on, scan on, then pair, trust and connect with its address; devices lists the known ones. Headphones then show in wpctl status; a game controller, keyboard or mouse on Bluetooth LE is an input device.

Add a service on the box with oath new and oath apply (below), or at install time with an overlay (see Catalog).

3.2Boot and converge§

At boot PID 1 mounts the floor, switch_roots into the root subvolume @, mounts the btrfs top level at /oath/run/fs, sets the hostname from host:local, loads the drivers its devices ask for, brings up net:net0, refreshes dev:*, writes ssh:local keys, then starts every enabled service in wants order (and the DHCP client, see Network). Host, network and keys come from what was last applied, not from pending oath set edits. Display and sound drivers load after that, so svc:sshd is up before any GPU driver. oath logs lists the modules each pass loaded.

PID 1 runs only what was applied: each service's applied.json, which oath apply writes after the spec passes its checks. A set without apply never reaches a running service, even across a reboot. A spec PID 1 cannot read (a torn file) leaves the running process alone.

PID 1 reaps children and restarts them per restart (never / always / on-failure). It stops services that are disabled or whose applied spec changed. Each service runs in its own cgroup (/sys/fs/cgroup/oath/svc-<name>): stopping it sends SIGTERM to every process in it, waits up to 5 s, then kills what is left. With keep_children (as on svc:sshd, and svc:sola-session on a desk) only the main process is signalled, its exit leaves the rest running, and what they write still reaches the service's log. Applying keep_children or restart reaches a running service at once, without restarting it.

oath apply on a svc writes applied.json, notifies PID 1 on /oath/run/init.sock, waits for PID 1 to answer that it has converged, then checks that actual state is running or stopped. oath undo puts back the previous applied.json and does the same.

wants is start order: enabled wanted services start first. Disabled or unknown wants are skipped. A cycle refuses apply. If a cycle reaches PID 1 anyway, it skips the services in the cycle (and those waiting on them) and starts everything else, svc:sshd included.

oath ls --kind svc
oath get svc:time
oath set svc:time enabled=false
oath apply svc:time
oath get svc:time --actual
oath undo

3.3State, restarts and logs§

oath get svc:time --actual
oath logs svc:time          # last 50 lines; -n 200 for more
oath logs                   # PID 1's own log

Actual has state, pid, restarts (restarts this boot), last_exit ({"code": 3} or {"signal": 15}), since (UTC) and, while it runs from a package, pkg (that package and hash). state is:

  • running — the process is up.
  • restarting — it exited and waits for its backoff.
  • stopped — disabled, or a finished restart=never run.
  • failed — it could not be started, it is in a wants cycle (or waits on one), or it died within 10 s of starting five times in a row; reason in actual says which. Restarts wait 1, 2, 4, then 8 s between those quick failures; a run of 10 s or more restarts at once. A failed service stays down until its applied spec or its program's package changes: fix it with oath set and oath apply.

A service's stdout and stderr go to /oath/log/svc/<name>.log, one line each with a UTC time in front, rotated to <name>.log.1 past 256 KiB. PID 1 writes its own messages to /oath/log/init.log; the DHCP client writes to /oath/log/net/net0.log (oath logs net:net0).

3.4Add and remove a service§

sudo oath new svc:tick --from-json '{"exec":["/bin/sleep","1000"],"wants":[],"restart":"always","enabled":true}'
sudo oath apply svc:tick        # starts it
sudo oath rm svc:tick
sudo oath apply svc:tick        # stops it and deletes the object
sudo oath undo                  # brings it back

oath new needs only exec; it writes the defaults into the object (restart: always, enabled: true, wants: []), so oath get shows what will run. Until oath apply, a new service has status new and does not run; once applied it is not new, and if it cannot start (a wrong exec, say) its status is error and its actual has the reason; oath rm before that deletes it at once.

svc:sshd changes that can cut off SSH (disabling or removing it, or changing its exec, user or env) need oath apply --confirm and go on probation: a new SSH session runs sudo oath keep, or PID 1 undoes the change after 180 s.

3.5Account, environment, low ports§

Oath has four Unix accounts: root, home (uid 1, gid 1), work (uid and gid 2, home /work), for servers and workloads, which sudo refuses and which cannot log in, and void (uid and gid 65534, home /var/empty), the unprivileged one that owns nothing and cannot log in. user is root, home, work or void; absent means root. A service that keeps data and needs no sudo runs as work; one that needs no files of its own and no privilege runs as void.

Every service gets PATH=/bin, HOME, USER, LOGNAME, SHELL=/bin/sh and XDG_RUNTIME_DIR for its account (the desk profile's Sola services set SHELL=/bin/zsh, so terminals start the login shell), then host:local.env, then TZ from host:local.timezone (unless env already sets TZ), then its own env. The service's env wins on the same name.

bind_low_ports lets a service of any account but root bind ports below 1024 (CAP_NET_BIND_SERVICE). Root can bind them either way. On a service an overlay added (here svc:web):

oath set svc:web user=home bind_low_ports=true
oath set svc:web --from-json '{"env":{"WEB_ROOT":"/home/site"}}'
oath apply svc:web

rtprio (1–99) lets a service give its threads realtime priority up to that value and renice itself to -20 (RLIMIT_RTPRIO, RLIMIT_NICE); the desk profile gives PipeWire 95.

Changing exec, user, env, bind_low_ports or rtprio restarts the process. So does an apply that moves the package its program comes from: PID 1 follows exec[0] through /bin into the store (/oath/store/pkg/<name>/<hash>/…), and when that realization changes it restarts the service. Actual pkg names it ({"id": "pkg:sola", "hash": "sha256-…"}). A service whose program is in another package keeps running; one that was failed tries again with the new program. Moving pkg:openssh restarts svc:sshd, so it needs --confirm and goes on probation.

An update does not restart a service with a seat (the desk's, the console's): oath update, and oath apply --keep-seat, leave it on the program it started with, its actual reason says so, and it takes the new one when it next starts (a reboot, Exit Sola and oath desk, or its own exit). With --reboot, oath update restarts no service before the reboot. A PID 1 from desk output fdecb1ec3d68 or older restarts the seat's services anyway, unless the update reboots.

3.6Reboot and power off§

sudo reboot, sudo poweroff and sudo halt ask PID 1 to shut down, and so does oath set host:local power=reboot (or halt) with oath apply --confirm. PID 1 stops every service (SIGTERM, up to 5 s, then SIGKILL), stops everything else, syncs, unmounts the btrfs top level, remounts / read-only, then reboots or powers off. Ctrl-Alt-Del on the console reboots the same way.

3.7Off-box backup§

The image ships two helpers, not a service:

  • /lib/oath/backup-send host:/export/path — snapshot @, mount the NFS export (version 4, else 3), write <hostname>.send (one btrfs send stream, overwritten each time) and <hostname>.json (generation, UTC time, size, sha256). It is not a second undo; oath undo never reads it.
  • /lib/oath/backup-daily host:/export/path — sleep until 04:00 local time (TZ, from host:local.timezone; UTC when unset), run backup-send, repeat.

To run it daily, add a service in the machine's install overlay:

{"objects": {"svc:backup": {
  "exec": ["/lib/oath/backup-daily", "nas:/export/backup/canto"],
  "wants": [], "restart": "always", "enabled": true}}}

A present package may ship /oath/store/pkg/<name>/live/libexec/oath-backup-quiesce and oath-backup-thaw; backup-send runs them around the snapshot. Without them the copy is crash-consistent (sync plus btrfs CoW).

3.8Scheduled services§

A schedule (R48) makes a oneshot start at times, as cron would: a pattern (five fields, or six with seconds; @daily; croner's L, W and #) in the machine's time zone, host:local.timezone, so 04:00 stays 04:00 across daylight saving.

sudo oath new svc:backup --from-json '{"exec":["/lib/oath/backup-send","nas:/export/backup/canto"],"wants":[],"restart":"never","enabled":true,"schedule":"0 4 * * *"}'
sudo oath apply svc:backup
oath get svc:backup     # state: "scheduled", next: "…04:00:00-06:00", last_run

Apply does not start it, and refuses a schedule without restart: "never" or one croner cannot read. PID 1 starts it at each match; at boot, once, when a match passed while the machine was down (the last run is in /oath/log/svc/<name>.schedule.json); and never twice at once: a match while the last run still goes is skipped, and init.log says so. Its output goes to oath logs svc:<name> as any service's.

3.9If PID 1 panics§

A panic in PID 1's main loop is logged and survived: PID 1 hands its state to a fresh copy of itself, and every service keeps running and stays supervised (oath logs shows panic: re-executing PID 1 and resumed after a panic). A third panic within a minute makes it shut down cleanly and reboot.

3.10A new PID 1 without a reboot§

sudo oath update --confirm (no --reboot) has PID 1 re-execute into the output's binary the same way, every service running (R73, install.md). oath logs shows upgrade: … rehearsing, upgrade: re-executing PID 1 into … and upgrade: this binary took over; /run/oath/init.json says which binary PID 1 runs (exe) and how it started (how: boot, panic or upgrade). A binary with the wrong hash, or one that cannot read PID 1's state, is refused (upgrade: not into …), and PID 1 goes on as it was.

4Packages§

A package is a pkg:* catalog object. Bits live under /oath/store/pkg/<name>/<hash>/. /bin is a symlink farm, not an unpack target, and so is /lib: glibc's loader (/lib/ld-linux-x86-64.so.2) and every linked pack's lib/ entries (decision R17). There is no /lib64 and no RPATH; a program finds its libraries through /lib. Name is a slot; hash is the bits (sha256- plus 64 hex of oath-tree-v1). There is no apt and no oath install.

The slot also has live → <hash> and compatibility links (bin → live/bin, …) so paths like /oath/store/pkg/<name>/bin resolve. Those links are not part of the hash.

4.1What ships§

The base profile:

Not removable (present=false is refused):

IdWhat
pkg:glibcthe C library and its loader
pkg:bashbash, also /bin/sh
pkg:coreutilsls, cp, cat, sort, hostname, …
pkg:util-linuxmount, dmesg, sfdisk, wipefs, blkid, lsblk, findmnt
pkg:iproute2ip, ss, bridge, tc
pkg:dhcpcdthe DHCP client of net:net0
pkg:btrfsbtrfs, mkfs.btrfs
pkg:oathoath
pkg:opensshthe SSH server sshd (R36), and ssh, scp, sftp, ssh-keygen, sftp-server
pkg:zshzsh, home's login shell

What the running oath's base profile fixes is what counts: a package a machine was seeded with as fixed and the base no longer fixes (pkg:dropbear, before R36) can be removed.

Present and removable: sed, grep, gawk, findutils, diffutils, tar, gzip, xz, bzip2, procps (ps, pidof, free, top, kill), less, vim (also vi), iputils (ping, tracepath, arping), chrony (svc:time), dosfstools (mkfs.fat), curl (Mbed TLS, CA bundle at /oath/store/pkg/curl/live/ssl/cert.pem), bind (dig, host, nslookup; no named), openssl (openssl; its default ssl/cert.pem links to curl's bundle), certs (Oath's CA roots, R50), and the libraries they load: ncurses, gmp, mpfr, zlib, zstd, mbedtls, libuv, liburcu, libcap, libxcrypt. A pack's needs lists the libraries it loads, so apply refuses removing pkg:ncurses while pkg:less is present.

Every pack is built from pinned source on the build host: the Oath binaries (pkg:oath) from this tree, the rest from pkgs/ (Build). There is no C compiler on the box yet: the toolchain (pkg:gcc) exists on the build host but is not in the base profile.

PID 1 stays at /lib/oath/init; it is not a package.

4.2Install / remove§

oath ls --kind pkg
oath get pkg:dosfstools
oath set pkg:dosfstools present=false
oath apply
ls /oath/store/pkg/dosfstools     # the tree stays

oath set pkg:dosfstools present=true
oath apply
readlink /bin/mkfs.fat
oath undo

present=false removes this object's /bin links. The store tree stays, so a re-install needs no network. A new hash relinks, and a program the new tree no longer has leaves /bin. Apply refuses to clobber a /bin name it does not own. present=false on a non-removable package is refused (not --confirm).

4.3Fields in practice§

  • hash is the pin. Apply hashes the tree again before linking it and refuses one that changed on disk. Empty hash is discovery (actual.hash reports what is live). Switch with oath set pkg:foo hash=sha256-…, then apply. Other hashes stay on disk.
  • url: if present and the store is missing, apply fetches it, then links. A .tar is a pack tree. An origin prefix plus hash fetches {url}/pkg/{name}/{hash}.tar. A single file becomes bin/<name>. With no url, a pinned pack the store lacks comes from the machine's origins (below).
  • file: with url, the URL is one file, a vendor's release, and the tree is that file alone at this path, not executable (a recipe's source; see Recipes).
  • needs lists other packs this one needs at runtime, each pinned by hash. Apply refuses a broken graph: a missing need, a need at the wrong hash, a cycle, or removing a need while a present pack still lists it. Apply does not change the other objects for you.
  • description and home are prose and a project URL.
  • source records where the pinned tree was built from: {repo, commit, branch, dirty, built_by, built_at} (oath schema pkg). Set it with the hash it describes, in one apply. It is in no hash, and an apply that changes only it (or the prose) touches no store, link or service.

4.4A tree built here§

oath store puts a tree built on the machine (a directory, or a .tar or .tar.gz of one) into the store and prints its hash. It changes no object and no link; pinning the hash and applying does:

sudo oath store pkg:hello ./out            # pkg:hello sha256-…
sudo oath --json store pkg:hello out.tar   # {"hash":"sha256-…","id":"pkg:hello"}
sudo oath set pkg:hello hash=sha256-… needs='[{"id":"pkg:glibc","hash":"sha256-…"}]'
sudo oath apply pkg:hello

A tool that pins a tree it built records where from, with the hash:

sudo oath set pkg:hello --from-json '{"hash":"sha256-…","source":{"repo":"https://github.com/me/hello","commit":"<40 hex>","branch":"next","dirty":false,"built_by":"home@canto","built_at":"2026-11-01T12:00:00Z"}}'

A pin that is not the release's, or not the head this machine took (below), is an overlay: oath get pkg:hello ends with overlay: pkg:hello is next@1a2b3c4d, not the release's sha256-… (output …); oath update replaces it (or not its head sha256-… (signed by …)), and oath diff lists every overlay above its in-sync. sudo oath update --confirm replaces each; --keep pkg:hello keeps one.

A new name is oath new pkg:hello hash=… instead of set; a source pack a plan builds with is oath new pkg:hello-src hash=… present=false and goes in the plan's build_needs. Every file is stored 0755 or 0644 (the hash records only whether it is executable), and a pipe, socket or device in the tree is refused, as a fetch refuses it.

A service follows its package: when an apply moves the pin of the package a service's program comes from (exec[0], through /bin), PID 1 restarts that service, and no other (an update leaves the desk's running: Services). So a change to a program is build, oath store, oath set … hash=, oath apply.

4.5Publishing a package on its own (R72)§

Whoever makes a package can publish it from the machine it was built on, and every machine that trusts that key for the name takes it at its next oath update: no profile, release or build host in between. After the four steps above:

oath publish pkg:hello                      # what it would publish, and the head it replaces
oath publish --confirm pkg:hello            # the pack (if the origin lacks it), then its head
oath publish --confirm pkg:hello --dest d   # or write the same files into a directory
oath publish --confirm pkg:hello --committed 2026-11-06T12:00:00Z

The head is pkg/hello.json on the origin, beside pkg/hello/<hash>.tar: the pin, its needs, source, version and description, signed beside itself (.sig, namespace oath-pkg) by ~/.config/oath/sign_ed25519 (OATH_SIGN_KEY) and written with the origin's write key, as for cargo make publish (below). The pin must be applied and built from a clean tree (source.dirty false). With --committed (the commit's time, any offset; the head keeps it in UTC) it refuses to replace a head built from a later commit. Publishing needs no sudo.

A machine trusts a publisher for some names only by putting a list before its key in host:local.publishers:

sudo oath set host:local publishers='["ssh-ed25519 AAAA… oath@buccina", "names=\"sola,sola-*\" ssh-ed25519 AAAA… oath@novus"]'
sudo oath apply host:local

Such a key signs heads of those names only: never a release or the audit's report, so it cannot ship PID 1, the kernel or sshd. oath update reads the head of every package the machine has (and of what those heads need) from each origin and moves each package to the newest head a key may sign for it; oath update alone says what would move. It records each head it took in /oath/heads.json and refuses an older one after. A head wins over a release's pin of the same package, and a package with a head stays when a release drops it. With host:local.profile empty a machine takes heads only.

A need is a pin. When the moves leave one at another pin (a new pkg:mesa under a package built against the old one), the update re-points it if every library the package's ELF files load is still in /lib after the moves, or in its own tree; otherwise it holds the moved package back and says which package and library hold it:

pkg:mesa stays at sha256-…; sha256-… waits: pkg:sola needs pkg:mesa at sha256-…, and would lose libEGL.so.1
pkg:sola keeps its pin; its needs move: pkg:mesa sha256-… -> sha256-… (every library it loads is still in /lib)

A preview, whose packs are not fetched yet, says it checks the libraries once they are; --confirm fetches them and checks before anything changes. A library loaded with dlopen is not seen.

Upstream identity is components.json at the pack root (name, version, purl); it is part of the tree hash, and missing means unknown. Advisories for a pin live at /oath/store/adv/<name>/<hash>.json, fetched from the origin when url is one, or from the origin that just served the pack. oath get shows both in actual. An empty list is not a safety proof.

4.6Origins§

A pack is its hash; it does not know where it is hosted. The machine says where it looks: host:local.origins, a list of http:// or https:// prefixes tried in order as {origin}/pkg/<name>/<hash>.tar until one serves the pinned tree. Apply uses them for a pinned pkg with no url; oath build uses them for a build pack the store lacks. The base profile lists the default origin, the R2 bucket at https://store.oath.moonlight.pm, which carries every locked pack (build packs too) with its plan and source; its INDEX.md lists names and hashes.

oath get host:local                                   # origins
curl -fsS https://store.oath.moonlight.pm/INDEX.md    # what it serves
sudo oath new pkg:m4 --from-json '{"present":true,"hash":"sha256-…"}'
sudo oath apply pkg:m4
sudo grep '"fetched"' /oath/log/oath.jsonl | tail -1  # where it came from

Some packs are on the origin and in no profile, and a machine adds them this way: Node.js (pkg:node, which needs pkg:gcc), the GitHub CLI (pkg:gh, which needs pkg:certs for its CA roots) and Cloudflare's CLI (pkg:cf, which needs pkg:node). Each need is pinned at the hash the pack's plan names (run_needs in {origin}/plan/<name>/<plan>.plan), which must be the one the machine runs:

sudo oath new pkg:node --from-json '{"present":true,"hash":"sha256-…","needs":[{"id":"pkg:gcc","hash":"sha256-…"}]}'
sudo oath apply pkg:node
sudo oath new pkg:gh --from-json '{"present":true,"hash":"sha256-…","needs":[{"id":"pkg:curl","hash":"sha256-…"}]}'
sudo oath apply pkg:gh
sudo oath new pkg:cf --from-json '{"present":true,"hash":"sha256-…","needs":[{"id":"pkg:node","hash":"sha256-…"}]}'
sudo oath apply pkg:cf
node --version                           # v26.13.0
gh --version                             # gh version 2.103.3 (2026-10-26)
cf --version                             # 1.0.0-beta.26

4.6.1Servers: Blocky, restic, route, cell, PostgreSQL (R49)§

These are built from source here and are in no profile: pkg:blocky, pkg:cloudflared, pkg:restic, pkg:rsync, pkg:route, pkg:cell (with pkg:crun, pkg:passt), pkg:postgresql and pkg:postgis (with pkg:geos, pkg:proj, pkg:libxml2, pkg:json-c, pkg:icu). A machine takes them from the origin as above, with each need the plan names, once a release has published them. pkg:go is what builds the Go ones; a machine needs it only to build. cloudflared does not update itself: cloudflared update says so, and a newer one is a new pin in pkgs/cloudflared.

Go programs read their CA roots from pkg:certs (R50), through Go's own crypto/x509, unless SSL_CERT_FILE names others: /oath/store/pkg/certs/live/ssl/cert.pem. A machine without pkg:certs makes it first, at the hash the plan names. It is Mozilla's roots for servers, from NSS's certdata.txt, less any with a distrust-after date and what pkgs/certs/distrust.txt names. curl and OpenSSL still read pkg:curl's copy, until the rebuild that takes CEF 157 (R77).

PostgreSQL finds PostGIS, a separate pack, through one line in postgresql.conf:

extension_control_path = '$system:/oath/store/pkg/postgis/live/share'

The server refuses to run as root, so its svc names a user. A new major version changes the data directory's format, so moving across one is pg_upgrade, not a new pin alone.

pkg:cell runs a container image (R59), in apps/cell, with nothing but Go's standard library over crun and pasta. There is no daemon: cell run stays in the foreground, so a svc supervises it like any program, and its output is the service's log. Without -t that output goes through pipes cell makes, so a program can open /dev/stderr.

sudo -u work cell run --name web -p 8080:8080 -v /work/web:/data --env-file /work/web.env nginxinc/nginx-unprivileged:alpine
sudo oath new svc:web --from-json '{"exec":["/bin/cell","run","--name","web","-p","8080:8080","nginxinc/nginx-unprivileged:alpine"],"wants":[],"restart":"always","enabled":true,"user":"work"}'
sudo oath apply svc:web
CommandDoes
cell pull <image>fetch an image, or move its tag; prints the digest for linux/amd64
cell run [flags] <image> [args...]pull it if it is not on disk, and run it
cell exec [-t] [-w dir] <name> <cmd...>run a command in a running cell, under the same limits
cell stop <name>stop it: the image's STOPSIGNAL, then SIGKILL after --stop-after (10 s)
cell ls, cell imageswhat is running, what is on disk
cell rmi <image>, cell pruneforget a name; delete images no name holds, and what a killed cell left
cell login <registry> <user>keep a password, read from stdin, in auth.json

run takes -p [addr:]host:port[/udp], --reach port[:hostport], -v /host:/path[:ro], --tmpfs /path, -e NAME=value, --env-file file, -w dir, --entrypoint cmd, --user uid[:gid], --read-only, --no-net, -t, --name.

Each run starts from a fresh copy of the image, a reflink on btrfs, and leaves nothing behind: what is kept goes on a volume. Images live under ~/.local/share/cell. A name with @sha256:... holds one image for good; a tag only moves on cell pull. Every manifest, blob and layer is checked against its digest, and a layer's own urls are never fetched.

A cell holds one uid (R53): the account that ran it, not root, which cell refuses. It owns every file, and is root inside, or the number the image's USER or --user names, so software that refuses to run as root is given --user. It cannot change to another uid, so an image whose start script drops privileges (su-exec, gosu, stock nginx's user nginx;) fails unless told to stay where it is. Setuid bits, devices and xattrs in an image do not land.

A published port is on 127.0.0.1 unless it names an address. Nothing else comes in. Going out, a cell reaches what its account reaches, the network and other machines included, except this machine's loopback: --reach 5432 opens one of its ports at the cell's own 127.0.0.1:5432, which is also how one cell talks to another. --no-net leaves it only -p and --reach. New namespaces, mount, bpf, ptrace and io_uring, among others, are refused by seccomp.

There is no way to build an image here, no bridge between cells, and no memory or CPU limit: those need CONFIG_MEMCG and CONFIG_CFS_BANDWIDTH, which Oath's kernel leaves off. PID 1 loads tun and opens /dev/net/tun at boot, from the release after 3e42879d38e5; on a machine that has not rebooted into it, do so by hand (modprobe tun, chmod 666).

pkg:route is Oath's own HTTP router (R59), in apps/route, with nothing but Go's standard library: by host and path, to a local service, a redirect or a fixed answer. It holds no TLS and belongs on a loopback address, behind a tunnel or other proxy:

{
  "listen": "127.0.0.1:8080",
  "routes": [
    {"host": "a.example", "to": "127.0.0.1:9000"},
    {"host": "a.example", "path": "/admin", "to": "127.0.0.1:9001", "strip": "/admin",
     "request": {"X-Forwarded-Prefix": "/admin", "Cookie": ""},
     "response": {"X-Frame-Options": "DENY", "Set-Cookie": ""}},
    {"host": "a.example", "path": "/gone", "status": 404, "body": "not found\n"},
    {"host": "old.example", "redirect": "https://new.example", "status": 301},
    {"host": "*", "status": 404}
  ]
}

The longest path wins, at a segment's edge, and * stands in for a host no route names. strip takes a prefix off, then prefix puts one on. An empty header value removes that header. A service gets the path that matched: repeated slashes collapse, and a path with a . or .. segment gets a 400. route <file> runs it, route -check <file> tries a file, SIGHUP reads it again and keeps the old routes if the new file is bad, SIGTERM drains. Websockets and streams pass through. A forwarded X-Forwarded-* is kept, so it trusts whatever reaches its address.

cf (also cloudflare) carries the npm packages it was pinned with in its own tree and runs on pkg:node (R46); nothing goes in a global node_modules, and npm install -g has nowhere to write, since the store is read-only. cf still prints a notice when npm has a newer cf; that is a new pin in pkgs/cf. gh runs with its update notice off and gh auth login keeps its settings under ~/.config/gh. Both send telemetry unless DO_NOT_TRACK=1 (or GH_TELEMETRY=false, CF_SEND_TELEMETRY=false).

Put a LAN mirror first to prefer it (any directory cargo make publish --dest wrote, served over HTTP):

sudo oath set host:local origins='["http://10.0.0.5:8080", "https://store.oath.moonlight.pm"]'
sudo oath apply host:local

An empty list fetches nothing without a url. A failed origin is skipped; apply fails only when none serves the pin, and says what each answered.

4.6.2Minecraft: Prism Launcher (R74)§

pkg:prismlauncher is Prism Launcher, with pkg:jre-21 (Java 21) and Qt 6, in no profile; each is on the origin as its own head. prismlauncher starts it in a box: it sees its own folder (~/.local/share/PrismLauncher), the display, sound, the GPU, pads and the network; not the rest of /home, and sudo does not work in it, so a mod cannot reach them. Mods and modpacks install from Modrinth and CurseForge in its window. Signing in shows a code and a QR Code to use on any device, at microsoft.com/link; nothing plays until a Microsoft account that owns the game is added. To bring a Modrinth App profile over, see Minecraft.

4.7Plans§

A plan (plan:<name>) is a hashed plan.plan file that builds a product tree for pkg:<name>: a JSON head (name, produces, build_needs, run_needs, each need pinned by hash) and a /bin/sh script that writes /out. oath build runs it in a sandbox with only the pinned needs mounted and no network. Its /bin and /lib are link farms of those packs, as on a machine, so a plan pins every pack its tools load: apps/hello.plan pins bash, coreutils, gcc and glibc, and gmp, mpfr and mpc, which they run with.

oath fmt -w apps/hello.plan
oath build --file apps/hello.plan
oath set plan:hello --from-file apps/hello.plan
oath build plan:hello
oath set pkg:hello present=true
oath apply

A missing build_needs pack is fetched first when its pkg object has a url. The second build of an unchanged input set is a no-op. The script's output goes to /oath/log/plan/<name>.log, not the terminal: oath build prints the path (and a failed build the last lines), and oath logs plan:hello reads it. The previous build's log is hello.log.1.

The build cache is keyed on the plan hash and the input set, so an edited script always rebuilds.

Known gap: the base image does not carry pkg:gcc, so apps/hello.plan builds on the build host (in ~/.local/share/oath), not on the box. The plans in pkgs/ are the larger examples.

4.8Recipes§

A package whose licence does not allow redistribution is a recipe (R23), and so is a coding agent (R31): the image holds its plan, not its pack, and the machine builds the pack itself from the vendor's own file. oath (INDEX.md) lists the recipes under Recipes until their package is present. Claude Code is one:

sudo oath build plan:claude-code         # fetches ~240 MB from downloads.claude.ai
sudo oath set pkg:claude-code present=true
sudo oath apply pkg:claude-code
claude --version                         # 2.1.292 (Claude Code)

claude doctor then warns that ~/.local/bin/claude is missing: that is where the vendor's own installer puts it, and Oath does not use it.

pkg:claude-code-src is the vendor's file: url is the file itself and file is where it goes in the tree (src/claude); hash pins that tree, so a changed download is refused. The plan builds with packs every machine has (bash, coreutils, grep, glibc) and must make the very pack the build host made: pkg:claude-code's hash is that pin, and apply refuses another. claude runs with DISABLE_UPDATES=1; a newer Claude Code is a new pin from the build host (cargo make update moves the plan and the source; a built pkg:claude-code is left at its version, with the commands to rebuild it). Its settings, login and history are under /home/.claude and /home/.claude.json.

Codex (OpenAI) and Grok Build (xAI) are recipes the same way (R31), though their licences would allow a pack:

sudo oath build plan:codex               # fetches 117 MB from github.com
sudo oath set pkg:codex present=true
sudo oath apply pkg:codex
codex --version                          # codex-cli 0.164.0

sudo oath build plan:grok-build          # fetches 57 MB from x.ai
sudo oath set pkg:grok-build present=true
sudo oath apply pkg:grok-build
grok --version                           # grok 1.0.70 (f50904eaee6c)

codex is OpenAI's package as it ships, with its own ripgrep and its own bubblewrap, which sandboxes the commands it runs; it starts with its update check off. codex login signs in; its settings and sessions are under /home/.codex. OpenAI signs the package with Sigstore, and the build host checks that signature. grok sets auto_update = false in ~/.grok/config.toml before Grok starts, since that is the only switch its updater has. grok login signs in; its settings are under /home/.grok. xAI publishes no checksum or signature, so the pin is what the build host downloaded. Grok looks for bwrap only under /usr, so on Oath it runs commands without its sandbox. A machine installed before these recipes gets their objects the way the update names them: oath new pkg:<name>-src --from-json and oath new pkg:<name> --from-json with the output's documents, and oath set plan:<name> --from-file pkgs/<name>/<name>.plan.

Steam is another, in the desk profile (R29): pkg:steam is Valve's launcher, from repo.steampowered.com, checked against Valve's signed .dsc. Its needs (bubblewrap, glibc for i386, libxcrypt) are present on the desk. A machine whose catalog lacks the objects (an update reports pkg:steam pkg:steam-src plan:steam) gets them as oath new pkg:steam-src --from-json … with the output's document and oath set plan:steam --from-file pkgs/steam/steam.plan. Then:

sudo oath build plan:steam               # fetches 20 MB from repo.steampowered.com
sudo oath set pkg:steam present=true
sudo oath apply pkg:steam
steam                                    # on the desk; DISPLAY defaults to :0

steam starts Valve's launcher in a sandbox that gives Steam the root it assumes (/usr, /lib64, /lib32, /sbin/ldconfig, an ld.so cache, /etc/ssl) and leaves the machine's root as it is. On a first run it opts into Valve's 64-bit client (the beta channel), then downloads the client (about 1 GB) into ~/.local/share/Steam, where Steam updates itself; Oath pins only the launcher. Its logs are ~/.local/share/Steam/logs/console-linux.txt; the sandbox's tree is $XDG_RUNTIME_DIR/oath-steam/.

oath update builds a recipe the owner built again, from the output's plan, before it changes anything, and moves it in the same apply: Steam when its plan or its needs move (the 32-bit Mesa), NVIDIA's driver when pkg:linux moves.

NVIDIA's driver is a third, in desk (R34), for a machine with an NVIDIA card. Its build compiles the kernel modules against pkg:linux-build, so an update that moves pkg:linux builds it again, to the new kernel:

sudo oath build plan:nvidia              # fetches NVIDIA's 420 MB .run from download.nvidia.com
sudo oath set pkg:nvidia present=true
sudo oath apply pkg:nvidia
nvidia-smi                               # once its modules load (the pass after services, or the next boot)

libglvnd sends GL and EGL to it, the Vulkan loader and libgbm (32-bit too) find it, and its 32-bit libraries serve Steam. Its DLSS DLLs are in /lib/nvidia/wine, beside libGLX_nvidia, where Proton finds them; a game also needs PROTON_ENABLE_NVAPI=1 %command% unless Proton turns NVAPI on for it. The closed driver runs on the GSP firmware in its package, which PID 1 puts where the kernel looks. A card's own settings (GSP off, power caps) belong in the machine's overlay (Kernel module options).

4.9Security audit§

cargo make audit (R48) matches every package pkgs/ builds, and the Rust crates in pkg:oath, against cve.org's CVE List V5 (a local mirror in ~/.local/share/oath/cvelist, fetched each run), OSV, NVD, CISA's Known Exploited Vulnerabilities and EPSS, and writes build/audit/latest.json. --offline keeps the mirror as it is and skips NVD; --name <pkg> scans one. What it matches each package by is pkgs/audit.toml: its NVD CPE, the names CVE records use, its upstream repo and release tag (OSV's commit query), an OSV ecosystem name, or why there is nothing to match; no plan reads that file, so an edit there moves no hash. A CVE listed under [<name>.not_affected], or one a pkgs/<name>/patches/*CVE-….patch fixes, is listed apart.

A CVE record can come days after the release that fixes it, so the scan also reads each pin against upstream's newest stable release: from its repo and tag by git ls-remote, else from the directory its source comes from. Each package gets newer, or upstream_unread and why; cargo make audit --upstream is that check alone (under a minute, and it writes nothing). series holds a package to a release line, dev_odd and dev_micro mark development releases, and unchecked says why a package cannot be read. A newer release is a prompt to read its notes, not a finding: only one that names a security fix moves the pin.

On buccina, svc:audit runs it daily at midnight through check/audit/run: a headless Claude Code run of the oath-audit skill, in a box (check/audit/box, R77), triages the hits, upgrades to upstream's fixed release (or applies upstream's patch, or a backport marked oath-backport-CVE-…), commits and checks. Outside the box the runner then lands those commits, pushes and publishes, mails Joshua the report and moves buccina to the release. The box holds no key but its own Claude token, ~/.config/oath/claude-token (mode 600), which lasts a year. Without it the run mails that it did not run; to make one, run claude setup-token and save what it prints there. Every run, fix or no fix, it also runs cargo make pkgs --stale, which lists each package built against something the lock has since replaced, and rebuilds those (R75); a command may last 6 hours.

4.10Host store cache and publishing§

The build host keeps the same layout under ~/.local/share/oath/store (or $OATH_STORE), outside the checkout:

cargo make store --name hello --from apps/hello --tar
cargo make advisories            # OSV lookup from components.json
cargo make audit --publish       # the audit's latest.json, signed, on the origin (R61)
cargo make publish               # upload to the default origin what it lacks
cargo make publish --dest <dir>  # or write the same tree into a directory

To the origin, publish goes only from a clean master that is origin/master (it fetches first), and only with an output built from a clean tree at a commit master has (R47): commit, push and cargo make build again otherwise; nothing overrides it. --dest is not checked.

publish carries every pack pkgs/lock.json records, build packs included (gcc, rust, python, …), and every package a seed profile names, each at the hash this tree records for it: the product in pkgs/lock.json, or pkg:oath's last cargo make build. Each needs a licence on record: lock.json's, or for pkg:oath the workspace licence AND every crate's in its components.json. Each pack goes out with its source: the plan that built it (plan/<name>/<plan hash>.plan) and the source pack the plan pins (pkg/<name>-src/<hash>.tar). It refuses, before writing anything, a package that is not built, has no licence, has no plan matching the record, needs a package that is not published, or whose plan builds with a pack not published at the hash it pins (the lock's, or one the origin already has from an earlier publish, as CEF's older Mesa): an origin alone rebuilds every plan on it. INDEX.md lists name, hash, licence, plan and source, and goes up last. A recipe goes out as its plan only, listed with the product hash it builds. Test packs and other hashes in the cache are never published; the last line names what was left out.

To the default origin, publish lists the bucket, uploads only the hashed files it lacks (pkg/ and plan/ never change) plus adv/ and INDEX.md, and reads INDEX.md back from https://store.oath.moonlight.pm. The write key is the build host's alone: OATH_ORIGIN_S3 (https://<account>.r2.cloudflarestorage.com/oath-store), OATH_ORIGIN_KEY_ID and OATH_ORIGIN_SECRET, from the environment or ~/.config/oath/origin.env (mode 600, never committed). They are an R2 S3 key: a Cloudflare account token allowed to read and write objects in oath-store only; its id is the key id and the SHA-256 of its value the secret.

Then publish puts out the newest output cargo make check passed (--output <id> for another), which machines update to (R38, Install): out/<id>/update.tar, out/<id>.json and out/<profile>.json, each .json with a .sig from the build host's signing key, ~/.config/oath/sign_ed25519 (OATH_SIGN_KEY; make one with ssh-keygen -t ed25519 -N '' -C oath@<host> -f ~/.config/oath/sign_ed25519). Machines take it only if that key's public half is in their host:local.publishers. It refuses an output whose packs the origin lacks (a pack pinned from elsewhere is its owner's to publish), and an output already published stays as it was, so publishing an older one again never moves a profile back.

Optional backup hooks in a pack (not catalog fields): libexec/oath-backup-quiesce and libexec/oath-backup-thaw. See Services.

5Network and SSH§

There is one object: net:net0. PID 1 renames the machine's only non-loopback NIC to net0. Do not hunt eth0 or ens3. Do not admin with ip / dhcpcd; apply converges with /bin/ip and /bin/dhcpcd.

IdDefault
net:net0up: true, ipv4: dhcp
oath get net:net0
sudo oath set net:net0 ipv4=192.168.1.20/24 gateway=192.168.1.1
sudo oath apply --confirm
ssh home@192.168.1.20 sudo oath keep

ping works as home: PID 1 opens ICMP echo sockets to every group (net.ipv4.ping_group_range), so it needs no setuid bit.

ipv4 is dhcp or a.b.c.d/n; set refuses anything else. Any change to up, ipv4 or gateway can cut your SSH session, so oath apply refuses it (exit 3) without --confirm, and once applied it is on probation: unless a new SSH session runs sudo oath keep within 180 s (--probation <secs>), or if the machine reboots first, PID 1 undoes it. A bad address rolls itself back. Serial still works.

On QEMU, user networking hands out 10.0.2.15 by DHCP and forwards host 127.0.0.1:2222 to guest port 22 (OATH_SSH_PORT). OATH_BRIDGE=br0 attaches the guest to a host bridge instead (needs qemu-bridge-helper and /etc/qemu/bridge.conf; no port forward).

With ipv4: dhcp, converge takes the first lease with a one-shot dhcpcd -1, and PID 1 keeps dhcpcd -B … net0 running so the lease renews (IPv4 only, no config file; Oath's hook /lib/oath/dhcpcd.hook writes /etc/resolv.conf; the processes that read the network run as void, chrooted in /var/empty). ps shows it as dhcpcd: net0 [ip4]; sudo dhcpcd -4 -n net0 makes it rebind now. That client is PID 1's, not a svc object; it restarts with backoff like a service, and its log is oath logs net:net0. Applying a static address stops it before the address changes; applying dhcp again starts it. Each lease and renewal the client gets shows in actual: lease is the address and dhcp has the router, DNS servers, lease length and when it was renewed (oath get net:net0 --actual).

5.2Names§

Names resolve through /etc/hosts, then the DNS servers the lease names (or that /etc/resolv.conf has): /etc/nsswitch.conf says hosts: files dns. A LAN machine the DNS does not know goes in host:local.hosts, which /etc/hosts lists after localhost and this hostname:

sudo oath set host:local hosts='{"novus": "10.0.0.5", "nas": "10.0.0.12"}'
sudo oath apply host:local
getent hosts novus

set replaces the whole map; oath undo takes it back. Do not edit /etc/hosts: every apply and boot writes it again. When the network's DNS can serve a name, add it there instead, for every machine at once. A machine-specific map belongs in the host's install overlay (hosts/<name>.json).

dig, host and nslookup (BIND 9, pkg:bind) ask the DNS servers in /etc/resolv.conf directly and never read /etc/hosts; getent hosts <name> shows what programs get. dig @<server> <name> asks one server, dig +trace <name> walks down from the root.

5.3Time§

svc:time is chronyd -d against 0–3.pool.ntp.org (decisions R12, R18), configured on its command line, with no config file; after start-up it runs as void, keeping only the right to set the clock. In its first three updates it steps a clock that is off by more than a second (makestep 1 3), then slews. Its log is chrony's own:

oath logs svc:time
… Selected source 192.0.2.10 (0.pool.ntp.org)
… System clock wrong by -102.361823 seconds
… System clock was stepped by -102.361823 seconds

To use other servers, change its exec (oath set svc:time --from-json '{"exec": […]}' and oath apply).

5.4SSH§

SSH login is home (uid 1, HOME=/home, login shell /bin/zsh). Root SSH is denied and passwords are off. sudo has no password. Serial is root when svc:serial is enabled. There is no private key in the image.

Host keys are generated on first boot under /oath/ssh/. Login keys are ssh:local authorized (OpenSSH public key lines); apply writes /home/.ssh/authorized_keys.

oath get ssh:local
sudo oath set ssh:local --from-json '{"authorized":["ssh-ed25519 AAAA…"]}'
sudo oath apply

Each key must start ssh-, ecdsa- or sk-. Adding a key is a plain apply. Dropping one (or emptying the list) can lock you out, so it needs oath apply --confirm and goes on probation: log in with a key that is still listed and run sudo oath keep, or PID 1 puts the old list back. Undo restores the last keys; it never rolls back the host keys under /oath/ssh/.

Disabling or removing svc:sshd, or changing its exec, user or env, also needs --confirm and goes on probation.

Where the first keys come from:

  • cargo make install writes the build host's public keys (~/.ssh/*.pub, default identities without a .pub, ssh-add -L, or OATH_SSH_PUBKEY) into ssh:local.
  • cargo make run / up / start pass the same keys to the QEMU guest, and PID 1 adds them to ssh:local (desired and applied) at boot. cargo make check passes its own throwaway key the same way.

svc:sshd is OpenSSH's sshd (pkg:openssh, not removable, with OpenSSL: RSA, ECDSA and ed25519 keys log in), which also gives /bin/ssh, /bin/scp, /bin/sftp, /bin/ssh-keygen and /bin/sftp-server. Logins are home's, by key; passwords and root are refused. Before a login, sshd reads the network as void, chrooted in /var/empty. oath writes /etc/ssh/sshd_config from host:local with /etc/profile, so a command over SSH (ssh host 'cmd', no login shell) gets host:local env and TZ too, as of sshd's last start. The host key is /oath/ssh/ssh_host_ed25519_key; a machine that ran dropbear keeps its key. Guest /etc/ssh/ssh_config is StrictHostKeyChecking accept-new. scp and sftp as home work:

scp ./file home@host:/home/file
cargo make ssh                # QEMU: ssh -p 2222 home@127.0.0.1
cargo make ssh -- ls /        # extra ssh(1) args

6Generations§

Undo is a filesystem snapshot, not a journal.

6.1Layout§

The system disk is btrfs.

SubvolumeRole
@Live root, mounted at /
@gen-NRead-only snapshot of @ taken before apply N converges
@boot-NSnapshot of @ taken when cargo make esp archives a boot

PID 1 mounts the btrfs top level (subvolid=0) at /oath/run/fs:

/oath/run/fs/@          same tree as /
/oath/run/fs/@gen-1     generation 1
/oath/run/fs/@gen-2     …

Generations are siblings of @, not nested under /.

A machine keeps the newest five generations and every one taken since it booted; PID 1 removes the rest once a boot (oath reap, R62, from the next release), and undo stops at the last one kept.

A generation is all of @, so every apply pins whatever is there until that @gen-N is gone. Keep fast-changing bulk, such as a cargo target/, on another disk or in a nested subvolume, which a snapshot leaves out.

snap:current holds the last snapshot id. snap:N is a read-only record (generation, parent, time, reason) in its desired document. Do not set snap:N.

A generation holds spec only: status lives on the /oath/status tmpfs, so no @gen-N or @boot-N carries it, and an archived boot observes its own.

6.2Apply and undo§

oath apply first checks everything it can without touching the machine: every selected object parses as its kind, confirm-class changes have --confirm, there is no wants cycle, and the package needs graph holds. A refusal takes no generation, changes nothing, and is logged with "result":"refused". Then it snapshots @ to @gen-N (holding what was applied before, not your pending edits) and converges. N is never reused (it is max(existing)+1). The log line says ok, or error if converging failed after the snapshot.

oath undo works like a stack. It takes the newest apply, ok or error, that has not been undone yet, and puts back the spec of the objects that apply touched from its @gen-N. An object that apply created (oath new) is removed; one it removed (oath rm) comes back. Then it converges those objects (hostname, pkg:* links, net:net0, ssh:local, dev:*, and svc:* through PID 1), sets snap:current back, and logs undo:N. Run it again to step back one more apply.

Undo does not touch the apply log, the package store, the SSH host keys, or objects the apply did not touch. It keeps a package that the image fixes even when that apply created it (cargo make update makes one a machine lacks): the image's own files need it. A pending oath set on some other object survives an undo.

An apply that can cut off SSH is on probation (/oath/probation.json) until a new session runs oath keep. If its time runs out, or the machine boots again first, PID 1 runs the same undo itself (logged with tty probation).

A reboot or halt through the catalog (power=reboot) on its own takes no generation: power is not configuration, so there is nothing for undo to take back. It is logged ok without a generation, and PID 1 shuts down cleanly. Applied together with other changes, it is part of their generation. Plain sudo reboot reboots the same way.

6.3Boot menu§

On metal, EFI/BOOT/BOOTX64.EFI is oath-efi. It sets the native GOP mode, paints the Oath mark, and shows a 5 s menu: the current kernel plus the last five archived kernel+initrd pairs. Up/Down/Enter. An archive boots oath.subvol=@boot-N, the userspace snapshot taken when it was archived. oath-efi is the only loader; there is no fallback loader on the ESP.

6.4What the boot screen says§

Under the mark, a line says what the machine is doing and a bar moves while it works. A machine that stops booting shows where:

  • Reading the kernel or Reading the initrd, with the bar still: the firmware cannot read the disk; try another port.
  • Starting Linux 7.3.1 and nothing more: the kernel stopped before Oath's PID 1 ran.
  • Loading drivers: <name> with the bar still: that driver hung.
  • Waiting for the root disk … Disks: …: the disk Oath runs from has not appeared; the list is what the kernel sees.
  • Amber text: the boot failed, and says why. A shell waits on the serial console.
  • Oath is running, with no driver for this … display (vendor:device). Reach it with ssh home@<address>: the boot finished, but Oath has no driver for the graphics card.

Once a display driver loads, the screen is the desk's or tty1's.

cargo make esp --esp <vfat node> --confirm [--root /dev/sda2] [--output <id>] takes the kernel, initrd and oath-efi of a build output (default: the newest one cargo make check passed; --unchecked for another), archives the live ESP kernel as a new boot, keeps five, snapshots @ to @boot-<id>, and copies the new bits. It does not format anything. It mounts the node you name on the machine where you run it, so it refuses to run on a machine that is not Oath (no /oath/INDEX.md), such as the build host.

QEMU cargo make run / up / start boot with -kernel and have no menu. cargo make install --qemu boots through OVMF with a 0 s timeout.

qcow2 snapshots on the QEMU host are debugging aids, not this mechanism.

7Build and QEMU§

Everything is built on the build host with cargo make (crate oath-make), from a plain shell.

Built here, from source: the kernel; the toolchain (GCC, binutils, glibc); every package in the image (pkgs/); PID 1 (oath-init, serial-login, sudo), oath, and the UEFI loader oath-efi (pkg:oath, built by the vendor's Rust release, pkg:rust). On a NixOS build host, Nix gives QEMU and its UEFI firmware for running images (image/host.nix), and the host compilers that build stage 0 of the toolchain and oath-make itself. None of it ships. An Oath machine can be the build host too (R32): its own gcc builds stage 0, its pkg:rust builds oath-make, and pkg:qemu (in no profile; add it to the machine) runs images, with the edk2 firmware from QEMU's release as OVMF. home has /dev/kvm there (R33). Without it, build, check and the QEMU rehearsals stop and say so. QEMU loads libslirp, which no profile has either, and a pack's needs must name the machine's packs at the hashes it runs with: those are run_needs in its plan on the origin (INDEX.md gives each pack's hash and plan).

curl -fsS https://store.oath.moonlight.pm/INDEX.md | grep -E '`(qemu|libslirp)`'
sudo oath new pkg:libslirp --from-json '{"present":true,"hash":"sha256-…"}'
curl -fsS https://store.oath.moonlight.pm/plan/qemu/sha256-….plan   # its run_needs
sudo oath new pkg:qemu --from-json '{"present":true,"hash":"sha256-…","needs":[…]}'
sudo oath apply pkg:libslirp pkg:qemu

7.1The toolchain and packages§

cargo make bootstrap        # once: stage 0 + stage 1 toolchain (an hour or more)
cargo make pkgs             # every package: the base until it settles, then the rest
cargo make pkgs -j 4        # the same, four packages at a time after the base
cargo make pkgs --list      # versions, licences, what pkgs/lock.json pins
cargo make pkgs curl        # one package (and what it builds with)

bootstrap builds a cross binutils, GCC and glibc with the host's gcc (stage 0, never shipped), then with stage 0 the base packs (stage 1): pkg:gcc (GCC and binutils), pkg:glibc, and bash, coreutils, sed, grep, gawk, findutils, diffutils, patch, tar, gzip, xz, bzip2 and make. pkgs builds each pkgs/<name>/ as an oath build plan in the sandbox (no network, only the declared packs, glibc's loader and every pack's libraries under /lib), writes the exact plan to pkgs/<name>/<name>.plan and the product to pkgs/lock.json. Every plan builds with the base packs; the base packs rebuild themselves this way too. Build-only packs (python, meson, ninja, m4, flex, bison, bc, perl, elfutils) are not in the image. pkg:rust is the vendor's Rust release (pinned by sha256, its binaries pointed at /lib's loader); it builds pkg:oath, static against glibc. The image takes every userspace package from the lock. How packages are written: pkgs/README.md.

7.2The kernel§

There is one kernel, pkg:linux: vanilla kernel.org Linux, pinned by sha256 in pkgs/linux/pkg.toml, plus pkgs/linux/patches/ and the config fragment pkgs/linux/linux.fragment over defconfig. It is a plan product like every other package, built by pkg:gcc with the flex, bison, m4, bc, perl and elfutils (libelf, for objtool) packs, which are build-only.

cargo make pkgs linux

The product holds boot/vmlinuz, boot/config and lib/modules/<release>/ with the depmod index (modules.dep, modules.alias, modules.softdep) written from each module's .modinfo. cargo make takes it from pkgs/lock.json (or OATH_KERNEL / OATH_MODULES). Firmware for the initrd comes from pkg:firmware, not the kernel build. Packing refuses a bzImage whose release does not match the module directory or the vermagic of btrfs.ko. Bump the version and the sha256 together (pkgs/glibc takes its UAPI headers from the same tarball).

7.3The image§

Needs: the packs in pkgs/lock.json (pkg:rust among them) and mkfs.btrfs and unshare on the host (shell.nix has them); no root. The build host's Rust compiles only oath-make.

cargo make build                  # profile base → build/out/<id>/

build builds pkg:oath from this tree: it packs the workspace and every crates.io crate in Cargo.lock (fetched once into build/fetch, checked against the lock's sha256) and runs cargo build with pkg:rust in the sandbox, offline. A tree that has not changed is not rebuilt; cargo make pkgs oath does this step alone. Then build writes the boot initrd and the installer initrd, stages the root tree, seeds /oath from the profile, packs the profile's packages into the hashed store (also cached under ~/.local/share/oath/store), and writes a build output:

build/out/<id>/
  manifest.json          what the image ships; its hash is <id>
  oath.qcow2             btrfs, subvolume @
  bzImage  initrd.gz  BOOTX64.EFI (oath-efi)
  initrd-install.gz  initramfs-install/   the installer ramdisk
  stage/                 the root tree install copies
  check-pass             written by a clean `cargo make check`
build/out/latest -> <id>

A build packs into build/out/.<id>.part/ and renames it when it is complete; after that only check adds check-pass. The id is the first 12 hex digits of the manifest's SHA-256 (without the commit), so a tree that has not changed reuses its output in seconds. The newest five outputs are kept, and the newest checked one.

manifest.json lists every pack the image ships: its hash, version, licence and plan hash; where it ships (rootfs for the profile's store, initrd, installer for the install initramfs's own tools, esp); and its sources, each with its sha256 and signing key (pkgs/keys/…) or why it is unsigned. pkg:oath records the commit of this tree instead, with -dirty when its sources had uncommitted changes.

A profile can pin a package Oath does not build (R28): its entry has hash and needs (and url if it is not on the profile's origins), and there is no pkgs/<name>/:

"pkg:hello": {
  "present": true,
  "hash": "sha256-…",
  "needs": [{ "id": "pkg:glibc", "hash": "sha256-…" }]
}

When the profile ships it (present), build fetches it into ~/.local/share/oath/store and the manifest marks its entry "pinned": true; the manifest's pinned lists every package the profile pins this way, shipped or not. When a need's hash is not the one the image ships, the manifest lists it under stale_needs, the image takes the shipped hash, and build, check and update print a warning: its owner should build it again. publish leaves it out, and update keeps whatever pin the machine has for it, moving only its needs.

7.4Test it§

cargo make check              # every case in check/cases.toml
cargo make check --build      # pack first
cargo make check --case index --case pkg-fetch

Each case boots its own VM from the latest output's oath.qcow2 (--output <id> for another; four at a time) and runs its steps over SSH as home. The image is the one install ships; a case makes any fixture it needs with oath new. The summary marks each case PASS, FAIL, GAP (a known failure; the gap line names the phase that fixes it) or FIXED (a gap that now passes; remove its gap line). Logs, the serial console of every boot and report.json are under build/check/<time>/<case>/. A clean full run writes check-pass into the output it booted, which cargo make install requires.

cargo make check --target home@10.0.0.108

--target runs the cases marked metal = true on an installed machine instead: one at a time, as home, trusting only the host key that install pinned in build/known_hosts. The machine must start in-sync (oath diff). After each case the harness runs oath undo back to the generation the case started from and requires oath diff to say in-sync again; if it does not, the run stops there. A reboot step waits up to five minutes for a new boot id. A case that needs hardware (needs = { what = "a connected display", run = "…" }) runs that test first and is SKIPped on a machine that fails it: the desk's display and Sola cases on a headless machine, the RADV case without an AMD GPU. QEMU always has what they need. Logs are under build/check/<time>-<host>/. No stamp is written.

7.4.1A real model§

ANTHROPIC_API_KEY=… cargo make check --agent              # every task, haiku
cargo make check --agent --case rename --model sonnet
cargo make check --agent --target home@<host>             # on a machine

--agent gives a model that has never seen Oath one MCP server, the machine's ssh -T home@… oath mcp, and each task in check/agent.toml (rename the machine, run a service, undo a mistake, fix a failing service, reboot), then checks the machine over SSH. The client is Claude Code, headless, with no shell, files, memory or CLAUDE.md; it is a template in check/agent.toml, so another client can run the same tasks. A task fails if its checks fail, if a change in the apply log did not come through MCP, or if the model passed confirm where the task did not ask for it. It is opt-in and never gates install: it costs money (the budget per task is in the file) and a model is not deterministic. The key is ANTHROPIC_API_KEY from the environment; it refuses to run without one and writes it nowhere. Results are in build/out/<id>/agent-check.json (task, model, pass, cost, turns; a run of some tasks replaces only their entries), transcripts in build/out/<id>/agent/<task>.jsonl, and each VM's serial console and steps under build/check/<time>-agent/<task>/; with --target, all of it is under build/check/<time>-<host>-agent/.

7.5Run it in QEMU§

cargo make run            # interactive serial on this terminal
cargo make run --build    # pack first (same flag on up / start)
cargo make up             # headless, foreground; Ctrl-C kills QEMU
cargo make start          # headless, background
cargo make ssh            # ssh -p 2222 home@127.0.0.1
cargo make stop

Each run gets build/runs/<id>/ with a qcow2 overlay (the built image is never changed), serial.log, and meta.json. Every mode passes the build host's SSH public keys to the guest. QEMU uses KVM when /dev/kvm is there, 4 GiB of RAM, virtio disk, net, and rng.

The serial console is a root shell; SSH is home. Then:

oath
oath ls

When DISPLAY is set, run also opens a gtk window on the guest's virtio-gpu: its screen, a shell as home (svc:screen). OATH_DISPLAY=none hides the window.

7.6Leave the VM§

Ctrl-D leaves the shell; svc:serial is restart=always, so PID 1 starts a new one. There is no QEMU monitor, and Ctrl-C goes to the guest under run. To stop the VM, power it off from inside:

sudo poweroff

(or oath set host:local power=halt and oath apply --confirm). PID 1 stops the services, syncs and remounts / read-only first.

run also exits when the guest reboots (-no-reboot). If the prompt is dead, pkill -f qemu-system-x86_64 from another terminal.

7.7Telemetry§

PID 1 and oath write JSON lines prefixed oath-tel to stderr, and to /oath/log/*.jsonl once the disk is mounted.

8Install§

Wipe a named disk and copy the packed Oath tree onto it.

cargo make build
cargo make install --target user@host --disk /dev/sda --confirm \
  --hostname canto --overlay hosts/canto.json

--confirm is required. --disk is the whole disk (GPT). The live OS on that host must be reachable over SSH as root, or as a user with sudo -n. Every command is fed to the target's /bin/sh on stdin and files are written with dd, so the user's login shell does not matter.

8.1What it does§

  1. Picks a build output: --output <id>, else the newest one that passed cargo make check (its check-pass must match the kernel, initrd and image). --unchecked takes the latest output without that; --qemu rehearsals take the latest and are not gated. Everything below comes from that one output; the build host's keys are baked into a copy of its installer ramdisk under build/runs/<time>-install/, never into the output. Reads and checks --overlay against a scratch catalog, before anything is wiped, and refuses a --target that is the machine it runs on (the same boot_id).
  2. Boots an installer ramdisk on the target (oath.install=1, OpenSSH's sshd, the build host's keys for root). On EFI machines this is a one-shot loader entry; kexec is the fallback (Oath's own static kexec, copied to the target, so the live OS needs none).
  3. Partitions GPT with the ramdisk's native tools (wipefs, sfdisk): a 512 MiB ESP (mkfs.fat), then btrfs (mkfs.btrfs) with subvolume @ and a new UUID, which the boot entry names the root by (oath.root=UUID=…): the disk's node changes with the order disks are found in.
  4. Copies the packed root tree (GNU tar).
  5. Merges the catalog overlay with the ramdisk's oath (oath --root /mnt/oath seed --overlay): host:local.hostname (default oath), the build host's public keys as ssh:local, then your --overlay on top. Makes the new system's SSH host key (/oath/ssh/host_ed25519) and pins it in build/known_hosts.
  6. Writes the ESP: oath-efi as EFI/BOOT/BOOTX64.EFI (the only loader), vmlinuz, initrd.gz, loader/entries/oath.conf, and loader/oath-boots. The kernel and the initrd modules are the same checked build.
  7. Reboots and waits for SSH as home, accepting only the pinned host key, then runs oath ls.

net:net0 stays dhcp from the seed.

8.2The overlay§

Machine-specific facts live in an overlay, not in the seed. Each object is a JSON merge patch on the seeded desired; a new id creates the object. See Catalog.

{
  "objects": {
    "host:local": { "timezone": "MST7MDT,M3.2.0,M11.1.0" },
    "svc:serial": { "enabled": false },
    "svc:backup": {
      "exec": ["/lib/oath/backup-daily", "nas:/export/backup/canto"],
      "wants": [], "restart": "always", "enabled": true
    }
  }
}

An overlay that sets ssh:local.authorized replaces the build host's keys rather than adding to them. Each patched object must pass the same checks as oath set, and the overlay counts as applied: the machine boots with it, no oath apply needed (except to link a package an overlay turns on).

8.3Check the installed machine§

cargo make check --target home@10.0.0.108

Runs the harness cases marked metal on the machine, one at a time, over SSH with the key pinned at install. See Build.

8.4Rehearse in QEMU§

cargo make install --qemu --disk /dev/vda --confirm

That runs the same steps against a fresh qcow2 in QEMU with OVMF, then boots it through oath-efi.

8.5Onto a USB disk plugged into this host§

cargo make install --local --disk /dev/sdd --confirm \
  --hostname novus --overlay hosts/novus.json

For a machine that boots Oath from a USB disk, the build host itself included. The disk must be removable, USB and not mounted. The installer ramdisk runs in QEMU with the raw disk as its own (/dev/vda; the disk is lent to your user for the run), does steps 3 to 6, then OVMF boots the disk once, oath ls must answer, and it is powered off cleanly. Its loader waits 5 s, as on metal. Boot it from the firmware's boot menu; nothing changes the firmware's boot order. Such a stick is also an installer: oath install (below).

8.6From a stick, on the machine: oath install§

Boot an Oath stick (the one above) on the machine, open a terminal:

oath install                          # the disks: node, size, port, model, serial
sudo oath install /dev/sdb            # what it would erase and write; writes nothing
sudo oath install /dev/sdb --confirm  # wipe it and install
sudo poweroff                         # then remove the stick and start the machine

The disk is a node (/dev/sdb, sdb) or a serial; sudo oath install also shows each filesystem's type and label. Node names follow the order the kernel found the disks in and can differ from another system's, so check the model and serial the preview repeats.

--confirm wipes that disk: a GPT with a 512 MiB ESP and btrfs for the rest, with @ (the system) and @home (mounted at /home, outside every generation and boot snapshot; the first boot adds @work, at /work, likewise). It copies the system the stick runs: its packages (NVIDIA's driver too, if the stick built it), its settings (hostname, overlay, SSH keys it trusts) and its release, at generation 0, with an empty home, a new SSH host key (printed at the end), and none of the stick's history, password or pairings. It writes the ESP, adds Oath to the firmware's boot list first and removes the other systems' entries; the firmware's own (USB, network, setup) stay, and nothing else in the firmware's variables is touched.

Remove the stick only once the machine is off: the system runs from it until then, and reboot with the stick gone finds no programs. Some firmware keeps its own order or makes entries again: AMI's (an ASUS board) kept UEFI OS, its entry for another disk's fallback loader, and did not start Oath first. If the machine starts something else, move Oath first in the firmware's setup.

It checks no backups and reads no other disk: knowing what is on them is yours. It refuses only the disk the stick runs from and a disk with something mounted. Agents run the same over SSH (--json for each step).

8.7USB installer stick§

cargo make install --usb --disk /dev/sdd --confirm

The disk must be a removable USB device. Boot the stick's EFI entry (hold Option on a Mac). The ramdisk gets a shell on tty0 and DHCP on the NIC with carrier. Then run the --target command from the build host; it sees /run/oath-install/ready and skips step 2.

8.8Update an installed machine§

Each machine updates itself from its origins (R38). The build host publishes every output it checked (cargo make publish, Packages); on the machine:

oath update                           # the newest output for this machine, and what moves
sudo oath update --confirm            # move to it; PID 1 re-executes into its binary
sudo oath update --confirm --reboot   # move to it, then boot it

It takes the newest release for host:local.profile from the first of host:local.origins that serves one a key in host:local.publishers signed (the base profile trusts buccina's), and refuses one older than the release the machine runs (/oath/output.json, written by the install too), or one it cannot order against it, unless --output <id> names it. It also refuses pending changes and an apply on probation, and a set of moves that would leave a package needing another at a hash it will not have (the check oath apply makes, made first, so the preview says it too). The preview names each overlay, a package the machine runs at a pin other than the release it runs set (R47): pkg:sola is sessions-anywhere@0d99e340, not the release's: the update replaces it with sha256-… (master@…). The update replaces it; --keep pkg:sola (repeat for more) keeps the machine's pin, and only its needs move. With --confirm, it:

  1. fetches every pack it will pin that the store lacks, and builds each recipe the owner built (Steam, NVIDIA's driver) from the output's plan, before it changes anything: a fetch or build that fails (a network error is tried twice more), or a build that is not the output's pin, leaves the machine as it was, and running it again fetches the rest. Then no two packages as the moves leave them may ship one name in /bin or /lib (a package kept with --keep that the output split in two), or it stops there;
  2. without --reboot, has PID 1 re-execute into the output's binary (below), then snapshots @ as @boot-<n>;
  3. writes the output's accounts (/etc/passwd, group, shadow) and /var/empty, since services that restart next drop to void (R36);
  4. sets each package the machine has to the output's pin, needs and source; makes each package and service new to the profile since the release the machine runs (its record lists that release's objects, seed; a record from before R47 tells only for the packages pinned from elsewhere, pinned), each package the output fixes (the image's files need it: /etc/passwd names pkg:zsh's shell) and each one a package it has present needs, down their needs (R41's pkg:libglvnd); files each plan new to the profile; and runs oath apply once (a name in /bin or /lib that moves from one package to another, libEGL.so.1 there, moves in it whichever converges first). oath undo takes it back but keeps a fixed package. An object the release the machine runs had and the machine lacks was its owner's to remove and is not made again; one new to the profile that the machine has absent (oath build leaves a product it made so) is made present, never the other way; any other the output's seed has is listed, for oath new. A package the machine's output had and this one does not is removed, unless --keep names it or something that stays needs it (R62, from the next release); one its owner added keeps its pin. A recipe the owner built moves to the pin step 1 built. Each running service whose program moved restarts on it, except one with a seat (the desk's and the console's, R42), which keeps its program until it next starts: the desk takes its new programs at a reboot, or at Exit Sola and oath desk. With --reboot no service restarts, and the boot starts each on its new program. When sshd's program moved, the apply is on probation: the services restart then (the seat's still keep theirs), and the update waits until you open a new SSH session and run sudo oath keep there;
  5. replaces /lib/oath, the image's static /etc files, /sbin/init and /bin/sudo;
  6. archives the ESP's kernel and initrd as boot <n> (with the snapshot from step 2) and writes the new kernel, initrd and oath-efi, keeping five archives (--esp, default partition 1 of the root disk);
  7. with --reboot, reboots; without it, says what waits for a boot (a new kernel, a PID 1 that could not re-execute, and what PID 1 does only at boot). oath update afterwards says whether PID 1 is the output's. An oath from before R47 that made the update wrote the record without the release's commit, pins and seed; oath update says so, and sudo oath update --confirm (the same release) writes the record as signed. Objects that release's seed has and the machine lacks are, from then on, the owner's to add (oath new).

A new PID 1 without a reboot (R73). When the output's PID 1 is not the one running, an update without --reboot asks PID 1 to re-execute into the output's binary before it applies anything, so the new PID 1 is the one that restarts what moved. PID 1 checks the binary's hash, lets it read a copy of its state first, and refuses one that cannot; every service keeps its process. The update prints PID 1 runs …, and --json says "pid1": "re-executed". If the update stops after that, PID 1 goes back to the binary it ran. A new kernel still takes a reboot, and so does what PID 1 does only at boot (mounts, modules, sysctls, device modes, /etc). A PID 1 from before R73 is not asked: its first update to one takes a reboot, as before.

A machine that still runs dropbear as svc:sshd (from before R36) moves to OpenSSH after it has booted the new PID 1, which writes /etc/ssh/sshd_config and carries dropbear's host key over: run sudo oath update --confirm again, then sudo oath keep from a new SSH session. If none gets in, PID 1 undoes it and dropbear serves again.

An oath from before step 1 (pkg:oath sha256-6e563f2d6c69… and older) fetches during the apply instead; if a fetch fails there, the machine is half moved and oath update refuses the pending changes. sudo oath apply finishes them; the update run after it snapshots the half-moved @ as @boot-<n>, so move the first one aside and back if the archived boot should keep the old userspace. Two builds of step 1 (pkg:oath sha256-e3790fbd2f8b… and sha256-fa9f700146a9…) also pinned each pack as they fetched it, so the apply skipped a package that had been applied before and left it unlinked: on a machine with either, take the newer oath by hand first, as above, then update.

An oath from before R42 (pkg:oath sha256-618db152642b… and older) does not make the new packages the others need, nor hand /lib's names from one package to another, so its apply refuses: nothing converges, but the specs it set stay as drift, and the next update sets them again. An oath from before 2026-10-30 (pkg:oath sha256-92735e24e583… and older) does not build a recipe the owner built, so a Steam pinned to the old 32-bit Mesa makes its apply refuse the same way (canto, 2026-10-29), and it restarts the desk under the person at it, --reboot or not. Take the newer oath (desk output fed6fb141a5e's) first, then update:

sudo oath set pkg:oath hash=sha256-a937565c5f8a5a210f233f9ec58c56ec1ed844e93fc5847fb2edddedbdf890a5 && sudo oath apply pkg:oath
oath update && sudo oath update --confirm --reboot

A machine from before R38 has no oath update yet. Give it the newer oath first (the pin is pkg:oath's hash in the output's seed, oath get pkg:oath on any newer machine), then the profile it follows and the key it trusts:

sudo oath set pkg:oath hash=sha256-… && sudo oath apply pkg:oath
sudo oath set host:local profile=desk \
  publishers='["ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIH+YKD3yXYsGJMxwpBIWSYP7VuQvjhzGG+LWvYhKTESD oath@buccina"]'
sudo oath apply host:local
sudo oath update --confirm --reboot

To try an output first, cargo make update --qemu <older output> installs the older one in QEMU and runs the same update from a local origin on the build host.

If the new boot fails, pick the archived boot in the firmware menu: it is the old kernel with the old userspace.

cargo make esp does steps 2 and 6 on the Oath machine itself, for an ESP you mount there. See Generations for why it must not be pointed at the build host's own ESP.