fleetd #360: deploy units that do not silently disable the daemon

11-Features.md

@@ -4609,3 +4609,54 @@ than closing a live session on evidence that has already been seen to lie.
 
 Measured on merge: 1412 tests green; making `PendingCloseMarker.strip()` the identity function
 fails 4 tests. fleetd #359.
+
+---
+
+## The shipped systemd units no longer disable the daemon they start
+
+**What it does.** `deploy/fleetd.service`, `deploy/herdr.service` and `deploy/herdr-inner.sh` are
+the units and helper script that actually run fleet01. Copy them, edit the paths in the headers,
+`systemctl --user enable --now` both. fleetd starts from a login shell, neither unit takes a mount
+namespace, and neither gets a private `/tmp`.
+
+**The knob that turns it on.** Nothing to switch on — these are the deployment artefacts. What
+matters is what must stay *off*, and the files say so in their own comments.
+
+**Why it exists.** The old `deploy/fleetd.service` carried `ProtectSystem=strict`,
+`ProtectHome=read-write`, `ProtectKernelTunables=true`, `ProtectControlGroups=true`,
+`PrivateTmp=true`, and an `ExecStart` that ran `java` directly. Every one of those looks like good
+hardening and each breaks the daemon silently:
+
+- Each of the four `Protect*` directives gives the unit its own **mount namespace**. fleetd
+  resolves a caller's role by running `lsof` to find the loopback peer PID
+  (`mcp/LsofPeerPidLookup`). Inside such a namespace `lsof` returns nothing, every caller falls
+  back to ANONYMOUS, and the primary is refused every orchestration call with *"unauthenticated:
+  anonymous may not SPAWN"*. The daemon still starts. `/healthz` still returns ok. The only
+  symptom is that the fleet cannot be driven at all. Bisected on fleet01 (lsof line count): no
+  sandbox 3, `ProtectSystem=strict` 0, `ProtectHome=read-only` 0, `ProtectKernelTunables` 0,
+  `ProtectControlGroups` 0, `RestrictSUIDSGID` 3, `NoNewPrivileges` 3 — so the last two are safe
+  and are kept.
+- `PrivateTmp=true` must be false on **both** units. fleetd writes the member ZDOTDIR
+  credential-scrub directory and the opencode config directory under `java.io.tmpdir`, and the
+  member pane — a child of the *other* unit — has to read them back. A private `/tmp` turns the
+  credential scrub into a silent no-op.
+- systemd runs no login shell, and every secret the daemon needs lives in a file only the login
+  shell sources. Started with a bare `ExecStart=java`, fleetd boots fine with empty credentials
+  and the failure appears hours later as a member that cannot open a pull request.
+
+`deploy/herdr.service` did not exist at all, even though `fleetd.service`'s `After=`/`Wants=`
+already named it. fleetd #360.
+
+**One thing to know for maintenance.** A unit file has no compile step, so `SystemdUnitSafetyTest`
+is the guard: it reads all three files and fails on an active forbidden directive, on
+`PrivateTmp=true`, on an `ExecStart` that skips the login shell, on a `herdr-inner.sh` that does
+not exec a login shell, and on one that does not set a **non-zero** pty size (`stty rows N cols M`
+with both positive — a 0x0 pty makes every pane spawn fail with `ghostty error -2`). Each failure
+message names the consequence rather than the rule, because the rule alone is what someone deletes.
+
+A commented-out mention inside the file's own DO-NOT-add block must **not** trip the test — that
+comment is the whole point of the ticket, and a vacuity guard pins that it is still there, that all
+three files exist, and that none is trivially small. Measured on merge: 1420 green; adding
+`ProtectHome=read-only` to `herdr.service` fails 1 test; removing the login shell from
+`herdr-inner.sh` fails 1; removing its `stty` fails 1; deleting that file errors 3 and fails the
+build rather than passing vacuously.