From 68e32c68398156425ad9ef8ac33d51ad4b3da3b1 Mon Sep 17 00:00:00 2001 From: Dai Ha Date: Sat, 29 Aug 2026 06:26:06 +0700 Subject: [PATCH] =?UTF-8?q?Features:=20memberHerdrSocket=20=E2=80=94=20mem?= =?UTF-8?q?bers=20on=20a=20second=20herdr=20daemon=20(#185/#186/#187/#188)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- 11-Features.md | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/11-Features.md b/11-Features.md index a35de78..fcff40d 100644 --- a/11-Features.md +++ b/11-Features.md @@ -2171,6 +2171,49 @@ under the two names above in `${SHARED_ENV}/tools/secrets.sh`. Also: the skill f `sed -E 's#://[^@]*@#://@#g'`, and the `g` is not optional — without it a line holding two URIs leaks the second one, which `scripts/redeploy-fleetd.sh --check` prints. +## `memberHerdrSocket` — run members on a second herdr daemon + +**What.** A new top-level config key. Set it to a second herdr API socket path and every **member** +is spawned on that daemon, while everything the **lead** does stays on the first one. Absent (the +default), both are the same client and nothing changes. Inside the daemon a `HerdrRouter` owns the +split: it hands each consumer the lead client, the member client, or — for consumers that see both +kinds of traffic — routes per target id. + +```yaml +memberHerdrSocket: /run/fleet/members.sock # omit for one daemon (the default) +``` + +**On.** Add the key and restart. It is read once at startup, so it is a deferred key, not a hot one. + +**Why.** herdr forks every pane as **its own** OS user, and there is no user, uid or run-as parameter +anywhere in its socket API — measured against herdr 0.8.0's own docs, not just our client. So the +only way to make a member run as a different user than the operator is a second herdr server started +by that user. That matters because a member currently reads the operator's ssh key straight off the +filesystem: blocking `SSH_AUTH_SOCK` does not stop it, and no environment scrub can, because the +scrub removes variables and the key is a file (#184). A separate OS user is the control; this key is +the fleetd half of it. Design and measurements are in #185. + +**Gotcha — the feature is not ready to switch on yet.** Four things must land first, and #185 keeps +the running list. The two that bite hardest: herdr creates its API socket mode `0600` and `umask` +does not change it, so cross-user access still needs a post-start `chmod g+rw` (a race on every +start); and pane ownership lives only in memory, so after a restart every surviving pane is unowned +and `fleet_stop` on it refuses rather than guess which daemon owns it. + +**The shape to watch for.** herdr's workspace, tab and pane ids are **per-daemon sequential +counters**. Two daemons really do both hold `w1:p1`, pointing at different panes owned by different +users — measured, not theorised. Four separate defects came from code that assumed one daemon, and +every one of them passed the whole test suite, because with the key unset both clients are the same +object. When reviewing anything on this path, ask each herdr call one question: *which daemon does +this go to, and is that the daemon that owns the thing it is asking about?* + +**Observable change even with one daemon.** `GET /healthz` now requires **both** daemons to answer +before it returns 200 (with one configured that is the single ping it always was). `GET /sessions` +merges `workspace.list` across both. The 200 body still reports only the lead daemon's herdr version +and protocol — a known gap, on the #185 list, and it matters because in two-daemon mode it is the +*member* daemon's protocol that decides whether spawns work. + +--- + --- ## Backfill status