fleet/fleetd
3
0
Fork 0
Code Issues 73 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

LAVINMQ_URI reaches every member pane with the broker password inline — on neither credential list, and not credential-shaped #172

New Issue
Closed
opened 2026-08-28 00:23:01 +02:00 by ltms · 1 comment
ltms commented 2026-08-28 00:23:01 +02:00
Owner
Copy Link
Copy Source

Found on 2026-08-28 while checking memberCredentials before flipping the policy on the Mac fleet (#144). Not previously filed.

What

LAVINMQ_URI is an AMQP URI with its password inline (amqp://user:password@host/vhost). The login chain exports it, and it is on neither memberCredentials.known: nor allow: in the live fleetd.yaml — 41 names are listed and this is not one of them.

Under the policy the daemon actually runs today (policy: deny-by-default) the control is a sentinel overlay applied to known: minus allow:. A name on neither list gets no overlay at all. So the variable passes into every member pane untouched.

Measured, names and lengths only, never values:

$ env -i HOME=$HOME PATH=/usr/bin:/bin BRIDGED_MEMBER=1 /bin/zsh -lc '...'
LAVINMQ_URI: PRESENT (len=86)

Why the existing guards all miss it

This is the CONFLUENCE_USERNAME lesson from #144 again, one variable over. logCredentialGap only reports names matching *TOKEN*|*KEY*|*SECRET*|*PASSWORD*|*AUTH*|*CREDENTIAL*. LAVINMQ_URI matches none of them, because the secret is in the value, not the name. So the daemon's own gap warning has never mentioned it and never will.

Eleven other listed names are in the same position — BESZEL_HUB_URL, CF_ACCOUNT_ID, GRAFANA_ADMIN_USER, N8N_OWNER_EMAIL, TELEGRAM_CHAT_ID and so on. Those are on known: because somebody thought of them. LAVINMQ_URI is the one nobody did.

Blast radius

This is not an outside credential like the AWS key in #144. It is a credential to the fleet's own control plane. A member holding it can connect to the shared broker and, on the vhost it names:

  • read other members' reply queues, including replies addressed to the lead;
  • publish into them, so a message from one member can be made to look like a reply from another;
  • reach the lead.<coordId>.inbox mailbox used for cross-host lead coordination (#137 / the coordinator block).

Two fleets share the one LavinMQ instance, isolated by vhost. This URI names one vhost, so the isolation holds across fleets — but inside a fleet, every member has the keys to the message bus that the whole authorization model assumes only the daemon can reach.

The fix is the one already built

Do not add LAVINMQ_URI to known:. That is the enumeration this keeps losing to.

policy: allow-list (CB-633, #144) blocks it correctly and for the right reason: it is absent from the derived allow-list, so it is blanked because nothing asked to keep it — not because someone predicted its name or its shape. That policy is merged and switched off; see the 2026-08-28 comment on #144.

So this issue is mostly evidence, not new work. What it does add:

  1. A test that the daemon's own broker URI never survives into a member's environment. Name it explicitly, because it is the control plane and deserves a named assertion, not just coverage-by-default.
  2. Reconsider the credential-shaped pattern's role in the gap report. It is documented as a warning signal rather than a control, and it is — but a warning that structurally cannot mention a URI-shaped secret is a warning with a blind spot worth stating in its own log line.
  3. Rotate the broker credential once the policy is flipped. Every member spawned on this fleet has had it.

Not verified

I have not checked whether the fleet01 daemon's config has the same gap, and I cannot reach that host. Its members run in non-login shells and inherit much less, so the exposure may not exist there — but that is a different reason, not this fix.

Found on 2026-08-28 while checking `memberCredentials` before flipping the policy on the Mac fleet (#144). Not previously filed. ## What `LAVINMQ_URI` is an AMQP URI with its password inline (`amqp://user:password@host/vhost`). The login chain exports it, and it is on **neither** `memberCredentials.known:` nor `allow:` in the live `fleetd.yaml` — 41 names are listed and this is not one of them. Under the policy the daemon actually runs today (`policy: deny-by-default`) the control is a sentinel overlay applied to `known:` minus `allow:`. A name on neither list gets **no overlay at all**. So the variable passes into every member pane untouched. Measured, names and lengths only, never values: ``` $ env -i HOME=$HOME PATH=/usr/bin:/bin BRIDGED_MEMBER=1 /bin/zsh -lc '...' LAVINMQ_URI: PRESENT (len=86) ``` ## Why the existing guards all miss it This is the `CONFLUENCE_USERNAME` lesson from #144 again, one variable over. `logCredentialGap` only reports names matching `*TOKEN*|*KEY*|*SECRET*|*PASSWORD*|*AUTH*|*CREDENTIAL*`. `LAVINMQ_URI` matches none of them, because the secret is in the *value*, not the *name*. So the daemon's own gap warning has never mentioned it and never will. Eleven other listed names are in the same position — `BESZEL_HUB_URL`, `CF_ACCOUNT_ID`, `GRAFANA_ADMIN_USER`, `N8N_OWNER_EMAIL`, `TELEGRAM_CHAT_ID` and so on. Those are on `known:` because somebody thought of them. `LAVINMQ_URI` is the one nobody did. ## Blast radius This is not an outside credential like the AWS key in #144. It is a credential **to the fleet's own control plane**. A member holding it can connect to the shared broker and, on the vhost it names: - read other members' reply queues, including replies addressed to the lead; - publish into them, so a message from one member can be made to look like a reply from another; - reach the `lead.<coordId>.inbox` mailbox used for cross-host lead coordination (#137 / the coordinator block). Two fleets share the one LavinMQ instance, isolated by vhost. This URI names one vhost, so the isolation holds across fleets — but inside a fleet, every member has the keys to the message bus that the whole authorization model assumes only the daemon can reach. ## The fix is the one already built Do not add `LAVINMQ_URI` to `known:`. That is the enumeration this keeps losing to. `policy: allow-list` (CB-633, #144) blocks it correctly and for the right reason: it is absent from the derived allow-list, so it is blanked because nothing asked to keep it — not because someone predicted its name or its shape. That policy is merged and switched off; see the 2026-08-28 comment on #144. So this issue is mostly evidence, not new work. What it does add: 1. **A test that the daemon's own broker URI never survives into a member's environment.** Name it explicitly, because it is the control plane and deserves a named assertion, not just coverage-by-default. 2. **Reconsider the credential-shaped pattern's role in the gap report.** It is documented as a warning signal rather than a control, and it is — but a warning that structurally cannot mention a URI-shaped secret is a warning with a blind spot worth stating in its own log line. 3. **Rotate the broker credential** once the policy is flipped. Every member spawned on this fleet has had it. ## Not verified I have not checked whether the fleet01 daemon's config has the same gap, and I cannot reach that host. Its members run in non-login shells and inherit much less, so the exposure may not exist there — but that is a different reason, not this fix.
ltms closed this issue 2026-08-31 09:27:33 +02:00
ltms commented 2026-08-31 09:27:51 +02:00
Author
Owner
Copy Link
Copy Source

Fixed and merged to main as ad587ea. Closed — with one honest limit recorded below.

What shipped

The exclusion is derived from the config, not hand-typed: whatever broker.uriEnv names is withheld from members automatically, because fleetd already knows that name is secret-bearing. coordinator.uriEnv gets the same treatment — on this host both resolve to the same variable, so fixing only one would have left the hole open.

It holds even when an operator lists the name under memberCredentials.allow:, following the SSH_AUTH_SOCK precedent. There is no override, since a member has no legitimate use for the broker password.

Verified: mvn clean install unpiped, 1047 tests, exit 0. The main test was sabotage-proven (expected: <false> but was: <true> with the exclusion disabled).

The limit — this is a guarantee only under policy: allow-list

A reviewer found this, and it is worth stating plainly rather than letting the fix imply more than it delivers.

Under allow-list, the exclusion is enforced by the generated ZDOTDIR scrub, which runs after the pane's shell has sourced the operator's chain. A login shell that re-exports the name is still blanked. That is a real guarantee.

Under the deny-list policy there is no scrub: the name is only removed from the pre-shell env map, and a login shell that sources the operator's secret store re-exports it. The member gets the URI anyway.

That is deny-list's long-standing weakness — a sourced file can undo it — and not a regression introduced here. But it means deny-list deployments do not get this protection, and the javadoc on brokerUriEnvNames now says so instead of implying otherwise. The same caveat applies to the non-zsh path, which has no scrub at all (#155).

This deployment runs policy: allow-list, so the live fleet is covered.

The reviewer's proposed stronger fix — force a scrub for known secret-bearing names even under deny-list — is reasonable and not done here, because it would mean deny-list starts generating a ZDOTDIR, which is a behaviour change well beyond this ticket. Worth its own ticket if anyone ever runs deny-list in earnest.

Fixed and merged to `main` as `ad587ea`. Closed — with one honest limit recorded below. ## What shipped The exclusion is **derived from the config**, not hand-typed: whatever `broker.uriEnv` names is withheld from members automatically, because fleetd already knows that name is secret-bearing. `coordinator.uriEnv` gets the same treatment — on this host both resolve to the same variable, so fixing only one would have left the hole open. It holds even when an operator lists the name under `memberCredentials.allow:`, following the `SSH_AUTH_SOCK` precedent. There is no override, since a member has no legitimate use for the broker password. Verified: `mvn clean install` unpiped, **1047 tests**, exit 0. The main test was sabotage-proven (`expected: <false> but was: <true>` with the exclusion disabled). ## The limit — this is a guarantee only under `policy: allow-list` A reviewer found this, and it is worth stating plainly rather than letting the fix imply more than it delivers. Under `allow-list`, the exclusion is enforced by the generated `ZDOTDIR` scrub, which runs **after** the pane's shell has sourced the operator's chain. A login shell that re-exports the name is still blanked. That is a real guarantee. Under the **deny-list** policy there is no scrub: the name is only removed from the pre-shell env map, and a login shell that sources the operator's secret store re-exports it. The member gets the URI anyway. That is deny-list's long-standing weakness — a sourced file can undo it — and not a regression introduced here. But it means deny-list deployments do **not** get this protection, and the javadoc on `brokerUriEnvNames` now says so instead of implying otherwise. The same caveat applies to the non-zsh path, which has no scrub at all (#155). **This deployment runs `policy: allow-list`**, so the live fleet is covered. The reviewer's proposed stronger fix — force a scrub for known secret-bearing names even under deny-list — is reasonable and not done here, because it would mean deny-list starts generating a `ZDOTDIR`, which is a behaviour change well beyond this ticket. Worth its own ticket if anyone ever runs deny-list in earnest.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
blocked
Cannot start until something else lands. The body says what.
 needs-live-proof
Merged and green, but never shown working on the running daemon. Not the same as done.
 ready-to-delegate
Scope, files and acceptance criteria are written. A worker can be briefed from the body alone.
 silent-default
A feature that compiles, passes tests, and ships turned off. Nine recurrences and counting.
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent kevin ltms
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: fleet/fleetd#172
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?