fleet/fleetd
3
0
Fork 0
Code Issues 73 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

CB-605: the systemd unit has launchd's login-shell secret gap, and does not mention the two tokens it drops #103

New Issue
Closed
opened 2026-08-16 18:32:52 +02:00 by ltms · 1 comment
ltms commented 2026-08-16 18:32:52 +02:00
Owner
Copy Link
Copy Source

Found while writing the systemd Features entry (CB-595).

The defect

deploy/bridged.service has the same login-shell secret gap that CB-594 fixed for launchd, and it is not documented.

The unit fixes PATH explicitly, with a comment that names the cause: "systemd does not source a login shell, so without it the daemon — and every worker — gets a bare default with no JDK/Maven." So the problem is understood in the file.

But it stops at PATH. For secrets the unit says "Secrets are NOT set here" and points the operator at a systemctl --user edit drop-in or an EnvironmentFile=, naming only BRIDGED_API_TOKEN. It never sources the secret store, and it never mentions WORKER_GITEA_TOKEN or AI_GATEWAY_TOKEN.

Why that is the dangerous shape

An operator who follows the unit file's own guidance verbatim sets BRIDGED_API_TOKEN and stops. The daemon then starts cleanly. Nothing fails at boot.

The failure appears much later, somewhere else: members that cannot open a PR, or a gateway profile returning 401. That is exactly the failure CB-594 was filed for on the macOS side, and exactly why the diagnosis took so long the first time — the symptom is far away from the cause, in time and in component.

launchd got scripts/bridged-launchd-wrapper.sh, which execs zsh -l so the secret store is sourced. Its header names WORKER_GITEA_TOKEN and AI_GATEWAY_TOKEN as the two secrets it closes the gap for. The Linux unit has no equivalent.

Bridged.reportRequiredSecrets does log which secret names resolved, on either platform. That helps only if the operator knows to look, and the unit file gives no prompt to.

Milestone

2.0, not 1.1. This host runs macOS under launchd; the Linux unit exists for the per-host gateways CB-308 introduces. Nothing is broken on a single macOS host today, which is the 1.1 admission rule. Filing it now so it is not rediscovered the hard way when the first Linux gateway is stood up — that is precisely when this bites, and precisely when nobody will remember it was known.

Acceptance criteria

  1. The Linux unit either sources the secret store the same way the launchd wrapper does, or its comments name WORKER_GITEA_TOKEN and AI_GATEWAY_TOKEN explicitly and say what happens if they are missing.
  2. Whichever route is chosen, the operator instruction is complete: following the file's own guidance must produce a working fleet, not just a running daemon.
  3. Verified on an actual Linux host, or the ticket says plainly that it was not.

Could not confirm

Whether this unit has ever been installed anywhere. Nothing in the repo confirms or denies it, and the worker was told not to run systemctl to find out.

Evidence

deploy/bridged.service (whole file), scripts/bridged-launchd-wrapper.sh (whole file, for the contrast), bridged/src/main/java/dev/ltms/bridged/Bridged.java:566-576.

Found while writing the systemd Features entry (CB-595). ## The defect `deploy/bridged.service` has the same login-shell secret gap that CB-594 fixed for launchd, and it is not documented. The unit fixes `PATH` explicitly, with a comment that names the cause: *"systemd does not source a login shell, so without it the daemon — and every worker — gets a bare default with no JDK/Maven."* So the problem is understood in the file. But it stops at `PATH`. For secrets the unit says *"Secrets are NOT set here"* and points the operator at a `systemctl --user edit` drop-in or an `EnvironmentFile=`, naming **only `BRIDGED_API_TOKEN`**. It never sources the secret store, and it never mentions `WORKER_GITEA_TOKEN` or `AI_GATEWAY_TOKEN`. ## Why that is the dangerous shape An operator who follows the unit file's own guidance verbatim sets `BRIDGED_API_TOKEN` and stops. The daemon then starts **cleanly**. Nothing fails at boot. The failure appears much later, somewhere else: members that cannot open a PR, or a gateway profile returning 401. That is exactly the failure CB-594 was filed for on the macOS side, and exactly why the diagnosis took so long the first time — the symptom is far away from the cause, in time and in component. launchd got `scripts/bridged-launchd-wrapper.sh`, which execs `zsh -l` so the secret store is sourced. Its header names `WORKER_GITEA_TOKEN` and `AI_GATEWAY_TOKEN` as the two secrets it closes the gap for. **The Linux unit has no equivalent.** `Bridged.reportRequiredSecrets` does log which secret names resolved, on either platform. That helps only if the operator knows to look, and the unit file gives no prompt to. ## Milestone **2.0**, not 1.1. This host runs macOS under launchd; the Linux unit exists for the per-host gateways CB-308 introduces. Nothing is broken on a single macOS host today, which is the 1.1 admission rule. Filing it now so it is not rediscovered the hard way when the first Linux gateway is stood up — that is precisely when this bites, and precisely when nobody will remember it was known. ## Acceptance criteria 1. The Linux unit either sources the secret store the same way the launchd wrapper does, or its comments name `WORKER_GITEA_TOKEN` and `AI_GATEWAY_TOKEN` explicitly and say what happens if they are missing. 2. Whichever route is chosen, the operator instruction is complete: following the file's own guidance must produce a working fleet, not just a running daemon. 3. Verified on an actual Linux host, or the ticket says plainly that it was not. ## Could not confirm Whether this unit has ever been installed anywhere. Nothing in the repo confirms or denies it, and the worker was told not to run `systemctl` to find out. ## Evidence `deploy/bridged.service` (whole file), `scripts/bridged-launchd-wrapper.sh` (whole file, for the contrast), `bridged/src/main/java/dev/ltms/bridged/Bridged.java:566-576`.
ltms added this to the 2.0 — one operation centre, many hosts milestone 2026-08-16 18:32:52 +02:00
ltms commented 2026-09-03 11:35:59 +02:00
Author
Owner
Copy Link
Copy Source

Fixed. PR #262 merged to main as e2fe861, with a follow-up correction in f0e7ac7.

What shipped

Route B — document it completely. The worker picked this over sourcing the secret store, and gave the right reason: the secret store is zsh-only, and a Linux host may not have zsh. Route A would have been a control that silently does nothing on the exact platform this unit exists for.

deploy/fleetd.service now names all three tokens, what each is for, and — the important part — the late symptom of each one going missing:

  • FLEETD_API_TOKEN protects fleetd's API.
  • WORKER_GITEA_TOKEN lets members open pull requests. Missing: fleetd still starts, and a member fails later when it tries to open a PR.
  • AI_GATEWAY_TOKEN authenticates gateway profiles. Missing: fleetd still starts, and a gateway profile later returns HTTP 401.

The follow-up commit, and why it was needed

The merged version told the operator to check journalctl --user -u fleetd for Fleetd.reportRequiredSecrets.

That string never appears in the log. It is a method name. The logger is d.l.f.Fleetd and the lines actually read:

startup secret WORKER_GITEA_TOKEN: set (profile 'sonnet' gitTokenEnv)
startup secret AI_GATEWAY_TOKEN: MISSING (profile 'local' tokenEnv) — ...

So the advice sent the operator hunting for text that does not exist — a small thing, but this whole ticket is about instructions that look complete and leave you stranded. Corrected to give the grep:

journalctl --user -u fleetd | grep 'startup secret'

I also added what the report does not cover. I checked requiredSecretEnvVars rather than assuming: it collects each profile's tokenEnv and gitTokenEnv, plus the broker's uriEnv. So a secret that no configured profile references is never reported — which is correct, but an operator should know it before treating a clean grep as a clean bill of health.

Good news for criterion 1: gitTokenEnv is covered, so WORKER_GITEA_TOKEN really does show up in that report. That was the one I most expected to be missing.

Acceptance criteria

  1. Comments name both dropped tokens and say what happens if missing. Done.
  2. Following the file's own guidance gives a working fleet, not just a running daemon. Done — the instructions now cover all three tokens and tell the operator how to confirm each resolved.
  3. Verified on a real Linux host, or the ticket says plainly it was not. It was NOT. No Linux host was touched, systemctl was never run, and the unit was never installed. The worker said so without being pushed, which is the right instinct.

Still could not confirm

Whether this unit has ever been installed anywhere. Nothing in the repo says. fleet01 runs fleetd headless but has no supervision for it (see #156), so the unit is probably still unused — which is exactly the situation where this bug waits to bite.

Closing.

Fixed. PR #262 merged to `main` as `e2fe861`, with a follow-up correction in `f0e7ac7`. ## What shipped Route B — document it completely. The worker picked this over sourcing the secret store, and gave the right reason: the secret store is zsh-only, and a Linux host may not have zsh. Route A would have been a control that silently does nothing on the exact platform this unit exists for. `deploy/fleetd.service` now names all three tokens, what each is for, and — the important part — the **late** symptom of each one going missing: - `FLEETD_API_TOKEN` protects fleetd's API. - `WORKER_GITEA_TOKEN` lets members open pull requests. Missing: fleetd still starts, and a member fails later when it tries to open a PR. - `AI_GATEWAY_TOKEN` authenticates gateway profiles. Missing: fleetd still starts, and a gateway profile later returns HTTP 401. ## The follow-up commit, and why it was needed The merged version told the operator to check `journalctl --user -u fleetd` for `Fleetd.reportRequiredSecrets`. **That string never appears in the log.** It is a method name. The logger is `d.l.f.Fleetd` and the lines actually read: ``` startup secret WORKER_GITEA_TOKEN: set (profile 'sonnet' gitTokenEnv) startup secret AI_GATEWAY_TOKEN: MISSING (profile 'local' tokenEnv) — ... ``` So the advice sent the operator hunting for text that does not exist — a small thing, but this whole ticket is about instructions that look complete and leave you stranded. Corrected to give the grep: ``` journalctl --user -u fleetd | grep 'startup secret' ``` I also added what the report does **not** cover. I checked `requiredSecretEnvVars` rather than assuming: it collects each profile's `tokenEnv` and `gitTokenEnv`, plus the broker's `uriEnv`. So a secret that no configured profile references is never reported — which is correct, but an operator should know it before treating a clean grep as a clean bill of health. Good news for criterion 1: `gitTokenEnv` **is** covered, so `WORKER_GITEA_TOKEN` really does show up in that report. That was the one I most expected to be missing. ## Acceptance criteria 1. **Comments name both dropped tokens and say what happens if missing.** Done. 2. **Following the file's own guidance gives a working fleet, not just a running daemon.** Done — the instructions now cover all three tokens and tell the operator how to confirm each resolved. 3. **Verified on a real Linux host, or the ticket says plainly it was not.** **It was NOT.** No Linux host was touched, `systemctl` was never run, and the unit was never installed. The worker said so without being pushed, which is the right instinct. ## Still could not confirm Whether this unit has ever been installed anywhere. Nothing in the repo says. `fleet01` runs `fleetd` headless but has no supervision for it (see #156), so the unit is probably still unused — which is exactly the situation where this bug waits to bite. Closing.
ltms closed this issue 2026-09-03 11:36:04 +02:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
blocked
Cannot start until something else lands. The body says what.
 needs-live-proof
Merged and green, but never shown working on the running daemon. Not the same as done.
 ready-to-delegate
Scope, files and acceptance criteria are written. A worker can be briefed from the body alone.
 silent-default
A feature that compiles, passes tests, and ships turned off. Nine recurrences and counting.
No Label
Milestone
No items
No Milestone 2.0 — one operation centre, many hosts
Projects
Clear projects
No project
Assignees
Clear assignees
agent kevin ltms
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: fleet/fleetd#103
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?