fleet/fleetd
3
0
Fork 0
Code Issues 73 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

CB-608: the credential probe hardcodes its own name list, so it silently under-reports the policy #111

New Issue
Closed
opened 2026-08-17 13:45:59 +02:00 by ltms · 1 comment
ltms commented 2026-08-17 13:45:59 +02:00
Owner
Copy Link
Copy Source

Found while running the CB-596 verification probe on 2026-08-17.

What happened

scripts/probe-member-credentials.sh carries its own hardcoded list of 31 names. The live policy in bridged.yaml covers 34. The probe never reads the config, so the three names added to known: after the script was written — N8N_ENCRYPTION_KEY, N8N_OWNER_EMAIL, N8N_OWNER_PASSWORD — were never checked.

The probe reported 26 blocked. The policy blocks 29. Both numbers were correct; they were counting different sets.

Nothing failed. The script exited 0 and its output looked like a complete pass. The three missing names had to be measured by a separate hand-written loop, and only because someone compared 26 against 29 and asked why they differed.

Why this matters more than three names

This is the same defect CB-596 exists to fix, in a third place.

  • CB-592 hardcoded one blocked name in Java. CB-596 moved it to config, because a hardcoded list is silently wrong the moment a new secret appears.
  • CB-596 needs a second copy of that list in secrets.sh, because the login shell overwrites the launcher's environment. That copy is guarded by a gap detector, which warns when the two disagree.
  • The probe is a third copy, and nothing guards it. It is the tool we use to decide whether the policy works — so when it drifts, it drifts in the direction of a false pass. That is the worst direction for a verification tool to fail in.

A checker that under-reports is worse than no checker, because its clean output is taken as evidence.

The fix

The probe should read the name list from the same place the daemon does, rather than keeping its own copy.

The probe runs inside a member pane, where bridged.yaml is not necessarily readable and the daemon is reachable. So the natural source is the daemon: have it expose the policy's name list (names only — never values, and the daemon does not have the values anyway), and have the probe ask for it. Then a name added to known: is covered by the next probe run automatically.

If exposing it over REST is unwanted, the fallback is to make the drift loud rather than silent: the probe compares its own count against the daemon's reported counts and fails when they differ, instead of printing a table that looks complete.

Acceptance criteria

  1. Adding a name to memberCredentials.known in bridged.yaml causes the very next probe run to cover it, with no edit to the script.
  2. If the probe cannot obtain the policy's name list for any reason, it fails loudly rather than falling back to a built-in list. A verification tool must not degrade quietly into a weaker check.
  3. The probe's output states how many names the policy contains and how many it checked. Those two numbers being equal is what a reader should look for.
  4. Still no values, no prefixes. The SHA256-12 column stays as it is — safe for comparing two readings on one machine, unsafe to publish, and that caveat should stay printed in the script's own header.
  5. Re-run against the live policy and confirm it reports 34 checked, 29 blocked, 5 allowed-and-present — matching the daemon's startup line.

Also worth recording from the same run

CLAUDE_CODE_MESSAGING_TOKEN is unset in a member pane, though it is present in the daemon's own environment — which is why the gap detector saw it. Allow-listing it was therefore harmless, but the reason it is harmless is not the reason it was allowed. Whoever picks up the open question about what that token grants should start from this: members do not receive it today.

Milestone

2.0. The policy itself is verified and correct — that was measured, name by name, in a live member. This is about keeping the verification honest as the policy grows, which nothing on one host depends on today.

Found while running the CB-596 verification probe on 2026-08-17. ## What happened `scripts/probe-member-credentials.sh` carries its **own hardcoded list of 31 names**. The live policy in `bridged.yaml` covers **34**. The probe never reads the config, so the three names added to `known:` after the script was written — `N8N_ENCRYPTION_KEY`, `N8N_OWNER_EMAIL`, `N8N_OWNER_PASSWORD` — were never checked. The probe reported **26 blocked**. The policy blocks **29**. Both numbers were correct; they were counting different sets. Nothing failed. The script exited 0 and its output looked like a complete pass. The three missing names had to be measured by a separate hand-written loop, and only because someone compared 26 against 29 and asked why they differed. ## Why this matters more than three names This is the **same defect CB-596 exists to fix**, in a third place. - CB-592 hardcoded one blocked name in Java. CB-596 moved it to config, because a hardcoded list is silently wrong the moment a new secret appears. - CB-596 needs a second copy of that list in `secrets.sh`, because the login shell overwrites the launcher's environment. That copy is guarded by a gap detector, which warns when the two disagree. - **The probe is a third copy, and nothing guards it.** It is the tool we use to decide whether the policy works — so when it drifts, it drifts in the direction of a false pass. That is the worst direction for a verification tool to fail in. A checker that under-reports is worse than no checker, because its clean output is taken as evidence. ## The fix The probe should read the name list from the same place the daemon does, rather than keeping its own copy. The probe runs inside a member pane, where `bridged.yaml` is not necessarily readable and the daemon is reachable. So the natural source is the daemon: have it expose the policy's name list (names only — never values, and the daemon does not have the values anyway), and have the probe ask for it. Then a name added to `known:` is covered by the next probe run automatically. If exposing it over REST is unwanted, the fallback is to make the drift loud rather than silent: the probe compares its own count against the daemon's reported counts and **fails** when they differ, instead of printing a table that looks complete. ## Acceptance criteria 1. Adding a name to `memberCredentials.known` in `bridged.yaml` causes the very next probe run to cover it, with **no edit to the script**. 2. If the probe cannot obtain the policy's name list for any reason, it **fails loudly** rather than falling back to a built-in list. A verification tool must not degrade quietly into a weaker check. 3. The probe's output states how many names the policy contains and how many it checked. Those two numbers being equal is what a reader should look for. 4. Still no values, no prefixes. The `SHA256-12` column stays as it is — safe for comparing two readings on one machine, unsafe to publish, and that caveat should stay printed in the script's own header. 5. Re-run against the live policy and confirm it reports 34 checked, 29 blocked, 5 allowed-and-present — matching the daemon's startup line. ## Also worth recording from the same run `CLAUDE_CODE_MESSAGING_TOKEN` is **unset in a member pane**, though it is present in the daemon's own environment — which is why the gap detector saw it. Allow-listing it was therefore harmless, but the reason it is harmless is not the reason it was allowed. Whoever picks up the open question about what that token grants should start from this: members do not receive it today. ## Milestone **2.0.** The policy itself is verified and correct — that was measured, name by name, in a live member. This is about keeping the verification honest as the policy grows, which nothing on one host depends on today.
ltms added this to the 2.0 — one operation centre, many hosts milestone 2026-08-17 13:45:59 +02:00
ltms commented 2026-09-03 08:37:21 +02:00
Author
Owner
Copy Link
Copy Source

Merged as 6b5f3f4, daemon redeployed onto it, and acceptance criterion 5 verified live.

Criterion 5 — the live run

The probe now refuses to run outside a member shell, so the real reading was taken inside a spawned sonnet member on the redeployed daemon:

policy       : mode=allow-list known=34 allowed=7 blocked=29
policy contains 34 name(s); this run checked 34 — they match.
5 of 34 names are set in this shell.

34 checked, 29 blocked, 5 allowed-and-present — matching the daemon's own startup line:

memberCredentials: policy=allow-list, 34 known name(s), 7 allowed — blocking 29 on every spawn

The old probe checked 31 hardcoded names and reported 26 blocked. Both numbers were right about different sets, which is exactly the defect.

The member also confirmed no credential value appears in the output — only name, set/unset state, length, and the SHA256-12 column.

That reading is a second useful result on its own: a member really does hold only 5 of the 34 names, so the ZDOTDIR scrub is doing what the policy says.

Criteria 1–4

1 — a name added to known: is covered with no script edit. The NAMES array is deleted, not updated. The probe fetches GET /member-credentials.

2 — it fails loudly rather than degrading. Unreachable daemon, absent or empty policy, or a knownCount that disagrees with the length of known[] all exit non-zero. No local fallback.

3 — it prints its own denominator. policy contains 34 name(s); this run checked 34 — they match. Two numbers a reader can compare, instead of one to trust.

4 — no values, no prefixes. MemberCredentialPolicyView reads no environment at all, so there is nothing to redact by construction. The endpoint returns names and counts. Confirmed against the live endpoint: fields are present, policy, known, knownCount, allowed, allowedCount, blockedCount — no value-shaped field of any kind.

The subtlety this could have got wrong

Before briefing the work I measured the live policy and found that blocked is not known - allowed: known is 34 and allow is 7, but only 5 of those 7 appear in known, so blocked is 29 while the naive subtraction gives 27.

The brief asked for the startup log line and the endpoint to share one code path, and they do — blocked comes from the existing creds.blockedSet() rather than a second subtraction. So the endpoint reports 29 and agrees with the log. Worth stating because a reader who recomputes it by hand will get 27 and think something is broken.

Worker notes worth keeping

It found and fixed a bug in its own first draft: jq's // operator treats false and 0 as missing, so .present // empty silently turned a genuine "present": false into "unknown". Fixed by reading the fields directly.

Mutation evidence: forcing MemberCredentialPolicyView.of(...) to return ABSENT turned 4 tests red with 0 compile errors — including MemberCredentialsGapReportTest, which proves the startup log really runs through this path. Reverted, confirmed with diff -q.

Same-shape observation, noted and not fixed: scripts/rename-checkout.sh and scripts/redeploy-fleetd.sh each hardcode HEALTH='http://127.0.0.1:8765/healthz' independently. Low risk today, same "two files, one fact" shape.

Build on the merged tree: 1255 tests, 0 failures, 0 compile errors.

Documented in the wiki Features page as "The credential probe asks the daemon what the policy is".

Merged as `6b5f3f4`, daemon redeployed onto it, and **acceptance criterion 5 verified live**. ## Criterion 5 — the live run The probe now refuses to run outside a member shell, so the real reading was taken inside a spawned `sonnet` member on the redeployed daemon: ``` policy : mode=allow-list known=34 allowed=7 blocked=29 policy contains 34 name(s); this run checked 34 — they match. 5 of 34 names are set in this shell. ``` **34 checked, 29 blocked, 5 allowed-and-present** — matching the daemon's own startup line: ``` memberCredentials: policy=allow-list, 34 known name(s), 7 allowed — blocking 29 on every spawn ``` The old probe checked 31 hardcoded names and reported 26 blocked. Both numbers were right about different sets, which is exactly the defect. The member also confirmed no credential value appears in the output — only name, set/unset state, length, and the `SHA256-12` column. That reading is a second useful result on its own: a member really does hold only 5 of the 34 names, so the ZDOTDIR scrub is doing what the policy says. ## Criteria 1–4 **1 — a name added to `known:` is covered with no script edit.** The `NAMES` array is deleted, not updated. The probe fetches `GET /member-credentials`. **2 — it fails loudly rather than degrading.** Unreachable daemon, absent or empty policy, or a `knownCount` that disagrees with the length of `known[]` all exit non-zero. No local fallback. **3 — it prints its own denominator.** `policy contains 34 name(s); this run checked 34 — they match.` Two numbers a reader can compare, instead of one to trust. **4 — no values, no prefixes.** `MemberCredentialPolicyView` reads no environment at all, so there is nothing to redact by construction. The endpoint returns names and counts. Confirmed against the live endpoint: fields are `present`, `policy`, `known`, `knownCount`, `allowed`, `allowedCount`, `blockedCount` — no value-shaped field of any kind. ## The subtlety this could have got wrong Before briefing the work I measured the live policy and found that **blocked is not `known - allowed`**: `known` is 34 and `allow` is 7, but only 5 of those 7 appear in `known`, so blocked is 29 while the naive subtraction gives 27. The brief asked for the startup log line and the endpoint to share one code path, and they do — `blocked` comes from the existing `creds.blockedSet()` rather than a second subtraction. So the endpoint reports 29 and agrees with the log. Worth stating because a reader who recomputes it by hand will get 27 and think something is broken. ## Worker notes worth keeping It found and fixed a bug in its own first draft: jq's `//` operator treats `false` and `0` as missing, so `.present // empty` silently turned a genuine `"present": false` into "unknown". Fixed by reading the fields directly. Mutation evidence: forcing `MemberCredentialPolicyView.of(...)` to return `ABSENT` turned 4 tests red with 0 compile errors — including `MemberCredentialsGapReportTest`, which proves the startup log really runs through this path. Reverted, confirmed with `diff -q`. Same-shape observation, noted and not fixed: `scripts/rename-checkout.sh` and `scripts/redeploy-fleetd.sh` each hardcode `HEALTH='http://127.0.0.1:8765/healthz'` independently. Low risk today, same "two files, one fact" shape. Build on the merged tree: 1255 tests, 0 failures, 0 compile errors. Documented in the wiki Features page as *"The credential probe asks the daemon what the policy is"*.
ltms closed this issue 2026-09-03 08:37:57 +02:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
blocked
Cannot start until something else lands. The body says what.
 needs-live-proof
Merged and green, but never shown working on the running daemon. Not the same as done.
 ready-to-delegate
Scope, files and acceptance criteria are written. A worker can be briefed from the body alone.
 silent-default
A feature that compiles, passes tests, and ships turned off. Nine recurrences and counting.
No Label
Milestone
No items
No Milestone 2.0 — one operation centre, many hosts
Projects
Clear projects
No project
Assignees
Clear assignees
agent kevin ltms
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: fleet/fleetd#111
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?