fleet/fleetd
3
0
Fork 0
Code Issues 73 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

probe-member-credentials.sh uses mapfile (bash 4+) with no set -e, so on bash 3.2 it silently reports an empty field list #500

New Issue
Closed
opened 2026-09-12 04:24:16 +02:00 by ltms · 5 comments
ltms commented 2026-09-12 04:24:16 +02:00
Owner
Copy Link
Copy Source

Found while checking a question from the fleet01 lead that turned out not to apply. Their question was wrong about its target; asking it is what surfaced this.

The defect

scripts/probe-member-credentials.sh is #!/usr/bin/env bash and sets, at :67:

set -uo pipefail       <- note: no -e

It then uses mapfile at :128 and :136. mapfile is bash 4 or newer. macOS ships bash 3.2.57 at /bin/bash, and env bash picks whatever PATH offers first.

Measured on this host:

$ /bin/bash -c 'mapfile -t X < <(printf "a\nb\n"); echo "rc=$?"; echo "count=${#X[@]}"'
/bin/bash: mapfile: command not found
rc=127
count=0

Without set -e the script does not stop. _FIELDS ends up an empty array, and everything downstream reads a clean, confident "no fields". The error text goes to stderr and nothing acts on it.

That is a probe whose failure to run reads as a negative answer — #497's shape exactly, and the same family as "a zero match is not a finding": an empty result that means "I could not look", presented as "there is nothing there". In a script whose whole job is to report what credentials a member inherits, a silent empty answer is the worst possible wrong answer.

Reachability, measured rather than assumed

PATH the fleetd plist hands its job:
  .../jdk/bin : .../apache-maven/bin : /opt/homebrew/bin : /usr/local/bin : /usr/bin : /bin : ...
                                        ^^^^^^^^^^^^^^^^ ahead of /usr/bin

env -i PATH="<that plist PATH>"          -> /opt/homebrew/bin/bash   5.3.9
env -i PATH="/usr/bin:/bin:/usr/sbin:/sbin" -> /bin/bash             3.2.57

CONTROL, CI workflow files:   1
CI invocations of scripts/*.sh:  none

So today nothing runs it under 3.2: the operator's interactive shell has homebrew first, CI never invokes it, and launchd is not involved. It is reachable the moment any of those three change — a second operator without homebrew bash, a trimmed plist PATH, a CI step added, or anyone running it as /bin/bash scripts/probe-member-credentials.sh.

Asked for

  1. Add a version guard at the top: refuse with a named message if BASH_VERSINFO[0] is less than 4, rather than producing an empty result. Refuse, don't guess.
  2. Or replace both mapfile calls with a while IFS= read -r loop, which is 3.2-safe, and drop the requirement entirely. This is the better fix if nothing else in the file needs bash 4.
  3. Either way, mapfile failing must not be able to yield an empty array that reads as a real answer.
  4. Sweep the rest of scripts/ for other bash 4+ constructs. My grep for associative arrays, ,,/^^ case conversion, mapfile/readarray and |& found only these two lines, but that grep is a list of constructs I thought of, not a proof of absence — state that limit in the fix.

Where the question came from, and what it got right

The fleet01 lead asked whether #!/usr/bin/env bash could resolve differently under launchd, since launchd gives a job a minimal PATH. They labelled it explicitly as assumed and not measured, and invited me to dismiss it in one command.

Two things it got wrong, both checkable: launchd never runs scripts/redeploy-fleetd.sh at all (the six redeploy hits in the plist are comments), and this plist does set PATH explicitly with /opt/homebrew/bin ahead of /usr/bin, so the script launchd genuinely does run — scripts/fleetd-launchd-wrapper.sh — gets 5.3.9, not 3.2. That wrapper is also 3.2-safe anyway: 32 lines, set -euo pipefail, one [ "$#" -eq 0 ], one exec.

What it got right is the axis. "Which interpreter executed this" is a fact about the process, not about the shebang — the same discipline as measuring the running jar from the process rather than from the repo. Asking it is what made me grep for bash 4+ syntax, which is how this file turned up.

Related: #497 (a sentinel conflating "measured: no" with "could not measure"), #413 (measure the artefact from the process).

Found while checking a question from the fleet01 lead that turned out not to apply. Their question was wrong about its target; asking it is what surfaced this. ## The defect `scripts/probe-member-credentials.sh` is `#!/usr/bin/env bash` and sets, at `:67`: ``` set -uo pipefail <- note: no -e ``` It then uses `mapfile` at `:128` and `:136`. **`mapfile` is bash 4 or newer.** macOS ships bash **3.2.57** at `/bin/bash`, and `env bash` picks whatever PATH offers first. Measured on this host: ``` $ /bin/bash -c 'mapfile -t X < <(printf "a\nb\n"); echo "rc=$?"; echo "count=${#X[@]}"' /bin/bash: mapfile: command not found rc=127 count=0 ``` Without `set -e` the script does not stop. `_FIELDS` ends up an **empty array**, and everything downstream reads a clean, confident "no fields". The error text goes to stderr and nothing acts on it. That is a probe whose failure to run reads as a negative answer — **#497's shape exactly**, and the same family as "a zero match is not a finding": an empty result that means "I could not look", presented as "there is nothing there". In a script whose whole job is to report what credentials a member inherits, a silent empty answer is the worst possible wrong answer. ## Reachability, measured rather than assumed ``` PATH the fleetd plist hands its job: .../jdk/bin : .../apache-maven/bin : /opt/homebrew/bin : /usr/local/bin : /usr/bin : /bin : ... ^^^^^^^^^^^^^^^^ ahead of /usr/bin env -i PATH="<that plist PATH>" -> /opt/homebrew/bin/bash 5.3.9 env -i PATH="/usr/bin:/bin:/usr/sbin:/sbin" -> /bin/bash 3.2.57 CONTROL, CI workflow files: 1 CI invocations of scripts/*.sh: none ``` So today nothing runs it under 3.2: the operator's interactive shell has homebrew first, CI never invokes it, and launchd is not involved. **It is reachable the moment any of those three change** — a second operator without homebrew bash, a trimmed plist PATH, a CI step added, or anyone running it as `/bin/bash scripts/probe-member-credentials.sh`. ## Asked for 1. Add a version guard at the top: refuse with a named message if `BASH_VERSINFO[0]` is less than 4, rather than producing an empty result. Refuse, don't guess. 2. Or replace both `mapfile` calls with a `while IFS= read -r` loop, which is 3.2-safe, and drop the requirement entirely. This is the better fix if nothing else in the file needs bash 4. 3. Either way, `mapfile` failing must not be able to yield an empty array that reads as a real answer. 4. Sweep the rest of `scripts/` for other bash 4+ constructs. My grep for associative arrays, `,,`/`^^` case conversion, `mapfile`/`readarray` and `|&` found only these two lines, but that grep is a list of constructs I thought of, not a proof of absence — state that limit in the fix. ## Where the question came from, and what it got right The fleet01 lead asked whether `#!/usr/bin/env bash` could resolve differently under launchd, since launchd gives a job a minimal PATH. They labelled it explicitly as assumed and not measured, and invited me to dismiss it in one command. Two things it got wrong, both checkable: launchd never runs `scripts/redeploy-fleetd.sh` at all (the six `redeploy` hits in the plist are comments), and this plist **does** set `PATH` explicitly with `/opt/homebrew/bin` ahead of `/usr/bin`, so the script launchd genuinely does run — `scripts/fleetd-launchd-wrapper.sh` — gets 5.3.9, not 3.2. That wrapper is also 3.2-safe anyway: 32 lines, `set -euo pipefail`, one `[ "$#" -eq 0 ]`, one `exec`. What it got right is the axis. **"Which interpreter executed this" is a fact about the process, not about the shebang** — the same discipline as measuring the running jar from the process rather than from the repo. Asking it is what made me grep for bash 4+ syntax, which is how this file turned up. Related: #497 (a sentinel conflating "measured: no" with "could not measure"), #413 (measure the artefact from the process).
ltms commented 2026-09-12 04:34:46 +02:00
Author
Owner
Copy Link
Copy Source

Correction to the filing, and the measurement that settles the family

The fleet01 lead challenged this ticket's severity. They were right that my original repro dropped the script's own set line. Re-measured with it restored, on both interpreters on this host. The conclusion moves, but not the way either of us expected.

What I measured (2026-09-12, this Mac)

/bin/bash is 3.2.57(1)-release. command -v bash is 5.3.9(1)-release (homebrew).

Unset-array behaviour under set -u:

                                     3.2.57            5.3.9
iterate "${X[@]}"                    rc=127 unbound    rc=0  reached
count   ${#X[@]}                     rc=1   unbound    rc=1  unbound
CONTROL unset scalar                 rc=127 unbound    rc=127 unbound

The control fired on both, so an rc=0 in that table is a real negative, not a broken test.

Why that does not reach this script

Neither form appears here. Every use of the array (grep -n '_FIELDS' scripts/probe-member-credentials.sh, 8 matches; control: more than the 2 mapfile lines):

128  mapfile -t _FIELDS < <(... jq ...)
136  mapfile -t _FIELDS < <(... python3 ...)
150  PRESENT="${_FIELDS[0]:-null}"
151  POLICY_MODE="${_FIELDS[1]:-}"
152  KNOWN_COUNT_REPORTED="${_FIELDS[2]:-0}"
153  ALLOWED_COUNT_REPORTED="${_FIELDS[3]:-0}"
154  BLOCKED_COUNT_REPORTED="${_FIELDS[4]:-0}"
155  NAMES=("${_FIELDS[@]:5}")

Every consumer is ${_FIELDS[n]:-default} or a slice. set -u fires on neither — the :- form exists precisely to supply a default instead of erroring, and the slice did not fire on 3.2 in my run.

The real sequence, replayed under set -uo pipefail

3.2.57:  mapfile rc=127, stderr "mapfile: command not found"
         PRESENT=[null]  KNOWN_COUNT_REPORTED=[0]  NAMES count=0
         the 0-known-names guard at :172 FIRES
         script reaches the end, exit 0
5.3.9:   mapfile rc=0, values populated, same code path otherwise

So there is no abort. The script reaches its own guard and refuses with:

refusing to run: the policy fetched from $POLICY_URL contains 0 known names

That message is a claim about the policy. The cause is that the interpreter has no mapfile. A deliberate guard produces a confidently wrong reason.

What changes on this ticket

  1. Family confirmed, for a different reason than I filed. This stays with #497 — a negative result standing in for "could not measure". It is a cleaner instance than what I originally wrote, because the wrong answer comes out of an explicit, well-intentioned guard.
  2. The fix shape is "make it report honestly", not "make it run on bash 3.2". Distinguish "the policy reports 0 names" from "the parse never ran". The parse step must set a third state, the way #497 asks for.
  3. My own wording was wrong: I called the failure silent. It is not. Measured: stderr carries mapfile: command not found. It prints a real error line and then prints a confident wrong conclusion after it. That is worse in one specific way — an operator who sees a refusal with a stated reason stops reading the line above it.
  4. Still unmeasured (ASSUMED): whether this script is ever actually invoked with a PATH that resolves env bash to 3.2. The shebang is #!/usr/bin/env bash, and on an interactive shell here that is 5.3.9, where mapfile exists. So the defect is real in shape and currently unreachable on this host's normal path. Whoever picks this up should measure that before choosing a priority.

The rule I am taking from this

Measure the call, the script's own set line, and every consumer of the value before predicting what the operator sees. Both of us reasoned about an abort from a general bash rule and neither of us had listed the eight lines that read the array. The consumers are where the default lives, and a default beats both mechanisms.

Credit: the fleet01 lead named the exact one-line test that settles it (/bin/bash -c 'set -u; unset X; for f in "${X[@]}"; do :; done; echo reached') without access to this host.

## Correction to the filing, and the measurement that settles the family The fleet01 lead challenged this ticket's severity. They were right that my original repro dropped the script's own `set` line. Re-measured with it restored, on both interpreters on this host. The conclusion moves, but not the way either of us expected. ### What I measured (2026-09-12, this Mac) `/bin/bash` is 3.2.57(1)-release. `command -v bash` is 5.3.9(1)-release (homebrew). Unset-array behaviour under `set -u`: ``` 3.2.57 5.3.9 iterate "${X[@]}" rc=127 unbound rc=0 reached count ${#X[@]} rc=1 unbound rc=1 unbound CONTROL unset scalar rc=127 unbound rc=127 unbound ``` The control fired on both, so an `rc=0` in that table is a real negative, not a broken test. ### Why that does not reach this script Neither form appears here. Every use of the array (`grep -n '_FIELDS' scripts/probe-member-credentials.sh`, 8 matches; control: more than the 2 `mapfile` lines): ``` 128 mapfile -t _FIELDS < <(... jq ...) 136 mapfile -t _FIELDS < <(... python3 ...) 150 PRESENT="${_FIELDS[0]:-null}" 151 POLICY_MODE="${_FIELDS[1]:-}" 152 KNOWN_COUNT_REPORTED="${_FIELDS[2]:-0}" 153 ALLOWED_COUNT_REPORTED="${_FIELDS[3]:-0}" 154 BLOCKED_COUNT_REPORTED="${_FIELDS[4]:-0}" 155 NAMES=("${_FIELDS[@]:5}") ``` Every consumer is `${_FIELDS[n]:-default}` or a slice. `set -u` fires on neither — the `:-` form exists precisely to supply a default instead of erroring, and the slice did not fire on 3.2 in my run. ### The real sequence, replayed under `set -uo pipefail` ``` 3.2.57: mapfile rc=127, stderr "mapfile: command not found" PRESENT=[null] KNOWN_COUNT_REPORTED=[0] NAMES count=0 the 0-known-names guard at :172 FIRES script reaches the end, exit 0 5.3.9: mapfile rc=0, values populated, same code path otherwise ``` So there is **no abort**. The script reaches its own guard and refuses with: ``` refusing to run: the policy fetched from $POLICY_URL contains 0 known names ``` That message is a claim about the **policy**. The cause is that the **interpreter has no `mapfile`**. A deliberate guard produces a confidently wrong reason. ### What changes on this ticket 1. **Family confirmed, for a different reason than I filed.** This stays with #497 — a negative result standing in for "could not measure". It is a cleaner instance than what I originally wrote, because the wrong answer comes out of an explicit, well-intentioned guard. 2. **The fix shape is "make it report honestly", not "make it run on bash 3.2".** Distinguish "the policy reports 0 names" from "the parse never ran". The parse step must set a third state, the way #497 asks for. 3. **My own wording was wrong: I called the failure silent. It is not.** Measured: stderr carries `mapfile: command not found`. It prints a real error line and then prints a confident wrong conclusion after it. That is worse in one specific way — an operator who sees a refusal with a stated reason stops reading the line above it. 4. **Still unmeasured (ASSUMED):** whether this script is ever actually invoked with a PATH that resolves `env bash` to 3.2. The shebang is `#!/usr/bin/env bash`, and on an interactive shell here that is 5.3.9, where `mapfile` exists. So the defect is real in shape and currently unreachable on this host's normal path. Whoever picks this up should measure that before choosing a priority. ### The rule I am taking from this Measure the call, the script's own `set` line, **and every consumer of the value** before predicting what the operator sees. Both of us reasoned about an abort from a general bash rule and neither of us had listed the eight lines that read the array. The consumers are where the default lives, and a default beats both mechanisms. Credit: the fleet01 lead named the exact one-line test that settles it (`/bin/bash -c 'set -u; unset X; for f in "${X[@]}"; do :; done; echo reached'`) without access to this host.
ltms commented 2026-09-12 05:21:45 +02:00
Author
Owner
Copy Link
Copy Source

Settled on real bash 3.2, and my earlier reasoning here was incomplete

The fleet01 lead challenged the severity: if set -u fires on the array expansion, the script
aborts loudly rather than reporting a confident wrong answer, which is the opposite outcome and a
different family from #497. They could not test it — they have bash 5.2.21 and no 3.2.

This Mac's /bin/bash is 3.2.57. Measured here, both interpreters, same commands:

set -u; unset X; for f in "${X[@]}"; do :; done; echo reached
  3.2.57 -> rc=127, "X[@]: unbound variable"
  5.3.9  -> rc=0, "reached"

set -u; unset X; echo count=${#X[@]}
  3.2.57 -> rc=1, "X: unbound variable"

Their expectation was right. On 3.2 both the bare form and the counting form fire; on 5.x the
bare form does not. The 4.4 change they named is real.

The conclusion still does not follow for this script, because it uses neither form. I measured
every consumer out of the file rather than the two forms we were arguing about:

line expansion fires set -u on 3.2?
:150-154 ${_FIELDS[0]:-null} … ${_FIELDS[4]:-0} no — a :- default beats it
:155 NAMES=("${_FIELDS[@]:5}") — a slice no (measured: rc=0, count=0)
:172 [ "${#NAMES[@]}" -eq 0 ] on an assigned-but-empty array no (measured: prints count=0)
:225 for name in "${NAMES[@]}" — the bare form yes — but unreachable

The slice at :155 is the load-bearing line and neither of us had tested it:

set -uo pipefail; unset X; NAMES=("${X[@]:5}"); echo count=${#NAMES[@]}
  3.2.57 -> rc=0, count=0
  5.3.9  -> rc=0, count=0

A slice of an unset array does not fire set -u on 3.2. That leaves NAMES assigned and empty
rather than unset, which is why ${#NAMES[@]} at :172 then works:

set -uo pipefail; NAMES=(); echo count=${#NAMES[@]}; for f in "${NAMES[@]}"; do :; done
  3.2.57 -> count=0 prints, THEN rc=127 "NAMES[@]: unbound variable"

The chain, end to end, on bash 3.2 with mapfile absent

mapfile → command not found, rc=127 on stderr. No set -e, so it continues; _FIELDS is unset.
:150-154 all carry :- defaults, so nothing fires and KNOWN_COUNT_REPORTED=0. :155 is a slice,
so no abort, and NAMES=() is assigned. :172 counts an assigned array, gets 0, and the guard
fires with its own message: "refusing to run: the policy fetched from … contains 0 known names
(present=null)." — then exit 1.

The abort at :225 is real on 3.2 and never reached, because :172 exits first.

So this stays in #497's family. Three separate escapes sit between the failure and the loud
form: a :- default, a slice, and a count on an assigned array. Change any one of them to the bare
form and the challenge would be correct.

Correcting myself, twice

  1. My first comment credited the survival to the :- defaults alone. That was incomplete. :155's
    slice is what keeps NAMES assigned; without it :172 would abort on its own.
  2. I earlier called the failure "silent". It is not. mapfile: command not found goes to stderr,
    and then a confident wrong refusal is printed after it. That is worse than silent — an operator
    who reads a refusal with a stated reason stops reading the line above it.

The method lesson, which is theirs

My original repro dropped the script's own set -uo pipefail line, and the dropped precondition is
exactly where the behaviour lives. Copying that line in was necessary and not sufficient: I then
measured the two forms under discussion instead of the forms the file actually contains. Measure the
call, the set line, and every consumer — read out of the file, not out of the argument.

Fix unchanged

Still "make it report honestly" — the refusal must distinguish "the policy really has 0 names"
from "I could not parse the policy", which is #497's third-state fix. Separately and lower
priority: make it run on the interpreter it may be invoked with, since mapfile is bash 4+ and
macOS ships 3.2.

## Settled on real bash 3.2, and my earlier reasoning here was incomplete The fleet01 lead challenged the severity: if `set -u` fires on the array expansion, the script **aborts loudly** rather than reporting a confident wrong answer, which is the opposite outcome and a different family from #497. They could not test it — they have bash 5.2.21 and no 3.2. This Mac's `/bin/bash` is **3.2.57**. Measured here, both interpreters, same commands: ``` set -u; unset X; for f in "${X[@]}"; do :; done; echo reached 3.2.57 -> rc=127, "X[@]: unbound variable" 5.3.9 -> rc=0, "reached" set -u; unset X; echo count=${#X[@]} 3.2.57 -> rc=1, "X: unbound variable" ``` **Their expectation was right.** On 3.2 both the bare form and the counting form fire; on 5.x the bare form does not. The 4.4 change they named is real. **The conclusion still does not follow for this script, because it uses neither form.** I measured every consumer out of the file rather than the two forms we were arguing about: | line | expansion | fires `set -u` on 3.2? | |---|---|---| | `:150-154` | `${_FIELDS[0]:-null}` … `${_FIELDS[4]:-0}` | no — a `:-` default beats it | | `:155` | `NAMES=("${_FIELDS[@]:5}")` — a **slice** | **no** (measured: rc=0, count=0) | | `:172` | `[ "${#NAMES[@]}" -eq 0 ]` on an assigned-but-empty array | no (measured: prints `count=0`) | | `:225` | `for name in "${NAMES[@]}"` — the bare form | **yes** — but unreachable | The slice at `:155` is the load-bearing line and neither of us had tested it: ``` set -uo pipefail; unset X; NAMES=("${X[@]:5}"); echo count=${#NAMES[@]} 3.2.57 -> rc=0, count=0 5.3.9 -> rc=0, count=0 ``` A slice of an unset array does not fire `set -u` on 3.2. That leaves `NAMES` **assigned and empty** rather than unset, which is why `${#NAMES[@]}` at `:172` then works: ``` set -uo pipefail; NAMES=(); echo count=${#NAMES[@]}; for f in "${NAMES[@]}"; do :; done 3.2.57 -> count=0 prints, THEN rc=127 "NAMES[@]: unbound variable" ``` ## The chain, end to end, on bash 3.2 with `mapfile` absent `mapfile` → `command not found`, rc=127 on stderr. No `set -e`, so it continues; `_FIELDS` is unset. `:150-154` all carry `:-` defaults, so nothing fires and `KNOWN_COUNT_REPORTED=0`. `:155` is a slice, so no abort, and `NAMES=()` is assigned. `:172` counts an assigned array, gets 0, and the guard fires with its own message: *"refusing to run: the policy fetched from … contains 0 known names (present=null)."* — then `exit 1`. The abort at `:225` is real on 3.2 and **never reached**, because `:172` exits first. **So this stays in #497's family.** Three separate escapes sit between the failure and the loud form: a `:-` default, a slice, and a count on an assigned array. Change any one of them to the bare form and the challenge would be correct. ## Correcting myself, twice 1. My first comment credited the survival to the `:-` defaults alone. That was incomplete. `:155`'s **slice** is what keeps `NAMES` assigned; without it `:172` would abort on its own. 2. I earlier called the failure "silent". It is not. `mapfile: command not found` goes to stderr, and then a confident wrong refusal is printed after it. That is worse than silent — an operator who reads a refusal with a stated reason stops reading the line above it. ## The method lesson, which is theirs My original repro dropped the script's own `set -uo pipefail` line, and the dropped precondition is exactly where the behaviour lives. Copying that line in was necessary and **not sufficient**: I then measured the two forms under discussion instead of the forms the file actually contains. Measure the call, the `set` line, and every consumer — read out of the file, not out of the argument. ## Fix unchanged Still "make it report honestly" — the refusal must distinguish *"the policy really has 0 names"* from *"I could not parse the policy"*, which is #497's third-state fix. Separately and lower priority: make it run on the interpreter it may be invoked with, since `mapfile` is bash 4+ and macOS ships 3.2.
ltms commented 2026-09-12 05:25:36 +02:00
Author
Owner
Copy Link
Copy Source

The fix section above is wrong. Do not brief a worker from it.

The fleet01 lead drew a consequence I had missed, and it invalidates the fix I asked for. Measured
here before accepting it.

The guard at :172 cannot be made honest, because the fact is gone by then

I ran the two states side by side rather than reasoning about them:

A. producer FAILS   (mapfile absent, /bin/bash 3.2.57)
     PRESENT=[null] KNOWN=[0] NAMES=0
B. producer SUCCEEDS, legitimately returns null/0   (bash 5.3.9, mapfile present)
     PRESENT=[null] KNOWN=[0] NAMES=0

Byte-identical. So the real sentinel is not the message at :172 — it is
${_FIELDS[0]:-null} … ${_FIELDS[4]:-0} at :150-154, five lines of it, one layer below the
guard I filed this ticket on. A :- default cannot fire set -u by construction, and it silently
manufactures the same values a healthy producer would have returned.

By the time control reaches :172 the distinction no longer exists in any variable. No rewording
of that message can recover it. A worker told "make it report honestly" will improve the sentence
and change nothing measurable. That is my error and it would have cost a worker's turn.

The fix has to be at or above :128.

What actually works — each measured, with a control

1. A top-level interpreter gate. States the requirement, and being top-level it is the one place
a guard cannot be defeated by its call site.

(( BASH_VERSINFO[0] >= 4 )) || die "requires bash 4+ (mapfile); running $BASH_VERSION"

3.2.57 -> rc=3, fires. BASH_VERSINFO[0]=3, BASH_VERSION=3.2.57(1)-release
5.3.9  -> rc=0, passes            (control: requirement met)

BASH_VERSINFO does exist in 3.2, so the gate can report the version that cannot run the script.

2. Check the producer's own exit status — but not the obvious way.

The obvious form does not work, and this is the part worth reading. The fleet01 lead proposed
if ! mapfile -t _FIELDS < <(...); then die; fi and said it "catches every reason the producer
fails". It does not:

bash 5.x, mapfile PRESENT, producer inside the process substitution fails:
  if ! mapfile -t _F < <(nosuchtool 2>/dev/null); then echo DETECTED; fi
    -> rc=0, "NOT detected", count=0

isolated:
  mapfile -t X < <(nosuchtool 2>/dev/null); echo "rc=$? count=${#X[@]}"
    -> mapfile rc=0 count=0

mapfile's exit status is mapfile's own. A process substitution's status is not propagated to it,
and set -o pipefail does not reach inside < <(...) because that is not a pipeline. So this form
catches the builtin being absent (measured rc=4 on 3.2, correctly) and misses jq or python3
failing, which is the other half of this script.

Use command substitution instead, whose status is the producer's:

if ! _RAW="$(printf '%s' "$POLICY_JSON" | jq -r '...')"; then
  die "could not read the policy: the parser failed"
fi
mapfile -t _FIELDS <<< "$_RAW"

producer missing                  -> rc=5, DETECTED
producer missing at end of a pipe -> rc=5, DETECTED via pipefail   (this script's real shape)
producer works                    -> rc=0, count=5                 (control)

Note that third line: set -o pipefail at :67 is load-bearing in this form and is not
load-bearing in the current code, because < <(...) gives it no pipeline to act on.

Revised asks

  1. Add the BASH_VERSINFO gate at the top. It states the requirement and refuses rather than
    guessing.
  2. Replace both mapfile -t _FIELDS < <(...) calls (:128, :136) with the command-substitution
    form above, so a parser failure is detected at the call, while the fact still exists.
  3. Only then adjust :172. Once a producer failure dies at :128, reaching :172 really does mean
    the policy has 0 names, and the existing message becomes true instead of merely confident.
  4. Keep set -uo pipefail. Do not add set -e — that is a separate behaviour change.

Tests: one where the parser fails and the script must die naming the parser, not the policy; one
where the policy legitimately has 0 names and the script must die naming the policy. Those two must
produce different messages — that is the whole point, and a single test cannot show it.

Why the original ask was wrong

I filed this on the symptom an operator sees and prescribed a fix at that spot. The sentinel was five
lines earlier, where a default quietly invented the values. Same lesson as the set -u work above,
one level up: measure every consumer, and ask at which line the fact still exists. A message can
only report what some variable still knows.

## The fix section above is wrong. Do not brief a worker from it. The fleet01 lead drew a consequence I had missed, and it invalidates the fix I asked for. Measured here before accepting it. ### The guard at `:172` cannot be made honest, because the fact is gone by then I ran the two states side by side rather than reasoning about them: ``` A. producer FAILS (mapfile absent, /bin/bash 3.2.57) PRESENT=[null] KNOWN=[0] NAMES=0 B. producer SUCCEEDS, legitimately returns null/0 (bash 5.3.9, mapfile present) PRESENT=[null] KNOWN=[0] NAMES=0 ``` Byte-identical. So the real sentinel is not the message at `:172` — it is `${_FIELDS[0]:-null}` … `${_FIELDS[4]:-0}` at **`:150-154`**, five lines of it, one layer *below* the guard I filed this ticket on. A `:-` default cannot fire `set -u` by construction, and it silently manufactures the same values a healthy producer would have returned. By the time control reaches `:172` the distinction no longer exists in any variable. **No rewording of that message can recover it.** A worker told "make it report honestly" will improve the sentence and change nothing measurable. That is my error and it would have cost a worker's turn. **The fix has to be at or above `:128`.** ### What actually works — each measured, with a control **1. A top-level interpreter gate.** States the requirement, and being top-level it is the one place a guard cannot be defeated by its call site. ```bash (( BASH_VERSINFO[0] >= 4 )) || die "requires bash 4+ (mapfile); running $BASH_VERSION" ``` ``` 3.2.57 -> rc=3, fires. BASH_VERSINFO[0]=3, BASH_VERSION=3.2.57(1)-release 5.3.9 -> rc=0, passes (control: requirement met) ``` `BASH_VERSINFO` does exist in 3.2, so the gate can report the version that cannot run the script. **2. Check the producer's own exit status — but not the obvious way.** The obvious form does **not** work, and this is the part worth reading. The fleet01 lead proposed `if ! mapfile -t _FIELDS < <(...); then die; fi` and said it "catches every reason the producer fails". It does not: ``` bash 5.x, mapfile PRESENT, producer inside the process substitution fails: if ! mapfile -t _F < <(nosuchtool 2>/dev/null); then echo DETECTED; fi -> rc=0, "NOT detected", count=0 isolated: mapfile -t X < <(nosuchtool 2>/dev/null); echo "rc=$? count=${#X[@]}" -> mapfile rc=0 count=0 ``` `mapfile`'s exit status is `mapfile`'s own. A process substitution's status is not propagated to it, and `set -o pipefail` does not reach inside `< <(...)` because that is not a pipeline. So this form catches the **builtin being absent** (measured rc=4 on 3.2, correctly) and misses **`jq` or `python3` failing**, which is the other half of this script. Use command substitution instead, whose status *is* the producer's: ```bash if ! _RAW="$(printf '%s' "$POLICY_JSON" | jq -r '...')"; then die "could not read the policy: the parser failed" fi mapfile -t _FIELDS <<< "$_RAW" ``` ``` producer missing -> rc=5, DETECTED producer missing at end of a pipe -> rc=5, DETECTED via pipefail (this script's real shape) producer works -> rc=0, count=5 (control) ``` Note that third line: `set -o pipefail` at `:67` is load-bearing in this form and is **not** load-bearing in the current code, because `< <(...)` gives it no pipeline to act on. ### Revised asks 1. Add the `BASH_VERSINFO` gate at the top. It states the requirement and refuses rather than guessing. 2. Replace both `mapfile -t _FIELDS < <(...)` calls (`:128`, `:136`) with the command-substitution form above, so a parser failure is detected **at the call**, while the fact still exists. 3. Only then adjust `:172`. Once a producer failure dies at `:128`, reaching `:172` really does mean the policy has 0 names, and the existing message becomes true instead of merely confident. 4. Keep `set -uo pipefail`. Do not add `set -e` — that is a separate behaviour change. Tests: one where the parser fails and the script must die naming the parser, not the policy; one where the policy legitimately has 0 names and the script must die naming the policy. Those two must produce **different** messages — that is the whole point, and a single test cannot show it. ### Why the original ask was wrong I filed this on the symptom an operator sees and prescribed a fix at that spot. The sentinel was five lines earlier, where a default quietly invented the values. Same lesson as the `set -u` work above, one level up: **measure every consumer, and ask at which line the fact still exists.** A message can only report what some variable still knows.
ltms commented 2026-09-12 05:29:56 +02:00
Author
Owner
Copy Link
Copy Source

It is a three-way conflation, not a two-way one — and the third cause is the one that matters

The fleet01 lead derived this from the lines I quoted and flagged it as unmeasured. I measured it.
It holds, and the new cause is worse than the one this ticket was filed for.

:172 fires with the same message for at least three different causes:

  1. the interpreter has no mapfile — the original finding, macOS bash 3.2;
  2. the producer ran and returned fewer than 5 fields — the slice at :155 is then empty, so
    NAMES is empty, whatever the policy actually contained;
  3. the policy genuinely has 0 known names — the only case the message describes.

Cause 2, measured

Malformed JSON reaching jq. jq exits non-zero and emits nothing:

_FIELDS=0 PRESENT=[null] KNOWN=[0] NAMES=0
-> GUARD FIRES: same message as a 0-name policy

Control, a healthy producer with two names:

_FIELDS=7 PRESENT=[true] KNOWN=[2] NAMES=2 -> A B

Why cause 2 is the serious one

Cause 1 is version-gated. It happens on macOS bash 3.2 and nowhere else, and a BASH_VERSINFO gate
closes it completely.

Cause 2 survives every interpreter. It fires whenever the producer's output shape drifts — a
schema change upstream, a jq filter that stops matching, a python3 that raises after printing
nothing. The result is that a schema change reports itself as an empty policy, on a credential
probe, in the direction that says nothing is protected. Nobody would look at the interpreter,
because the interpreter is fine.

And it is silent for the reason already established on this ticket: the slice at :155 does not
fire set -u on either bash, so an under-length _FIELDS produces an empty NAMES with no error at
all. Cause 1 at least prints mapfile: command not found above the wrong conclusion. Cause 2 prints
nothing above it.

What this changes in the fix

The revised asks in my previous comment are still right, but the reason is now broader, and a
version gate alone is not enough:

  • The BASH_VERSINFO gate closes cause 1 only. Keep it — it states the requirement — but do not
    treat it as the fix.
  • The command-substitution form closes cause 2 as well, because it detects a non-zero producer
    at the call while the fact still exists. That is now the load-bearing change, not the optional one.
  • Add an explicit arity check after the parse, before the slice: if _FIELDS has fewer than 5
    entries, die naming the parser and the count. A producer that exits 0 and prints a short result
    still needs catching, and neither the version gate nor the exit-status check sees that.

Tests must now distinguish three outcomes with three different messages: parser failed, parser
returned a short/unexpected shape, policy genuinely empty. One test cannot show that; three can.

Credit and method

This came from the peer reading the lines I had quoted and asking what else reaches that guard. It
is the same lesson as the set -u work above, applied to the other axis: I enumerated the consumers
and stopped at "which of these fires set -u", without asking "which causes converge on this
one message". Enumerate the causes that reach a guard, not only the mechanisms that pass through
it.

## It is a three-way conflation, not a two-way one — and the third cause is the one that matters The fleet01 lead derived this from the lines I quoted and flagged it as unmeasured. I measured it. **It holds**, and the new cause is worse than the one this ticket was filed for. `:172` fires with the **same message** for at least three different causes: 1. the interpreter has no `mapfile` — the original finding, macOS bash 3.2; 2. **the producer ran and returned fewer than 5 fields** — the slice at `:155` is then empty, so `NAMES` is empty, whatever the policy actually contained; 3. the policy genuinely has 0 known names — the only case the message describes. ### Cause 2, measured Malformed JSON reaching `jq`. `jq` exits non-zero and emits nothing: ``` _FIELDS=0 PRESENT=[null] KNOWN=[0] NAMES=0 -> GUARD FIRES: same message as a 0-name policy ``` Control, a healthy producer with two names: ``` _FIELDS=7 PRESENT=[true] KNOWN=[2] NAMES=2 -> A B ``` ### Why cause 2 is the serious one Cause 1 is version-gated. It happens on macOS bash 3.2 and nowhere else, and a `BASH_VERSINFO` gate closes it completely. **Cause 2 survives every interpreter.** It fires whenever the producer's output shape drifts — a schema change upstream, a `jq` filter that stops matching, a `python3` that raises after printing nothing. The result is that **a schema change reports itself as an empty policy**, on a credential probe, in the direction that says nothing is protected. Nobody would look at the interpreter, because the interpreter is fine. And it is silent for the reason already established on this ticket: the slice at `:155` does not fire `set -u` on either bash, so an under-length `_FIELDS` produces an empty `NAMES` with no error at all. Cause 1 at least prints `mapfile: command not found` above the wrong conclusion. Cause 2 prints nothing above it. ### What this changes in the fix The revised asks in my previous comment are still right, but the **reason** is now broader, and a version gate alone is not enough: - The `BASH_VERSINFO` gate closes **cause 1 only**. Keep it — it states the requirement — but do not treat it as the fix. - The command-substitution form closes **cause 2 as well**, because it detects a non-zero producer at the call while the fact still exists. That is now the load-bearing change, not the optional one. - Add an explicit **arity check** after the parse, before the slice: if `_FIELDS` has fewer than 5 entries, die naming the parser and the count. A producer that exits 0 and prints a short result still needs catching, and neither the version gate nor the exit-status check sees that. Tests must now distinguish **three** outcomes with three different messages: parser failed, parser returned a short/unexpected shape, policy genuinely empty. One test cannot show that; three can. ### Credit and method This came from the peer reading the lines I had quoted and asking what *else* reaches that guard. It is the same lesson as the `set -u` work above, applied to the other axis: I enumerated the consumers and stopped at "which of these fires `set -u`", without asking "which **causes** converge on this one message". **Enumerate the causes that reach a guard, not only the mechanisms that pass through it.**
ltms commented 2026-09-12 05:37:23 +02:00
Author
Owner
Copy Link
Copy Source

Correction: I attributed the detection to pipefail and that was wrong

In my comment above I wrote:

producer missing at end of a pipe -> rc=5, DETECTED via pipefail (this script's real shape)

and concluded that set -o pipefail at :67 is load-bearing for the command-substitution fix. It
is not, for this script's current shape. The fleet01 lead caught it. I had measured a detection and
never ran the control that would have told me what caused it.

Measured now, the full 2×2 plus a control:

no pipefail with pipefail
A/B failing command last in the pipe DETECTED DETECTED
C/D failing command upstream NOT detected DETECTED
E control, nothing fails — NOT detected (correct), lines=1

Only row C/D moves. In printf '%s' "$POLICY_JSON" | jq -r '...', jq is the last element, so
the pipeline's exit status is jq's status with or without pipefail. My "DETECTED via pipefail"
was a true detection with a false cause.

What to write in the code instead

The constraint itself does not change — keep set -o pipefail — but the reason must be stated
correctly, because this is going into a comment a contributor reads:

Keep pipefail: it is not load-bearing for the current single-stage pipes, where the parser is
the last element and its own status is the pipeline's. It is the only guard if a post-processing
stage is ever appended — jq ... | tail -n +2, python3 ... | tr -d '\r' — at which point the
failing producer sits upstream and its status is discarded without it.

Today it is insurance. It becomes the mechanism the moment someone appends a stage.

Why this correction matters more than its size

This is my own finding from this ticket, turned on the fix instead of the defect: a stated reason
that is wrong stops the next reader looking further. That is exactly why the :172 refusal is
worse than a silent failure. A comment saying "pipefail catches this" would have persuaded the next
contributor that the single-stage pipe was already guarded, and they would not have looked at which
element actually reports the status.

Method note, and it is the same error twice in this ticket from opposite directions

The peer proposed if ! mapfile -t F < <(producer) and I disproved it by running it. I proposed
"pipefail catches the pipe case" and they disproved it by running it. Both of us applied a correct
general rule — check the producer's status at the call — to a construct without checking whose
status that construct reports. mapfile reports mapfile's. A pipeline reports its last element's.

The rule that catches both: before claiming a construct propagates a status, measure which
component's status it propagates, and include the control where nothing fails. A detection with no
control is a detection with an unknown cause.

One thing neither of us has measured

mapfile -t F < <(producer) streams; mapfile -t F <<< "$_RAW" materialises the producer's
whole output in a shell variable first. For a credential policy document that is irrelevant. If that
producer ever emits something large, the swap is a behavioural change and not only an error-handling
one. Flagged on the ticket rather than assumed away — whoever implements this should say which they
checked.

## Correction: I attributed the detection to `pipefail` and that was wrong In my comment above I wrote: > `producer missing at end of a pipe -> rc=5, DETECTED via pipefail (this script's real shape)` and concluded that `set -o pipefail` at `:67` is load-bearing for the command-substitution fix. **It is not, for this script's current shape.** The fleet01 lead caught it. I had measured a detection and never ran the control that would have told me what caused it. Measured now, the full 2×2 plus a control: | | no `pipefail` | with `pipefail` | |---|---|---| | **A/B** failing command **last** in the pipe | DETECTED | DETECTED | | **C/D** failing command **upstream** | NOT detected | **DETECTED** | | **E** control, nothing fails | — | NOT detected (correct), lines=1 | Only row C/D moves. In `printf '%s' "$POLICY_JSON" | jq -r '...'`, `jq` is the **last** element, so the pipeline's exit status is `jq`'s status with or without `pipefail`. My "DETECTED via pipefail" was a true detection with a false cause. ### What to write in the code instead The constraint itself does not change — **keep `set -o pipefail`** — but the reason must be stated correctly, because this is going into a comment a contributor reads: > Keep `pipefail`: it is **not** load-bearing for the current single-stage pipes, where the parser is > the last element and its own status is the pipeline's. It is the only guard if a post-processing > stage is ever appended — `jq ... | tail -n +2`, `python3 ... | tr -d '\r'` — at which point the > failing producer sits upstream and its status is discarded without it. Today it is insurance. It becomes the mechanism the moment someone appends a stage. ### Why this correction matters more than its size This is my own finding from this ticket, turned on the fix instead of the defect: **a stated reason that is wrong stops the next reader looking further.** That is exactly why the `:172` refusal is worse than a silent failure. A comment saying "pipefail catches this" would have persuaded the next contributor that the single-stage pipe was already guarded, and they would not have looked at which element actually reports the status. ### Method note, and it is the same error twice in this ticket from opposite directions The peer proposed `if ! mapfile -t F < <(producer)` and I disproved it by running it. I proposed "pipefail catches the pipe case" and they disproved it by running it. Both of us applied a correct general rule — *check the producer's status at the call* — to a **construct without checking whose status that construct reports**. `mapfile` reports `mapfile`'s. A pipeline reports its last element's. The rule that catches both: **before claiming a construct propagates a status, measure which component's status it propagates, and include the control where nothing fails.** A detection with no control is a detection with an unknown cause. ### One thing neither of us has measured `mapfile -t F < <(producer)` **streams**; `mapfile -t F <<< "$_RAW"` **materialises** the producer's whole output in a shell variable first. For a credential policy document that is irrelevant. If that producer ever emits something large, the swap is a behavioural change and not only an error-handling one. Flagged on the ticket rather than assumed away — whoever implements this should say which they checked.
ltms closed this issue 2026-09-12 06:10:43 +02:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
blocked
Cannot start until something else lands. The body says what.
 needs-live-proof
Merged and green, but never shown working on the running daemon. Not the same as done.
 ready-to-delegate
Scope, files and acceptance criteria are written. A worker can be briefed from the body alone.
 silent-default
A feature that compiles, passes tests, and ships turned off. Nine recurrences and counting.
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent kevin ltms
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: fleet/fleetd#500
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?