fleetd #504 item 1: stop swallowing real launchctl/systemctl failures on the 'loaded but not running' path
The two 'loaded but not currently running' branches in the stop step (launchd/systemd, reached when $OLD_PID is empty) ran 'launchctl unload'/'systemctl --user stop' with '2>/dev/null || true' and printed 'ok' unconditionally. That swallowed a real supervisor failure (e.g. launchd or the systemd user bus unreachable) exactly like a harmless already-stopped answer, and let the script proceed to start a new daemon believing nothing was loaded -- the two-daemons failure fleetd #492 exists to prevent. Adds unload_launchd_if_loaded/stop_systemd_if_loaded, applying systemd_loaded's own pattern (capture stderr separately; a non-zero exit WITH stderr is a real failure, a non-zero exit with empty stderr is a clean already-stopped answer) to the write side. The two call sites now use these functions instead of the bare '|| true'. Adds 5 tests: dies-on-real-failure and tolerates-clean-negative for each function, plus a source-text check that the main flow calls the new functions instead of the original bare '2>/dev/null || true'. All 5 verified by mutation (reintroducing the swallow, and separately over-correcting to die unconditionally) -- each goes red with its own message, restores byte-identical (full sha256), and passes a green control.
This commit is contained in:
@@ -287,6 +287,49 @@ systemd_loaded() {
|
||||
return "$rc"
|
||||
}
|
||||
|
||||
# fleetd #504: the "loaded but not currently running" branches in the main stop step (case
|
||||
# launchd/systemd, reached when $OLD_PID is empty) used to run `launchctl unload`/`systemctl --user
|
||||
# stop` with `2>/dev/null || true` and then print `ok` unconditionally — the exact conflation
|
||||
# systemd_loaded/systemd_installed above already fixed on the READ side (fleetd #492 follow-up): a
|
||||
# genuine "already stopped" answer (nonzero exit, nothing on stderr) is harmless, but a real tool
|
||||
# failure (nonzero exit WITH a stderr message — e.g. launchd or the systemd user bus is
|
||||
# unreachable) is not, and reporting `ok` on THAT means the start step below can register a fresh
|
||||
# load on top of a supervisor that never actually let go: the exact two-daemons failure fleetd #492
|
||||
# exists to prevent, reached from the one state (already odd) where a false `ok` is least
|
||||
# affordable. These two functions apply the same "capture stderr separately, flag only a nonzero
|
||||
# exit WITH stderr as a real failure" pattern to the WRITE side. No ${VAR:-default} anywhere here —
|
||||
# see the #492 follow-up constraints comment above detect_supervisor for why a default would hide a
|
||||
# lost value instead of surfacing it (fleetd #497's defect class).
|
||||
unload_launchd_if_loaded() {
|
||||
local err_file rc=0
|
||||
if ! err_file="$(mktemp -t launchd-unload-err)"; then
|
||||
die "could not create a temp file to capture 'launchctl unload' stderr — cannot tell a real
|
||||
failure from a clean already-unloaded answer, so refusing to guess. The daemon's
|
||||
supervision state was NOT touched."
|
||||
fi
|
||||
launchctl unload -w "$LAUNCHD_PLIST" >/dev/null 2>"$err_file" || rc=$?
|
||||
if [ "$rc" -ne 0 ] && [ -s "$err_file" ]; then
|
||||
die "'launchctl unload -w $LAUNCHD_PLIST' failed: $(cat "$err_file")
|
||||
The daemon may still be under supervision; investigate before retrying."
|
||||
fi
|
||||
rm -f "$err_file"
|
||||
}
|
||||
|
||||
stop_systemd_if_loaded() {
|
||||
local err_file rc=0
|
||||
if ! err_file="$(mktemp -t systemd-stop-err)"; then
|
||||
die "could not create a temp file to capture 'systemctl --user stop' stderr — cannot tell a
|
||||
real failure from a clean already-stopped answer, so refusing to guess. The daemon's
|
||||
supervision state was NOT touched."
|
||||
fi
|
||||
systemctl --user stop "$SYSTEMD_UNIT" >/dev/null 2>"$err_file" || rc=$?
|
||||
if [ "$rc" -ne 0 ] && [ -s "$err_file" ]; then
|
||||
die "'systemctl --user stop $SYSTEMD_UNIT' failed: $(cat "$err_file")
|
||||
The daemon may still be under supervision; investigate before retrying."
|
||||
fi
|
||||
rm -f "$err_file"
|
||||
}
|
||||
|
||||
# fleetd #492: three real answers, not two — launchd, systemd, or genuinely unsupervised — plus a
|
||||
# fourth, "ambiguous", for the one case this script cannot tell apart: both signals firing at once.
|
||||
# That is exactly "I cannot tell who supervises this process", and guessing wrong here is how two
|
||||
@@ -881,16 +924,20 @@ if [ -n "$OLD_PID" ]; then
|
||||
elif [ "$SUPERVISOR_KIND" = "launchd" ]; then
|
||||
# Loaded but not currently running (e.g. throttled after a crash loop). Unload it anyway so the
|
||||
# start step below does a clean load, never a load stacked on an already-loaded label.
|
||||
# fleetd #504: unload_launchd_if_loaded (above) tolerates a genuine already-unloaded answer but
|
||||
# dies on a real `launchctl` failure — never a bare `|| true` that would print `ok` either way.
|
||||
say "stop"
|
||||
RESTART_MARK="$(wc -l < "$OUT" 2>/dev/null || echo 0)"
|
||||
launchctl unload -w "$LAUNCHD_PLIST" 2>/dev/null || true
|
||||
unload_launchd_if_loaded
|
||||
ok "launchd agent unloaded (was already not running)"
|
||||
elif [ "$SUPERVISOR_KIND" = "systemd" ]; then
|
||||
# Same case for systemd: the unit is known/active-capable but not currently running. `stop` on an
|
||||
# already-stopped unit is a harmless no-op — kept for symmetry with the launchd branch above.
|
||||
# fleetd #504: stop_systemd_if_loaded (above) tolerates that genuine no-op but dies on a real
|
||||
# `systemctl` failure — never a bare `|| true` that would print `ok` either way.
|
||||
say "stop"
|
||||
RESTART_MARK="$(wc -l < "$OUT" 2>/dev/null || echo 0)"
|
||||
systemctl --user stop "$SYSTEMD_UNIT" 2>/dev/null || true
|
||||
stop_systemd_if_loaded
|
||||
ok "systemd --user unit stopped (was already not running)"
|
||||
else
|
||||
RESTART_MARK="$(wc -l < "$OUT" 2>/dev/null || echo 0)"
|
||||
|
||||
Reference in New Issue
Block a user