24 Commits

Author	SHA1	Message	Date
Dai Ha	81c1d8e91c	fleetd #603: share HEALTH_WAIT between the pid poll and the health check ... The start step gave the new process its own short, fixed 10s budget before a hard die, while the health check right after it waits a full HEALTH_WAIT (60s) for the same daemon. Under launchd, launchctl load returns before the java process exists, and on a slow host that took longer than 10s -- so the script reported "no process appeared" on a deploy that had fully succeeded. wait_for_new_pid/await_daemon_started fold the pid poll and the health check into one decision: the pid poll now shares HEALTH_WAIT instead of its own shorter budget, and a miss there falls through to the health check (direct proof the daemon is up) instead of killing the run. A genuine failure still dies, and still prints the log tail. Adds behavioural tests for both acceptance criteria (slow start succeeds, genuine failure still fails and prints the log tail) in one suite run, plus unit tests for wait_for_new_pid and a call-site test for the new function.	2026-09-20 16:21:52 +07:00
Dai Ha	4b9ebda1b3	fleetd #593 CORRECTION 1: allowlist comm=java, not a denylist of shells ... The round-1 fix excluded known shell names (sh/bash/zsh/dash/ksh) from running_pid()'s pgrep candidates. Two holes remained, both the same false-positive shape the ticket exists to remove: 1. A pid pgrep lists can exit before the following `ps -o comm=` lookup runs. On a gone pid, ps prints nothing, comm is empty, and an empty string matches no denied shell name -- so a dead pid was still counted. 2. The denylist only knows the shells someone thought to name. ssh, perl, python3, ruby, tail -- anything else carrying the pattern in its own argv -- was still counted alongside the real daemon. The ticket names ssh as a live route. Both close with one change: allowlist comm=java instead of denying shells. The daemon is always `java -jar target/fleetd.jar`, so its comm is always `java`; an empty comm (hole 1) is not `java` either, closing that hole for free. Answers the objection in the code comment: an allowlist can under-count if fleetd ever stops being launched by `java` (a native image, a renamed launcher). That's a false negative, the worse direction for a guard -- but it is not a new assumption: PATTERN='target/fleetd.jar' already assumes a jar run by java, and that pattern breaks before this allowlist would. Replaces the round-1 "real second process" test (which gave its exec -a standin an argv[0] holding the pattern, but not comm=java) with one that forces comm=java via `exec -a java sh -c '...'`. Adds two stubbed pgrep/ps tests pinning the two holes directly (a non-java, non-shell comm such as perl; an empty comm from an already-exited pid) -- deterministic on every platform, unlike a live-process fixture, and immune to the BSD vs Linux difference in how `comm` is derived from a fabricated process. Adds a stubbed positive backstop (comm=java is counted). Confirmed the regression is caught: reverted to the round-1 denylist, reran the suite, watched the new non-shell-comm test fail at `set -e`'s first failure, then isolated the exited-pid test separately and confirmed it also fails against the same broken code. Restored the fix and reran green. Branch merged with origin/main (3 commits: hunter role + CLAUDE.md addendum) before this commit; unrelated, no conflicts.	2026-09-19 15:31:15 +07:00
Dai Ha	42820fbe75	fleetd #593 (pid-count half): running_pid() no longer matches the caller ... running_pid() was a bare `pgrep -f "$PATTERN"`, which matches ANY process whose full command line contains the pattern text -- including a shell that merely embeds it as literal text (a hand-typed investigation, an ssh-shaped `sh -c '...; ...'`, or a pipeline) rather than being the daemon. That self-match turns a working redeploy into a reported "racing supervisor" failure via assert_single_daemon. pgrep -c does not exist on BSD/macOS, so this can't be fixed by switching flags. running_pid() now keeps pgrep to find candidates (portable), then drops any candidate whose process name (comm) names a shell -- the daemon is always `java`, so a self-matching wrapper of this shape is always excluded while a genuine second daemon-shaped process still counts. assert_single_daemon's die message no longer hands the operator a bare `pgrep -f "$PATTERN"` as remediation -- that was exactly the self-matching invocation -- and now says in words that a pattern can match the caller. Adds three tests: a self-matching wrapper shell must be excluded, a real second daemon-shaped process must still be found, and the die message must not recommend the self-matching command. Verified the first test fails against the pre-fix implementation (confirmed the regression is caught). Leaves instance 1 (the fleetd.out log source, systemd-only) for a Linux host, per the ticket's scope split.	2026-09-19 15:18:32 +07:00
Dai Ha	8d79d229ff	fleetd #555: lift 8 main-flow decisions into tested predicate/dispatch functions ... redeploy-fleetd.sh's main flow had 8 bare if/case decisions (CHECK_ONLY short-circuit, drain-gate entry+confirm, supervisor report/stop/start dispatch, health-poll decision, HAD_OLD_PID computation) that lived outside any function, so the 67-test suite could not reach them and any one could be silently inverted with the whole suite green. Follows the existing swap_if_built/refuse_drain_gate pattern: each bare guard becomes a small predicate or dispatch function (should_stop_for_check, drain_gate_required/drain_confirmed/run_drain_gate, report_supervisor_state, dispatch_stop, dispatch_start, health_is_up/report_health, compute_had_old_pid), called unconditionally by the main flow so the decision itself is unit-testable in isolation. Adds a structural guard, test_no_untested_main_flow_conditionals, that scans the main flow (everything after the SOURCED guard) for bare if/elif/case lines outside any function body, tracking function boundaries via this file's one consistent name() { / } convention. It fails on any new bare conditional not covered by MAIN_FLOW_ALLOWED_CONDITIONALS, an explicit exact-text allowlist of the report-only/display conditionals and the two #504-family supervisor elif branches that stay out of scope for this ticket. This is the "shape, not the eight sites" guard the ticket asked for: a ninth bare decision fails immediately, naming its line. Out of scope, not touched: #504 items 2/3/4 and #528 item 2 (same untested-main-flow family) — the seam here generalizes to make them testable too, but lifting them was left for their own tickets.	2026-09-12 16:46:53 +07:00
Dai Ha	f188947750	fleetd #552: warn instead of aborting when the post-restart mktemp fails ... By the time the fresh-log mktemp ran, the daemon had already been stopped, the jar swapped, and the new daemon started — an unguarded mktemp failure there aborted the whole script anyway, so a caller read the resulting non-zero exit as "the redeploy failed" and would restart an already-correctly-restarted daemon. Extract the mktemp into capture_fresh_log_region, guarded the same way unload_launchd_if_loaded/stop_systemd_if_loaded guard their own, but warn instead of die: there is nothing left to protect by refusing after a successful restart. The trap is now installed before the assignment it cleans up, using an FRESH_LOG="" sentinel readers can check. classify_amqp_connection_errors and report_shutdown_drain both gain a new state (REDEPLOY_AMQP_CHECK_SKIPPED / REDEPLOY_DRAIN_STATE=skipped) for an uncapturable log region, distinct from "captured a region with nothing in it" — and the result section gains a matching branch, so a skipped capture can never read as a clean bill of health.	2026-09-12 15:55:04 +07:00
Dai Ha	3da44eed63	fleetd #550: replace shasum with a portable hash256 helper, add a Linux CI job for the shell suite ... jar_id() in redeploy-fleetd.sh called shasum directly, which does not exist on GNU coreutils Linux (Debian/Ubuntu/etc.) — there it silently reported an existing jar as "absent" with exit 0, because the missing command made `cut` succeed on empty input and pipefail's failure was then swallowed by the `|| echo "absent"` fallback. The shell test suite hit the same tool at test-redeploy-fleetd.sh:298-299 and died at exit 127 with zero FAIL lines printed — the same shape as a clean pass on the one channel anyone would check. Adds one hash256() helper (prefer sha256sum, fall back to shasum -a 256, same idiom already used in probe-member-credentials.sh) and points jar_id and the test suite's own reference hash at it. jar_id now has three distinct answers instead of two: absent, a hash, or "unhashable" when neither hasher is on PATH — "absent" is never used for a file that exists. Adds a CI job (shell-tests) that runs scripts/test-redeploy-fleetd.sh on ubuntu-latest, gated on the step's own exit code rather than a FAIL-line count, since a suite that dies before running is exactly what a green run also looks like by that count. New tests: test_jar_id_reports_unhashable_when_no_hasher_on_path (stubbed PATH with neither hasher) and test_no_unguarded_macos_only_hasher_calls (a shape check across every script under scripts/, not named lines — #545 already showed this idiom spreading from two sites to six).	2026-09-12 15:06:15 +07:00
Dai Ha	a476a14f1c	fleetd #545: fix mktemp -t templates for GNU coreutils, split unclear-supervisor detail ... Every mktemp -t template in redeploy-fleetd.sh lacked an X placeholder. BSD mktemp (macOS) tolerates that and appends its own suffix; GNU mktemp (every Linux distribution) refuses it and exits non-zero. All six sites now use .XXXXXX. detect_supervisor's 'unclear' detail used to cover two different facts with one message that always named 'systemctl exited non-zero and reported an error on stderr' — even when systemctl was never run, because mktemp failed first. The SYSTEMD_LOADED_ERRORED/SYSTEMD_INSTALLED_ERRORED flags now carry a third value (2 = the probe's own mktemp setup failed) alongside the existing 1 (systemctl ran and answered badly on stderr), and detect_supervisor gives each its own detail text. kind stays 'unclear' in both cases; require_drivable_supervisor is unchanged. Tests added to scripts/test-redeploy-fleetd.sh: - test_mktemp_dash_t_templates_have_x_placeholders: source-text check, fails if any mktemp -t template lacks an X. - test_detect_supervisor_systemd_probe_setup_failure_is_unclear: proves the SET-UP-FAILED detail when mktemp itself fails (systemctl never runs). - test_detect_supervisor_systemd_probe_error_is_unclear: extended with assertions that the PROBE-ANSWERED-WITH-STDERR detail is present and the SET-UP-FAILED wording is absent, so swapping the two messages fails a test in both directions.	2026-09-12 14:22:57 +07:00
Dai Ha	e4eb3dbed4	fleetd #504 item 1: stop swallowing real launchctl/systemctl failures on the 'loaded but not running' path ... The two 'loaded but not currently running' branches in the stop step (launchd/systemd, reached when $OLD_PID is empty) ran 'launchctl unload'/'systemctl --user stop' with '2>/dev/null || true' and printed 'ok' unconditionally. That swallowed a real supervisor failure (e.g. launchd or the systemd user bus unreachable) exactly like a harmless already-stopped answer, and let the script proceed to start a new daemon believing nothing was loaded -- the two-daemons failure fleetd #492 exists to prevent. Adds unload_launchd_if_loaded/stop_systemd_if_loaded, applying systemd_loaded's own pattern (capture stderr separately; a non-zero exit WITH stderr is a real failure, a non-zero exit with empty stderr is a clean already-stopped answer) to the write side. The two call sites now use these functions instead of the bare '|| true'. Adds 5 tests: dies-on-real-failure and tolerates-clean-negative for each function, plus a source-text check that the main flow calls the new functions instead of the original bare '2>/dev/null || true'. All 5 verified by mutation (reintroducing the swallow, and separately over-correcting to die unconditionally) -- each goes red with its own message, restores byte-identical (full sha256), and passes a green control.	2026-09-12 13:42:27 +07:00
ltms	7d711942fe	Merge #534: detect a died shutdown drain the ERROR count is blind to (fleetd #512 part 2) ... Verified independently. The branch is based on 8335b12 while main is at bec87f9, so I merged locally first and tested the MERGED tree, not the branch — a clean auto-merge is not a working merge. Merged tree checks: - `bash -n` exit 0 on both scripts, under /bin/bash 3.2.57 and env bash 5.3.9. - Suite exit 0, 0 lines matching `^FAIL:`, 255 bytes of output. 60 test functions defined, 60 invoked, and no defined-but-never-invoked orphan (checked with a comm against the invocation list, not by comparing two counts — two equal counts can both be wrong). - My own comment fix from bec87f9 survived the merge and is still at :317. I ran two mutations the worker did not, per "mutate the half the worker did not": (A) The one that matters, because it is the defect this ticket exists to prevent: collapsed the `unknown` state into `complete`, so "cannot tell" reports as a pass. Result exit 1, one FAIL: `cannot-tell fixture must set REDEPLOY_DRAIN_STATE=unknown: expected unknown, got complete`. So the third state is genuinely load-bearing, not decoration. (B) Broke the positive check: changed `find_drain_complete_line`'s pattern from `drain complete: released=` to `drain finished: released=`, one site. Result exit 1, one FAIL: `find_drain_complete_line did not capture the present line`. Proof that (B) applied, against a pristine copy: the full grep line 1 -> 0, the mutant form 0 -> 1, and the bare phrase 2 -> 1 with the comment occurrence untouched. Both files restored byte-identical; `git diff --quiet` clean; green control re-run. A note on my own proof cell for (B), because it was wrong the first time. I wrote the counts with escaped double quotes inside an already double-quoted command substitution, so the shell split the pattern on spaces and grep treated the words as filenames. It printed "2 and 2" alongside `ugrep: No such file or directory` warnings — a symmetric, plausible-looking pair that meant nothing. The kill itself was never in doubt, since the suite named the exact function, but the cell that was supposed to prove the mutation applied proved nothing. Re-done with single quotes. This is the same trap already written down for this repo, hit by me, in a cell whose only purpose was to guard against exactly this. One thing I checked that no test covers: the main flow's `HAD_OLD_PID=0; [ -n "$OLD_PID" ] && HAD_OLD_PID=1` runs under `set -euo pipefail`, and on a cold start the test fails. Sourcing stops before the main flow, so no behavioural test reaches that line. If `set -e` fired there, every cold start would abort before the health checks. It does not: `set -e` exempts the left side of an `&&` list, confirmed by running it under both shells — `survived, HAD_OLD_PID=0` on 3.2.57 and on 5.3.9. Safe, but it is untested main-flow wiring, which is the same class as #528's item 1 and belongs on that list. On the `n/a` fourth state, which the worker flagged for a reviewer's judgment rather than quietly keeping: accepted, and it is in scope. The ticket asked for a third state because a sentinel conflating "no" with "cannot tell" hides two causes needing opposite handling. "The question does not apply" is a third such cause, not a variant of "cannot tell". Without it, the new warning would fire on every clean cold start, and a warning that cries wolf on the most common path trains the operator to skip it — which destroys the absence signal just as surely as putting the completion line in a `finally` would. The worker also proved the gate is actually consulted, using a cold-start fixture whose content deliberately looks like a died drain, so the test would fail if the gate were skipped. That is the right way to test a gate. Both remaining outcomes are correctly excluded from the "no ERROR lines since restart" summary: only `complete` and `n/a` let it print.	2026-09-12 08:04:15 +02:00
Dai Ha	bec87f987c	scripts: name the mechanism in detect_supervisor's constraint 2, not a line number ... Constraint 2 read "This script runs under `set -euo pipefail` (line 50), so an unset variable is a loud failure." Two problems, both small and both the same family as fleetd #494 — a comment that states the wrong reason. The line number was stale: the `set` line is at 54, not 50. It was the only line-number citation in the file, and a citation like that goes stale on the next insert above it, silently, with nothing to catch it. The mechanism was also misattributed. What makes an unset variable a loud failure is `set -u`. Naming the whole `-euo pipefail` string invites the reader to credit pipefail for it, which is the mistake fleet01 flagged on a different cell this week: pipefail is insurance against a future pipeline stage, not what catches the current shape. Now names `set -u` and says where it is without a number, and records why the number is gone so nobody adds one back. Comment only. bash -n exit 0 under /bin/bash 3.2.57 and env bash 5.3.9; scripts/test-redeploy-fleetd.sh exit 0 with 0 lines matching ^FAIL:.	2026-09-12 12:58:37 +07:00
Dai Ha	190436c9cf	fleetd #512 part 2: detect a died shutdown drain the ERROR count is blind to ... The previous daemon's dead shutdown drain (an uncaught exception in a shutdown thread) never passes through the logger, so it never carries an ERROR/SEVERE token, so redeploy-fleetd.sh's existing ERROR-count classifier is structurally blind to it and prints a confident "no ERROR lines since restart" while the drain actually died. Add scan_uncaught_exceptions (greps the shutdown window for the failure's real shape: `Exception in thread`, `NoClassDefFoundError`) and find_drain_complete_line (checks for #522's SessionManager.drainAll completion line). Compose both in report_shutdown_drain, a single decision+action function the main flow calls unconditionally (same shape as swap_if_built/refuse_drain_gate from #521/#528), which resolves to one of four outcomes: complete, died, unknown ("cannot tell" — the line is absent for either of two reasons that need opposite handling: the previous daemon predates #522, or its drain failed without throwing), or n/a (no previous daemon was actually stopped this run). Never fails the redeploy; warns loudly instead. Gate the "no ERROR lines since restart" summary line on the new outcome so it never reads as reassurance when the drain died or the outcome is "cannot tell" (item 4 of the ticket). Tests: 11 new test functions (60 defined/invoked, was 49), covering both pure classifiers, all four report_shutdown_drain outcomes, a source-grep proof of the main-flow call site (sourcing stops before the main flow runs), an ordering check, and the item-4 gating. Full suite green (exit 0, 0 anchored FAIL lines). Five mutations applied and killed by hand during review, each restored to a byte-identical file afterward.	2026-09-12 12:55:54 +07:00
Dai Ha	7c34e8f4f9	fleetd #528: pin drain_gate_refusal's call site, not just the predicate ... drain_gate_refusal composes the correct abort message and is well tested, but the main flow built its own `die "$(drain_gate_refusal ...)"` call — nothing proved that call site was ever consulted. Mutating it to a flat `die "aborted -- nothing changed"` left the whole suite green, silently reinstating the exact defect #517 was filed to fix. Same shape as #521/#526's should_swap/swap_if_built: the decision and the die() now live together in refuse_drain_gate, which the main flow calls unconditionally. drain_gate_refusal stays separate and separately tested for the message logic; four new behavioural tests stub die() to prove refuse_drain_gate calls it correctly for all four cases, and a fifth source-text test pins the main flow's call site itself (the only thing that can catch deleting the call, since sourcing stops before the main flow runs).	2026-09-12 12:38:16 +07:00
Dai Ha	08771e270b	fleetd #521 gate fix: pin the swap at the call site, not just the predicate ... The extraction in the previous commit did what #521 asked for — a should_swap() predicate with a test for each value — and I measured that it does not close the defect. With the main flow reading `if should_swap "$DO_BUILD"; then`, changing that line to `if false; then` left the whole suite at exit 0 with zero FAIL lines. The swap still never ran, and a redeploy would still report success while starting on no jar. That is my ticket's fault, not the implementer's: "extract the decision so the suite can call it" pins the decision and never the wiring. Extraction moved the untested decision up one level instead of removing it. Fix: the decision and the action now live together in swap_if_built(), which the main flow calls unconditionally — there is no guard left in the main flow to get wrong. should_swap() stays, because it is the decision and is worth naming and testing on its own. Two new tests call swap_if_built() with a recording stub in place of the real mv, so they fail if the guard is removed, inverted, or stops being consulted. Also fixed, found while verifying this: * test_swap_ordered_after_wait_and_before_start had to follow the call site to `swap_if_built "$DO_BUILD"`. Left on the old needle it reported "swap_staged_jar (line 215) is not after wait_for_daemon_exit (line 730)" — true of a function definition, and nothing about step order. * That test's three `[ -n ... ] || fail "could not find ... call site"` guards were dead code. Under `set -euo pipefail` an absent needle fails the assignment and `set -e` kills the suite before the guard runs. Measured: deleting the swap call gave exit 1 with ZERO bytes of output, no FAIL line, nothing naming what was missing. Each grep now ends in `|| true` so the assignment succeeds empty and the guard can speak. Verified by me on this revision: * suite exit 0, 0 `^FAIL:` lines, 44 tests defined and 44 invoked * bash -n rc=0 on both scripts under /bin/bash 3.2.57 and bash 5.3.9 * four mutations, each killed with its own named FAIL line, each restored byte-identical, green control after the battery: - guard removed inside swap_if_built -> "must not swap, but it did" - guard inverted -> "must perform the swap, and did not" - should_swap's comparison changed -> "must return true" - main-flow call deleted -> "could not find the swap call site in redeploy-fleetd.sh" (this one printed 0 bytes before the dead-guard fix, which is the before/after proof for it) Not fixed here, filed separately: drain_gate_refusal has the same shape. Replacing `die "$(drain_gate_refusal ...)"` with `die "aborted — nothing changed"` leaves the suite at exit 0 with output byte-identical to a clean run, which reinstates the exact wrong message #517 was filed to fix.	2026-09-12 11:58:34 +07:00
Dai Ha	c89a375e5d	fleetd #521: extract should_swap so the swap guard can't be silently disabled ... Mutating the swap step's guard (if [ "$DO_BUILD" = 1 ] -> if false) left the whole test suite green: test_swap_ordered_after_wait_and_before_start only checks source positions, which an in-place if-condition edit never moves. Extracts the decision into should_swap(do_build), following the same shape as #510's wait_for_daemon_exit and #517's drain_gate_refusal, with a direct test for each value.	2026-09-12 11:42:46 +07:00
Dai Ha	3833d8e52b	fleetd #517: pin the drain-gate abort branch and jar_id's absent case ... Two mutation-testing survivors in scripts/redeploy-fleetd.sh: a source-text test pins what a message SAYS but never whether the branch that prints it is REACHED. - Extract the drain-gate abort decision into drain_gate_refusal(do_build, staged_path), a pure function the suite can call directly for all four build/staged combinations. The existing source-text grep test is kept alongside it (it catches a re-wording; the new tests catch a dead branch). - Extend jar_id's test to cover the missing-file path (both the no-argument default and an explicit path), which the #511 test never exercised.	2026-09-12 11:23:37 +07:00
Dai Ha	6e23bf8309	fleetd #511: fix wrong --no-build wording in drain-gate abort, pin jar_id() default ... The drain-gate abort message told the operator a rerun "with or without --no-build" would finish the restart. That is wrong: by the time this message can fire, stage_built_jar has already moved the jar off $JAR, so --no-build hits require_no_build_jar's own refusal. Reworded to say the rerun must NOT use --no-build, and why: the built jar is no longer at the live path that --no-build requires. Also added a test pinning jar_id()'s no-argument default (reports $JAR, the live path) and its explicit-argument behavior (reports that path instead), per fleetd #511 item 2. Not adding a test for the JAR_STAGED rm -f at line 578 (fleetd #511 documents it as an equivalent mutant — mvn clean install deletes target/ on the next line regardless).	2026-09-12 10:56:32 +07:00
Dai Ha	979adf82eb	fleetd #493: never build into the path a running daemon holds ... redeploy-fleetd.sh's build step wrote straight into fleetd/target/fleetd.jar via `mvn clean install` while the OLD daemon was still running from that exact path. A JVM loads classes lazily, so a class the daemon had not touched yet could be read from a jar already replaced or removed -- the failure landed on the shutdown drain (NoClassDefFoundError, exit 143, looks clean). Stage the freshly built jar at target/fleetd-new.jar (stage_built_jar), confirm the OLD pid has actually exited (wait_for_daemon_exit, extracted from the existing wait loop), and only then swap it into the live path (swap_staged_jar) -- strictly after the wait, strictly before start. A failed swap dies without starting. --no-build and --check keep their existing, truthful behavior (require_no_build_jar; jar_id still reads the live path by default). A leftover staged jar from an interrupted run is wiped before the next build. The build still runs before anything is stopped, so a failed build still never takes the fleet down. Also fixed: the drain-gate abort message ("aborted -- nothing changed") now names the staged jar when one exists, since staging already moves the freshly built jar off the live path before that prompt runs. Adds unit tests for stage_built_jar, swap_staged_jar, require_no_build_jar, wait_for_daemon_exit, and a source-order test proving swap sits after the wait and before start (sourcing stops before the main flow ever runs, so the ordering itself can only be checked by reading the script's own call sites).	2026-09-12 10:35:19 +07:00
Dai Ha	599419f9e6	fleetd #492 follow-up: carry the unclear detail across detect_supervisor's subshell boundary ... b17f37a set SUPERVISOR_UNCLEAR_DETAIL as a global inside detect_supervisor, but the real call site invokes it as $(detect_supervisor) — a subshell — so that global died with the subshell and the die() message's ${VAR:-fallback} silently masked the loss with generic text. - detect_supervisor now packs kind and detail onto its one stdout line (joined by the ASCII unit separator byte, $SUPERVISOR_DETAIL_SEP), the only channel that survives $( ). The real call site unpacks both with in-shell parameter expansion — no extra subshell. - Dropped the ${SUPERVISOR_UNCLEAR_DETAIL:-...} fallback at the die() message: under set -u, a missing detail now fails loudly instead of silently defaulting (same defect class as #497). - Added a constraints comment block above detect_supervisor for future callers: stdout-only, no ${VAR:-default} papering over a lost value, and every case on the return value needs an explicit *) arm. - Added *) arms to the three `case "$SUPERVISOR_KIND"` switches (report/stop/start): report warns and continues (display-only), stop/start die naming the value (they act on it). - Rewrote the "unclear" test to go through the real call-site shape ($(detect_supervisor) then the same split), not a hand-constructed value, and tightened its final assertion to check for the actual detail text rather than $SYSTEMD_UNIT alone (the die() boilerplate names the unit either way, so that check could pass on a lost value).	2026-09-12 09:58:45 +07:00
Dai Ha	b17f37a683	fleetd #492 follow-up: detect_supervisor must never read "could not tell" as "none" ... Two situations were silently landing in the "none" answer, which require_drivable_supervisor accepts and the script then falls back to a raw kill + nohup — exactly the wrong move when a supervisor actually IS present: - installed-but-not-loaded, on either supervisor. `systemctl --user is-active` answers "no" for activating/deactivating/failed and while an auto-restart is pending too, and every one of those is a host that IS under systemd (or launchd) and about to act again. `*_installed` already knew this; it was only ever consulted for a warning line, never by the decision itself. - a systemd probe that could not answer at all (e.g. systemctl cannot reach the user bus over a non-lingering ssh session) looked identical to a clean negative, because both probes redirected stderr straight to /dev/null. detect_supervisor now returns a fifth answer, "unclear", for both cases. systemd_loaded/systemd_installed capture systemctl's exit status and stderr separately and set their own *_ERRORED flag only on a real tool failure (non-zero exit WITH stderr), never on a clean negative. "none" now means only: neither supervisor installed, neither loaded, neither probe errored. require_drivable_supervisor die()s on "unclear" exactly like it already does on "ambiguous", naming the specific supervisor and reason via the new SUPERVISOR_UNCLEAR_DETAIL global. Tests: 4 new cases (systemd/launchd installed-but-not-loaded, a real systemd_loaded run through a systemctl stub that errors on stderr, and the die() refusal for "unclear" naming the unit). All 3 new guards were verified by mutation: each was removed from the real script, the suite caught it (a new FAIL line naming the exact broken assertion), then the file was restored byte-identically and the suite went green again.	2026-09-12 09:23:21 +07:00
Dai Ha	dcd505286f	fleetd #492: teach redeploy-fleetd.sh systemd --user as a third supervisor ... launchd, systemd --user, and unsupervised are three different answers, not two. Refuse (die) rather than fall through to kill+nohup when a supervisor is detected that this script cannot drive (e.g. both signals fire at once), and add a post-restart check that fails the run if more than one fleetd process is alive. detect_supervisor()/require_drivable_supervisor()/ count_daemon_pids()/assert_single_daemon() are pure, overridable functions so scripts/test-redeploy-fleetd.sh can exercise them without a real launchd or systemd.	2026-09-12 09:00:59 +07:00
Dai Ha	09159f2857	Classify named AMQP recovery errors	2026-09-05 06:05:52 +07:00
Dai Ha	0241e0d3a8	Keep unattributed AMQP errors loud	2026-09-05 05:36:53 +07:00
Dai Ha	e4973eb8a4	Classify recovered AMQP redeploy errors	2026-09-05 05:29:53 +07:00
Dai Ha	2e138a199b	CB-634: one shared "fleet" workspace + rename bridged -> fleetd cutover ... Two changes ship together here. 1. One shared herdr workspace. The lead and every worker now live in one workspace called "fleet", so the operator sees one "session" with many windows, not two. Before, the lead sat in a "leads" workspace and workers in "bridged-workers", which read as two sessions. The lead is still told apart from workers by its exact tab label ("lead: <name>"), so putting them in one space is safe. LeadTabScanner keeps the exclude-by-label mechanism for split layouts; Fleetd now passes an empty exclude set. 2. Rename the daemon from "bridged" to "fleetd" (the binary, config, scripts, launchd/systemd units, module dir, and MCP mount). - Module dir bridged/ -> fleetd/; jar finalName -> fleetd.jar. - Log line, comments, docs, and CLAUDE.md updated to say fleetd. - Scripts renamed: redeploy-bridged.sh -> redeploy-fleetd.sh, bridged-launchd-wrapper.sh -> fleetd-launchd-wrapper.sh. - Deploy units renamed: dev.ltms.bridged.plist -> dev.ltms.fleetd.plist, bridged.service -> fleetd.service; launchd Label -> dev.ltms.fleetd. - Config default bridged.yaml -> fleetd.yaml; the legacy bridged.yaml is still read as a fallback, and still gitignored. - MCP: drop the deprecated bridge_* tool twins; only fleet_* remain. The server name is "fleet". The mount name in the local .mcp.json becomes "fleet" (gitignored, not in this commit). - Env var defaults BRIDGED_API_TOKEN -> FLEETD_API_TOKEN, fixture BRIDGED_WORKER_TOKEN -> FLEETD_WORKER_TOKEN. Kept on purpose: the BRIDGED_MEMBER marker. Renaming it is a coupled change to the credential-scrub security control (an operator secrets.sh may guard on it), so it stays until that migration is done on its own. Metrics were already fleet_* (CB-632); MetricNamesTest still guards that no name says bridged_. The canonical CLAUDE.md block and the wiki template stay byte-identical (wiki working tree edited, committed to the wiki repo separately). 949 tests pass (mvn clean install). 4 fewer than before = the 4 removed bridge_* alias tests.	2026-08-25 04:01:08 +02:00