Resolves the one conflict in Fleetd.java. main's side is the inline boot block
that Unit A had already moved into FleetdAssembly.assembleAndStart, so the
resolution keeps Unit A's single assembly call.
That resolution is not purely mechanical. #622 (fleetd #621) landed on main
AFTER Unit A forked, and it added
boolean requireOperatorConfirm = cfg.leadRollover() == null
|| cfg.leadRollover().requireOperatorConfirm();
plus a 14th argument to the LeadHeartbeatLoop constructor, inside the very block
Unit A moved. Taking Unit A's side alone would have dropped both and silently
reverted the operator's #621 fix: the 13-argument overload still exists and
delegates with `true`, so the daemon would go back to telling every lead to ask
the operator before a context roll. Both are carried into FleetdAssembly here.
Measured: with the carried line removed, the full suite is
`Tests run: 1883, Failures: 0, Errors: 0` — nothing pins it. That is the fleetd
#612 defect shape applied to #621's own wiring, and it is filed separately
rather than fixed here, because this commit is a merge resolution and must not
also introduce new tests.
Full suite on this resolved tree: Tests run: 1883, Failures: 0, Errors: 0.
Ticket comment 17553 on fleetd #612 found that the test's single shared
FakeHerdr made router.leadAgents() and router.memberAgents() collapse to
the identical client (FleetdAssembly.java:140-142's no-distinct-socket
fallback), so a mutation swapping leadAgents() for memberAgents() at the
FleetdAssembly.java:408 call site was invisible to this test even though
the two are genuinely different daemons in production.
Configure two distinct herdr sockets and two distinct FakeHerdr instances
(the same TwoHerdrResourcePorts shape B2's FleetdAssemblyConnectionIdentityTest
uses) and assert the roll's /clear + bootstrap sends land on the LEAD fake
and never on the MEMBER one.
Proven red against the router.memberAgents() mutation, reverted, touched,
and re-run green — both outputs recorded in the PR.
and lead-rollover source-text guards
Replaces three FleetdAssembly.java source-text guards (each scraped
Fleetd.java for a call site that fleetd #612 Unit A moved into
FleetdAssembly.java) with tests that drive the real assembled objects
through FleetdAssembly.assembleAndStart(...) -> FleetdRuntime.mcp(),
per the step-2 B-unit split (issue #612 comment 17513).
- Deleted FleetdLeadSeatWiringTest (fleetd #176): pinned that
FleetMcp's LeadSeatSource construction still wires
Fleetd.leadSeatLookup(...) by scraping the constructor call's text.
Replaced by FleetdLeadSeatAssemblyTest, which seeds one FakeHerdr
tab labelled to match a configured fleet.leaders.opus.tab and
asserts the REAL assembled LeadSeatSource (via
runtime.mcp().leadSeatSource()) reports the live lead's seat against
its own subscription profile -- 1, not the 0 LeadSeatSource.none()
(the inert stand-in) could ever report.
- Deleted FleetdBackendQuarantineWiringTest (fleetd #466): pinned that
the escalating BackendQuarantine.withEscalation(...) text was
present and the flat two-argument constructor's text was absent.
Replaced by FleetdBackendQuarantineAssemblyTest, which quarantines
the same credential twice through the REAL assembled
BackendQuarantine (via runtime.mcp().quarantineSource().quarantine())
at controlled fake-clock offsets and asserts the second cooldown
doubles (200s vs 100s) -- the one behavioural difference escalation
and the flat constructor actually produce.
- Deleted FleetdLeadRolloverWiringTest (fleetd #480), all three
methods: unrelatedAnchorStillPresent was a scaffold anchor with no
independent claim, needing no replacement.
mainStillCallsTheLeadRolloverFactory pinned the leadRollover
assignment's call-site text. factoryGatesOnConfigPresence pinned
that an absent leadRollover: config yields no LeadRollover.
Replaced by FleetdLeadRolloverAssemblyTest's two tests:
assembledLeadRolloverRunsTheRealClearAndBootstrapSequence drives the
REAL assembled LeadRollover (via runtime.mcp().leadRollover())
through open()/confirm() end to end and asserts /clear then
bootstrapText were actually sent through the real herdr router,
reaching ROLLED. absentLeadRolloverConfigMeansNoRolloverIsBuilt
calls Fleetd.leadRollover(...) directly with no leadRollover: block
and asserts null -- this claim was found uncovered elsewhere
(LeadRolloverTest's only related assertion is vacuous, assertNull
(null), and never calls the real factory).
Each of the three FleetdAssembly.java call sites (quarantine
line 179-180, leadRollover line 408, lead seats line 479) was mutated
to its named inert variant, run against ONLY its new test (RED),
reverted, touch'd (Maven mtime trap) and re-run (GREEN) -- six proven
runs, pasted in the PR body.
FleetMcp.java: adds three accessors (quarantineSource(),
leadSeatSource(), leadRollover()) alongside the existing
registeredTools() -- but public, not package-private, and this is a
deliberate deviation from that precedent, not an oversight: these new
assembly tests cannot live in package dev.ltms.fleet.mcp the way
registeredTools()'s callers do, because they also build the
ResourcePorts FleetdAssembly.assembleAndStart(...) needs, and
ResourcePorts' methods return Fleetd-nested types visible only from
package dev.ltms.fleet. Package-private would compile but be
unreachable from there.
Full mvn -o test in fleetd/: Tests run: 1879, Failures: 6 (down from
the branch baseline's 1880/9 by exactly the 3 guards this unit
deletes) -- the remaining 6 are FleetdCompletionResolverWiringTest (4)
and FleetdConnectionIdentityConstructionTest /
FleetdFleetAppConstructionTest (1 each), all out of this unit's scope
(B1/B2).
Per ticket comment 17525: my second independent mutation on
FleetdAssembly.java:518 survived — new FleetApp(memberHerdr, memberHerdr, ...)
(dropping the LEAD client instead of the member one) left both existing
FleetdAssemblyFleetAppTest cases green. That is the symmetric form of the
CB-185 defect (/healthz green while the LEAD daemon is down), and the guard
this PR deletes would have caught it: its positive assertion required the
exact pair "new FleetApp(herdr, memberHerdr, workers,", which does not
survive either daemon being dropped.
Adds healthzGoesRedWhenTheLeadDaemonIsDownEvenThoughTheMemberIsUp, symmetric
to the existing member-down case.
Proven red today: reverting FleetdAssembly.java:518 to
"new FleetApp(memberHerdr, memberHerdr, workers, ..." and running only
FleetdAssemblyFleetAppTest gives Tests run: 3, Failures: 1 — the new case
fails ("expected: <503> but was: <200>", body has no "member" key); the other
two cases stay green. Reverted, git diff --stat empty, file touched, re-ran:
Tests run: 3, Failures: 0.
Full mvn -o test: Tests run: 1884, Failures: 7 (same 7 B1/B3-scope failures
as before this fixup; 1884 = 1883 + 1 new case).
FleetdCompletionResolverWiringTest read Fleetd.java's literal source text and
asserted the CompletionResolver construction call still named the right
arguments — proof of spelling, not behaviour. Unit A (FleetdAssembly) moved
that call site out of Fleetd.main, breaking all four of its tests on a
harmless relocation.
FleetdCompletionResolverAssemblyTest replaces it, driving the real
FleetdAssembly.assembleAndStart(...) and reading FleetdRuntime.completion() —
the exact CompletionResolver instance production uses — through its public
onDelivered/resolveBeforePostAction API, with a controllable ResourcePorts
nanoClock in place of real sleeps.
Deleted test -> what it pinned -> replacement:
- worktreeBranchLookupIsStillPassedAtTheCallSite (8th constructor arg) ->
assembledResolverReportsTheMembersWorktreeAndBranchInAFallbackReport: a
real git-worktree-provisioned MemberSession's branch must appear in a
noReportMessage fallback (fleetd #241).
- backendErrorArgumentsAreStillNamedAtTheCallSite,
backendErrorPatternsComesFromTheFactory, backendErrorSinkComesFromTheFactory
(5th/6th args) -> assembledResolverClassifiesAndCoolsOffOnAConfiguredBackendErrorPattern:
a configured errorPattern the built-in fallback never matches must classify
as FAILED (not COMPLETION), transition the session to BACKEND_ERROR, and
cool the credential off after two distinct targets within the window
(fleetd #201 Unit 5).
Both behaviours were proven red today by mutating FleetdAssembly.java's real
call site to its inert variant (_ -> null; BackendErrorPatternLookup.legacy();
BackendErrorSink.none()), confirming the new test failed with the expected
message, then reverting (touching the file to defeat Maven's stale-mtime
skip) and confirming green again. FleetdAssembly.java itself is unchanged in
this commit.
Full suite: Tests run: 1878, Failures: 5 (down from the baseline 9 — the
remaining 5 are the other in-flight workers' own guard files:
FleetdBackendQuarantineWiringTest, FleetdConnectionIdentityConstructionTest,
FleetdFleetAppConstructionTest, FleetdLeadRolloverWiringTest,
FleetdLeadSeatWiringTest), Errors: 0, Skipped: 0.
Deletes the two source-text guards fleetd #612 Unit A broke by moving their
scraped call sites from Fleetd.java into FleetdAssembly.java, replacing each
with a behavioural test that drives the real assembled graph instead.
- FleetdConnectionIdentityConstructionTest pinned that Fleetd.java contained
"new PaneLocator(herdr, memberHerdr)". Replaced by
FleetdAssemblyConnectionIdentityTest, which drives the real PaneLocator a
real FleetdAssembly.assembleAndStart(...) built (reached via
runtime.mcp().identity().panes(), never a copy) with two distinct FakeHerdr
daemons, and proves it finds a pane that exists on only one of them —
first the lead-only case (the CB-185 bug: a lead's own connection going
unresolvable), then the member-only case, plus a no-match control.
- FleetdFleetAppConstructionTest pinned that Fleetd.java contained
"new FleetApp(herdr, memberHerdr, workers,". Replaced by
FleetdAssemblyFleetAppTest, which binds the real Javalin app
FleetdAssembly built (runtime.app()) to a real ephemeral port and proves
GET /healthz goes 503 when only the member daemon is down — the consequence
named in the deleted test's javadoc (a down member daemon invisible behind
a healthy lead). The GET /sessions merge half of that javadoc could not be
driven the same way: it requires Authz.Action.READ, which needs a real
positive pid from the real assembly's hardcoded LsofPeerPidLookup, and an
in-process test's HTTP client and server share one JVM pid so that pid is
always -1 (fleetd #317's fail-closed rule then refuses the request before
the route, and its merge, is ever reached) — documented in the new test's
javadoc; FleetAppTwoDaemonTest remains the full behavioural proof that
FleetApp itself merges /sessions correctly given two clients.
Both replacements were proven red today: reverting FleetdAssembly.java:443-444
to "new PaneLocator(memberHerdr)" turns the identity test red (expected
"term_a", got null); reverting :518 to "new FleetApp(herdr, herdr, ..." turns
the app test red (expected 503, got 200 with no "member" key). Both mutations
reverted, tree confirmed clean, and the mutated file touched afterward so
Maven does not skip recompiling a stale .class.
Adds two small production accessors needed to reach the real objects rather
than a copy, since FleetdRuntime may not gain a field (three workers touch
that file): ConnectionIdentity#panes() exposes the PaneLocator it resolves
against, and FleetMcp#identity() exposes the ConnectionIdentity it was built
with (now also stored as a field).
Baseline on 608e449: 1880 tests, 9 failures (the known source-text set).
After this change: 1883 tests, 7 failures — the remaining #248/#176/#466/#480
guards, which are B1's and B3's scope, not this one's.
Gap 1: generalise Fleetd.LeadMailboxOpener (and FleetdRuntime's field) from the concrete
LeadMailbox to a new closeable LeadChannelHandle (LeadChannel + AutoCloseable), so a test can
fake the configured-coordinator path without a real broker. FleetdAssemblyCoordinatorLifecycleTest
drives FleetdAssembly.assembleAndStart with a coordinator: block and a fake channel, proving the
assembly builds it and shutdown closes it.
Gap 2: move reportRoleFallbackGaps(cfg) and assertChartersNameOnlyRegisteredTools(cfg) out of
Fleetd.main and into FleetdAssembly.assembleAndStart, immediately after cfg.validateAll(), so
both run inside the tested assembly boundary before any I/O. FleetdAssemblyRoleFallbackBoundaryTest
pins the moved call site behaviourally (captured log output), not by reading source text.
Both new tests were verified red under a targeted mutation (a non-closing/null coordinator for
gap 1; deleting the moved call for gap 2) and restored.
contextNotice hardcoded 'ask the operator' and 'Only the operator can
approve the roll', so setting leadRollover.requireOperatorConfirm to
false stopped the daemon refusing the roll but never stopped the lead
being told to ask. Thread the effective config value into
contextNotice: when true the text stays byte-identical, when false it
tells the lead to confirm on its own judgement against the three
handover-file checks instead.
LeadRollover.confirm's own enforcement is untouched — this is the
message only.
Two bullets in the handover skill were telling every outgoing lead something
that is no longer true.
1. The skill said the bootstrap prompt "has never yet landed, and the fix is
unproven (fleetd #489)", and told the lead to warn the operator it may fail.
Measured today from fleetd/fleetd.out:
grep -c "lead-rollover: rolled" -> 4
grep -c "lead-rollover:" -> 16 (positive control)
grep -c "Unknown command" -> 0
Three of the four rolls ran on 2026-09-22 (10:01:43, 10:38:28, 11:15:47).
Each cleared the old lead and bootstrapped a fresh one against the handover
file. The old "Unknown command: /clearFresh" failure does not appear at all.
The paragraph now carries the measured result and the three re-measure
commands, including the control line, because a broken grep pattern returns
a clean 0 that reads like good news.
It also records that the "/clear was never observed as WORKING ... releasing
rather than wedging the roll" WARN accompanies every successful roll. That
is the safe branch, not a failure, and it was being misread as one.
2. The skill said requireOperatorConfirm "defaults to true and this is the only
thing standing between a judgement call and a wiped session", which reads as
if asking is always required. The default is still true
(FleetConfig.java:1426), but this host set it to false on 2026-09-22 on the
operator's explicit grant. The bullet now says to read the live value rather
than assume, and notes the key is deferred, not hot.
It also warns that until fleetd #621 merges, LeadHeartbeatLoop.contextNotice()
still hardcodes "ask the operator" and takes no config, so the nudge text and
the config disagree. Trust the config. That warning names the ticket that
removes it.
Documentation only. No code or test changes.
Fleetd.main kept config loading, startup reports and validation. Everything from
the herdr socket connect onward moved verbatim, same order, into
FleetdAssembly.assembleAndStart(AssemblyInputs, ResourcePorts), which returns a
FleetdRuntime owning the real objects (package-private accessors, never a copy)
and their single ordered close(). ResourcePorts/SystemResourcePorts abstract every
boot-time side effect (env, herdr connect, broker openers, clocks, schedulers,
shutdown-hook registration, HTTP start) with no inert production variant, per the
architect proposal on the ticket.
sleepHerdrPoll widened from private to package-private so FleetdAssembly can pass
a method reference to it; no other signature changed.
FleetdAssemblyLifecycleTest drives the real assembly with FakeHerdr, a temp
FleetConfig and a fake ResourcePorts recording a start/close ledger, asserting it
against the order recorded from the pre-move main() and shutdown hook, and proving
every resource the ledger can observe (herdr client, three schedulers, the AMQP
reply inbox) closes via FleetdRuntime.close(). FakeHerdr gained a closed flag for
this.
The method-level javadoc on FleetConfig.warnConflictingAutoCompactWindows
(above the log.warn call) still claimed the autoCompactWindow vs
CLAUDE_CODE_AUTO_COMPACT_WINDOW precedence was 'intentionally not
asserted' and cited fleetd.yaml's now-corrected comment as evidence the
question was open. Replace it with the measured answer from #618: the
env var wins, so autoCompactWindow is inert on a profile that sets both.
Kept the WARN-not-throw rationale paragraph above it untouched (#601)
and kept the ClaudeCodeArguments cross-reference, which now points to an
agreeing claim instead of a contradicting one. No behaviour change.
ClaudeCodeArguments.withAutoCompactWindow's javadoc and FleetConfig's
warnConflictingAutoCompactWindows WARN text both used to say the
precedence between --autocompact and CLAUDE_CODE_AUTO_COMPACT_WINDOW was
not verified. fleetd #618 measured it: the env var wins, so the flag has
no effect when both are set. Update both texts to say so, name #618, and
warn that deleting the env var to resolve the conflict LOWERS the live
window rather than fixing anything. No behaviour change; the WARN still
fires on the same condition and stays a WARN (per #601).
LeadRollover.runRollover made two unwrapped agents.send calls. HerdrException
is unchecked, and the production continuationRunner is a bare virtual thread
with no uncaught-exception handler, so a throw from either send call killed
the continuation silently — confirm() had already written IN_PROGRESS into
outcomes before scheduling it, and nothing ever overwrote that entry with a
terminal state.
Wrap the whole continuation body in one try/catch(RuntimeException), matching
the local convention already used around agents.status in
waitUntilAtTurnBoundary. On a throw, write a new terminal RollState.FAILED
entry naming the exception, in the same diagnostic style as
TURN_NEVER_SETTLED and CLEAR_NEVER_SETTLED.
Two new tests make send() throw on the /clear call and on the bootstrap-text
call respectively, each asserting status(token) reports FAILED, not
IN_PROGRESS. Reverting only the production catch (keeping the tests) turns
both red; restoring it turns them green again.
rejectConflictingAutoCompactWindows threw and stopped fleetd from starting when a
Claude Code profile's autoCompactWindow flag and CLAUDE_CODE_AUTO_COMPACT_WINDOW
env var disagreed. Under launchd that is a restart loop, and the config that
would fix it (fleetd.yaml) is gitignored, so the cause is invisible on the host
where it bites (measured live: 4 profiles on this host trip it, including the
lead's own profile and the one every worker spawns on).
Renamed to warnConflictingAutoCompactWindows: it now logs a WARN naming each
offending profile with BOTH values (autoCompactWindow=... and
env.CLAUDE_CODE_AUTO_COMPACT_WINDOW=...) instead of throwing, so the daemon
starts and an operator can fix the config without reading the source. Equal
values still load silently.
Also reworded ClaudeCodeArguments' javadoc, which stated as fact that the env
var takes precedence over the flag. That was never measured, and this host's
own fleetd.yaml comment asserts the opposite — the javadoc no longer picks a
side.
- reportRoleFallbackGaps(cfg), called right after cfg.validateAll() in Fleetd.main, logs every
MemberRole with no fleet.<role>s: pool (naming the profile count and the resolved
defaultProfileFor(role) first choice) and, separately, every role with no
fleet.charters.<role>: entry. Log only — the deliberate 'unconstrained' fallback in
FleetConfig#candidateProfiles / CompositePeerLauncher#poolFor is unchanged, and a config with
profiles: and no fleet: block still starts and still spawns.
- LeadHeartbeatLoop#start()'s boot line now also names contextHighNudge (fleetd #609), alongside
the three settings it already logged.
- RoleFallbackGapReportTest (new) and two new LeadHeartbeatLoopTest cases pin both lines' content
via a ListAppender, raising the dev.ltms.fleet logger past logback-test.xml's WARN override for
the INFO-level lines.
CI has been red on main itself since #602/#606, on this one test, so the build has been giving no second opinion on any PR. Cause: the CI job runs in a container as root. setReadable(false) really does clear the read bit, so the test's own setup guard passes, but root opens the file anyway and the gauge correctly returns OK. The test was asserting on a condition the environment never created.
The fix adds an assumeFalse(Files.isReadable(file), ...) after the chmod and before the gauge is built, inside the existing try, so the finally still restores the bit on a skip.
Verified by me on a scratch worktree merging this onto 955b9ea:
- 1864 tests, 0 failures, 0 errors, 0 skipped, 149 surefire reports, mvn exit 0. The suite-wide skipped=0 is the point: the fix did not quietly turn the test into a permanent skip.
- LeadContextGaugeTest on this non-root Mac: 9 tests, 0 skipped, and unreadableFileIsUnknown present in the report. The assumption does not fire here, so developers keep the coverage.
- Mutation: made the IOException path return OK instead of UNKNOWN. unreadableFileIsUnknown failed with "expected: <UNKNOWN> but was: <OK>". The test still has teeth. Production file reverted, git diff clean before merge.
Known trade, recorded rather than hidden: under root this case is now covered by nothing at all. A skip is honest about that, which an assertion on an unreachable state was not. The durable fix is to run the CI build as a non-root user; that is a CI configuration change and out of scope here.
The test set the file's read bit off via setReadable(false), but on the
Gitea CI runner (root inside the container) the OS ignores that bit and
opens the file anyway, so the test asserted on a condition it never
actually created (LeadContextGaugeTest.java:142 UNKNOWN vs OK, CI run
1887 job 3104, commit fa62e99 on main).
Add Files.isReadable(file) after setReadable(false) and before the gauge
runs, and assumeFalse on it: a skip means "could not set up the case",
never "the behaviour is fine". Restores the read bit either way so
@TempDir cleanup still works.
Closes fleetd #609. Completes the second half of the context work: #602/#606 could detect a full lead context, and nothing acted on it. LeadHeartbeatLoop now offers a handover when the lead's own gauge reads HIGH.
Never rolls a pane by itself. The nudge is text only; the lead still has to call fleet_handover, and that still needs operatorConfirmed.
Verified by me on a scratch worktree merging 89cb8ff onto fa62e99:
- 1864 tests, 0 failures, 0 errors, 149 surefire reports, mvn exit 0.
- Four mutations, all killed: the two the implementer ran (latch back in applyDecision: 4 failures; call site drops the latch argument: 2 failures), one of my own at the line the logic moved TO (latch set regardless of send outcome: 1 failure), and a control on an untouched line (quiet-cap boundary < to <=: 4 failures). The control is what makes the other kills evidence.
- The new tests assert on herdr.sentTexts() — what actually reached the fake pane — not on source text. That is the right observable for a defect whose essence was "the latch says told, the pane got nothing".
The review blocker from the first round is fixed: the latch used to be committed by applyDecision before injectNudge tried to send, and injectNudge swallows its own RuntimeException. On quietNudgeCap: 0, which is this host's configuration, that was the normal path and not an edge case. The latch is now set only when a notice was actually included and the send returned.
Known and deliberately not blocked: Fleetd.main's own one-line call to leadContextSource is not pinned by a test. That is pre-existing and class-wide, tracked in #612.
Fixes the PR #610 review blocker: LeadHeartbeatLoop committed contextNotified
before injectNudge attempted the send, so a transient herdr failure marked the
lead as told when nothing reached its pane, and contextNotice() carried no
latch at all, so a pending-driven INJECT re-appended the notice on every tick
while the context stayed HIGH.
- injectNudge now reports whether agents.send succeeded and persists
contextNotified only when a notice was actually included in the text and
the send did not throw. The latch is split out of applyDecision (kept for
idleSinceNanos/quietCount, applied unconditionally as before) so it is
written on the success path only, once per branch in tick().
- contextNotice gained an overloaded 3-arg form gated on the latch as it
stood before the tick's decision; the existing 2-arg form delegates to it
with alreadyNotified=false, so all pre-existing callers/tests are unchanged.
- tick() is now package-private (mirrors ReplyPushLoop#tick(String)) so tests
can drive the real send path with a fake AgentControl instead of only the
pure decide() function.
- Added tests I-L covering: a failed send does not consume the notice and
retries; a successful send does; the text is gated when the latch is
already set; and the notice appears exactly once across three differently
driven INJECTs.
Both required mutations verified red and reverted:
1. Setting the latch from the Decision regardless of send outcome -> test I
(iAFailedSendDoesNotConsumeTheNotice) fails.
2. Dropping the latch argument at the contextNotice call site -> tests K
(kAPendingDrivenInjectWithTheLatchAlreadySetSendsNoNotice) and L
(lTheNoticeAppearsExactlyOnceAcrossThreeDifferentlyDrivenInjects) fail.
Replaces a wall-clock bet with a manually-driven scheduler, so the tick runs
only when the test runs it. Also closes a second, smaller race the brief did
not name: waiting on Phase.DONE is not enough, because complete() can publish
isDone() before every whenComplete dependent has run.
Verified by the lead: full suite 1841/1841 green in a clean worktree, and an
independent mutation (hasTicketWork forced true) diagnosed as an equivalent
mutant — ReplyPushLoop.injectNudge re-reads the pending collections and returns
early, so that line cannot reach agent.prompt. The worker's own mutation
(ticketCollected made a no-op) is the one that reaches the observable, and it
killed.
The brief's sentence about a null token count at HIGH was broken, and the
worker copied it into the source verbatim. The code was already right; only
the comment was unreadable.
Says what is actually true: a HIGH reading always carries a non-null token
count today, because LeadContextGauge only reaches HIGH by comparing a number
against HIGH_THRESHOLD_TOKENS. That invariant lives in another class and
nothing asserts it, so the branch stays.
Replace the real ScheduledExecutorService backing ReplyPushLoop in this one
test with ManualScheduler, a fake that only runs a tick when the test calls
runDueTasks(). The old test bet a 300ms backoff was wide enough that
collecting the ticket always won the race against the scheduler's own timer
- true on an idle machine, false under a loaded full-suite run, which is
exactly the flake reported.
The rewritten test also waits on setAfterFinishAsyncTaskCompleteHookForTest
(already used elsewhere in this file) instead of polling Phase.DONE, so it
does not race CompletableFuture.complete()'s own publish-then-run-dependents
gap (fleetd #399) while proving ReplyPushLoop.onTicketTerminal really ran
before the ticket is collected.
Verified: backoff=1 (the most hostile value) still passes; mutating
ReplyPushLoop.ticketCollected to a no-op turns the test red with the same
assertion message the original flake reported; three consecutive full-suite
runs are green (1841/1841 each).
LeadHeartbeatLoop can now append a text-only notice to its nudge when the lead's
own LeadContextGauge reading is HIGH and leadHeartbeat.contextHighNudge is on.
Fires once per HIGH stretch (a latch, cleared only by a later OK reading; UNKNOWN
neither sets nor clears it), never spends the quietNudgeCap budget, and never
rolls a pane itself — only the operator can approve a handover.
- LeadContextGauge.Reading.unknown() widened to public for LeadContextSource.none()
- FleetConfig.LeadHeartbeat gains contextHighNudge (null/false = off, unchanged default)
- LeadHeartbeatLoop.decide gains context/contextNotified; Fleetd wires a new
leadContextLookup/leadContextSource factory pair (LeadHeartbeatLoop.LeadContextSource)
- fleetd.example.yaml documents the new key
LeadContextGauge reports OK / HIGH / UNKNOWN for each lead in fleet_list, read
from the transcript Claude Code writes for itself — never from the lead's pane,
so it does not touch the control plane invariant 5 protects.
Why: a lead on this host auto-compacted 30 times in one session, discarding
roughly 250,000 tokens each time, and nothing could see it coming. The handover
feature already existed; what was missing was any way to know when to use it.
Three design points that earned their place:
- finds the transcript by NAME under <configDir>/projects/, never by deriving
the project slug, which is an undocumented Claude Code internal
- three states, not two. Every path that cannot establish a token count says
UNKNOWN with no number, so a lead is never told it is fine when the honest
answer is "I could not look"
- bounded twice: TAIL_BYTES caps bytes read, a 5s cache TTL caps how often
Includes #606, which fixed the two defects this PR's body recorded before it
was ever merged: the null configDir that made the gauge inert on this fleet,
and the torn final line that would have made it flap.
The authoring member died mid-task on a DNS error with the work uncommitted;
the lead recovered it, verified it, and committed it with that stated.
Verified by the lead on the combined state with current main merged in:
mvn -o clean install exit 0, 147 reports, 1841 tests, 0 failures, 0 errors,
0 skipped. LeadContextGaugeTest 9/9, FleetMcpLeadContextGaugeWiringTest 2/2,
FleetdLeadConfigDirLookupTest 6/6, FleetdLeadConfigDirSourceWiringTest 3/3.
Not deployed by this merge. A merge is not a deployment; the running daemon
still holds its old jar until redeploy.
Fixes the two defects recorded in #602's own body.
1. FleetMcp.contextView passed configDir=null, so the gauge read <user.home>/
.claude while this host's lead profile sets an override. Measured before the
fix: 19 transcripts under the real directory, 0 under the fallback. The gauge
would have deployed green and reported UNKNOWN forever, for every lead.
Now threaded via FleetMcp.LeadConfigDirSource, built by Fleetd
.leadConfigDirSource, following fleet.leaders.<name>.profile to that
profile's configDir and reading config.get() live inside the lambda.
2. A torn final line no longer means UNKNOWN. fleet_list reads a transcript
Claude Code may be mid-write on, so the last line can be cut. The old code
treated that as fatal, which would make the gauge flap at random. The stated
reason ("a format change should show as UNKNOWN") does not hold: a real
format change makes EVERY line unparseable, and that case is still caught.
Verified by the lead on the combined state with current main merged in, not on
the branch alone: mvn -o clean install exit 0, 147 reports, 1841 tests, 0
failures, 0 errors, 0 skipped.
Mutation-checked independently by the lead:
Fleetd.leadConfigDirSource body -> none() -> KILLED (1 failure)
FleetMcp.contextView configDir -> null -> KILLED (worker-measured)
KNOWN RESIDUAL, documented rather than overclaimed. main's own one-line call to
leadConfigDirSource could be swapped for none() and the suite stays green. Every
member of this wiring-test family (loopHealthSource, capacitySource,
healthCoverageSource) has the identical gap — no test runs Fleetd.main far enough
to observe which factory it called. Filed separately as a class-wide problem
rather than patched here. The empirical close is the dogfood check after redeploy.
Extract the inline new FleetMcp.LeadConfigDirSource(leadConfigDirLookup(...))
construction in Fleetd.main into a package-private factory,
Fleetd.leadConfigDirSource, mirroring loopHealthSource/capacitySource/
healthCoverageSource. Add FleetdLeadConfigDirSourceWiringTest, which calls the
factory directly with real Profile/Leader fixtures and asserts the returned
source resolves a real configDir -- a property that is false if the factory's
body is mutated to return LeadConfigDirSource.none().
Neither FleetMcpLeadContextGaugeWiringTest nor FleetdLeadConfigDirLookupTest
could catch main losing this wiring: each builds its own instance instead of
calling what main calls. This closes that gap at the factory level, matching
the standard already accepted for loopHealthSource's own wiring test.
fleetd #603. The pid poll had its own fixed 10s budget and then hard-died,
while the health check right after it was allowed 60s for the same daemon.
Under launchd the java process does not exist yet when launchctl load returns,
so the script reported FAIL on a fully successful deploy — and a false FAIL in
that direction invites the hand-rolled stop/start this script exists to replace.
The pid poll now shares HEALTH_WAIT, and a miss falls through to the health
check rather than killing the run. A genuine failure still dies and still
prints the log tail. Worst-case time-to-fail roughly doubles; that cost lands
only on real failures and is the right trade.
Verified by the lead, not taken from the report. Suite exit 0, unpiped.
Two mutations run independently:
budget back to a hardcoded 10 -> KILLED (suite exit 1)
warn back to die on a pid miss -> KILLED (suite exit 1) after 856dfc6
The second survived on the first submission, which is why that commit exists:
nothing covered "pid never appears but healthz answers, so do not die" — the
one case the fall-through is for, and reachable because running_pid() only
recognises a plain java -jar.
PR review (comment 17358) found a real gap via mutation testing: replacing
the "pid never found" warn with die "no process appeared" left the whole
suite green, because neither existing test drove the case the fall-through
exists for -- running_pid() never finds anything (as its own doc comment
says it eventually will) while /healthz answers anyway.
Adds test_await_daemon_started_pid_never_found_but_healthy_warns_and_survives:
running_pid always empty, poll_health_body succeeds. Asserts DIED_CALLED=0,
the warn line is emitted, and NEW_PID stays empty (the honest "could not
establish this" answer, never a guessed pid).
Verified both halves myself: reverting the warn to die "no process appeared"
turns this one test red (FAIL: await_daemon_started must not die...);
restoring it returns the suite to green.
The start step gave the new process its own short, fixed 10s budget before a
hard die, while the health check right after it waits a full HEALTH_WAIT
(60s) for the same daemon. Under launchd, launchctl load returns before the
java process exists, and on a slow host that took longer than 10s -- so the
script reported "no process appeared" on a deploy that had fully succeeded.
wait_for_new_pid/await_daemon_started fold the pid poll and the health check
into one decision: the pid poll now shares HEALTH_WAIT instead of its own
shorter budget, and a miss there falls through to the health check (direct
proof the daemon is up) instead of killing the run. A genuine failure still
dies, and still prints the log tail.
Adds behavioural tests for both acceptance criteria (slow start succeeds,
genuine failure still fails and prints the log tail) in one suite run, plus
unit tests for wait_for_new_pid and a call-site test for the new function.
FleetMcp.contextView hardcoded LeadContextGauge.read(null, ...), so a lead whose
profile sets its own CLAUDE_CONFIG_DIR always read the wrong transcript directory
and reported UNKNOWN forever, with no error anywhere.
- Add FleetMcp.LeadConfigDirSource (same idiom as LeadSeatSource) and thread it
through the constructor / listFleet overload chain / leadView / contextView.
- Add Fleetd.leadConfigDirLookup, wired at construction, following the same
fleet.leaders.<name>.profile link leadSeatLookup already uses, one step
further to that profile's own configDir.
- LeadContextGauge.parse: a single unparseable line (typically the final one,
torn by a write this read raced) is now skipped rather than forcing UNKNOWN;
only when every line in the read window fails to parse does it report
UNKNOWN, which is the real format-change signal.
- Tests: FleetdLeadConfigDirLookupTest (lookup logic), FleetMcpLeadContextGaugeWiringTest
(end-to-end: config naming directory A vs B decides which is read; a lead with
no configured dir degrades without throwing), and two replacement properties in
LeadContextGaugeTest for the torn-line fix plus its all-unparseable control.
#604 item 1. A digest answers "same or different" and cannot say how much. The
byte count is the second signal, needed exactly when two hosts find they differ.
Verified by the lead before merging, from fleetd/: mvn -o clean install exit 0,
143 reports, 1821 tests, 0 failures, 0 errors, 0 skipped; SessionManagerTest 74/74.
Includes ad3d819, correcting an invariant the added comment claimed but that
CharterReceipt.compose() does not hold: digestOf() returns null for blank text
while getBytes().length does not, so a whitespace-only charter pairs a null
digest with a non-zero size. No behaviour change — the digest gate already
omits both on that path, which is the right answer.
The comment said CharterReceipt never pairs a null digest with a non-zero
byte count. It can. compose() derives the digest with digestOf(), which
returns null for blank text, while the byte count is getBytes().length,
which does not. A whitespace-only role charter on a profile with no MCP
produces exactly that pair.
No behaviour change. The gate already omits both fields on that path, which
is the right answer — a size with no digest would describe an artifact we
cannot fingerprint. Only the stated reason was wrong, and a false invariant
in a comment is worse than no comment, because the next reader will widen
the gate on the strength of it.
CharterReceipt carries a byte count next to its digest, but the roster
projection in SessionManager.rosterView only ever copied the digest
across. A digest tells a lead whether two members' charters match; it
cannot say how far apart they are when they don't. Report charterBytes
too, nested in the same conditional as charterSha256 so the two travel
together: the receipt's own contract only ever pairs a non-null digest
with a real byte count, and a member with no composed charter reports
charterSource alone, unchanged from before.
Tests: the existing charter-receipt roster test now asserts charterBytes
against the receipt's own value (not a literal), plus two new cases —
no charter composed (source "none", no digest, no size) and the receipt
itself absent (no charter keys at all).
The fleet01 lead found this. My edit in f5c6a0e left one prose line at 128
columns inside a paragraph wrapped at about 96. It is invisible to any check
that normalises by paragraph, and visible in any raw digest of the block.
No wording changed. Block stays 20937 bytes. The wiki template gets the same
rewrap in its own commit, so the two stay byte-identical.
fleetd had no way to see how full a lead's context window is. On this host a
lead auto-compacted 30 times in one session, discarding roughly 250,000 tokens
and costing 46s to 3m16s each time, and nothing could see it coming.
LeadContextGauge reads the transcript Claude Code itself writes, never the
lead's pane. It finds <sessionId>.jsonl by NAME under <configDir>/projects/
rather than deriving the project slug, which is an undocumented internal.
Three states, not two: OK, HIGH, UNKNOWN. Every path that cannot positively
establish a token count reports UNKNOWN with no number, so a lead is never
told it is fine when the honest answer is "I could not look".
Bounded two ways: TAIL_BYTES caps bytes read off disk, and a 5s cache TTL caps
how often that read happens, because fleet_list is polled constantly.
Recovered by the lead: the authoring member ended on a backend error (DNS
ENOTFOUND) with this work uncommitted and unpushed in its worktree. Verified
before committing: mvn -o clean install exit 0, 1813 tests, 0 failures,
0 errors, 0 skipped, 144 reports; LeadContextGaugeTest 8/8.
KNOWN INCOMPLETE - see the PR. The fleet_list call site passes configDir=null,
which falls back to ~/.claude, but this host's lead profile sets configDir to
an override. Measured: 19 transcripts under the real configDir, 0 under the
fallback. The gauge is therefore INERT on this fleet until that is wired.
Both specifics in the "consult architects" paragraph were mine, not the
operator's. The operator declined twice to rule on them and directed the lead
to consult architects instead, so two architects on different models settled
them over two rounds.
"after two rounds" is gone. It was a ceiling nobody had evidence for, and it
implied a counter fleetd does not have - nothing in the daemon counts rounds.
The bound is now expressed as a shape: form independent positions, then
compare. That is a floor of two without naming a number.
The three-item operator list read as complete, so a lead hitting anything not
on it would conclude it must not ask. It is now explicitly examples, and
"granting access" replaces "credentials" - the case that motivated this was a
forge merge refusal on a protected branch, which "credentials" covers only
awkwardly.
Canonical block and the wiki template updated together; sync check passes.
ltms
lead
Dai Ha
Dai Ha