Compare commits

...

65 Commits

Author SHA1 Message Date
Dai Ha c043d149cf fleetd #775: name the trigger flag for what it now reads
CI / shell-tests (push) Failing after 8s
CI / contract (push) Successful in 1m0s
CI / build (push) Failing after 1m57s
The flag tested leader.tab() and was named for it. It now tests whether
fleet.leaders has any entry at all, so the old name states a condition the
code no longer checks.
2026-10-05 14:21:19 +02:00
Dai Ha fc786d0f67 fleetd #775: say which remedy applies to a lead vs a collaborator
The ticket's correction comment pointed out the refusal message still
implied removing a lead's tab helps, when only placement: tab does.
Restate the message so the lead and collaborator remedies are not
conflated, and keep the variable name the correction specified.
2026-10-05 14:19:17 +02:00
Dai Ha b314cb4d51 fleetd #775: pane-placement guard trigger keys on lead existence, not tab:
The guard's lead half now fires whenever fleet.leaders has any entry,
since a lead's tab is always labelled by the fixed LEAD_TAB_LABEL
constant regardless of its own deprecated tab: field. The refusal
message no longer advises removing a lead's tab:, which cannot
satisfy the guard any more.
2026-10-05 14:19:17 +02:00
Dai Ha a3d296f639 fleetd #770: the lead identity check is the label AND the space
CI / shell-tests (push) Failing after 10s
CI / contract (push) Successful in 49s
CI / build (push) Failing after 2m11s
The redeploy skill's check 4 and the script's closing hint both told the
operator that a lead is found by fleet.leaders.*.tab. Identity is now the
fixed 'lead' label together with the lead's configured workspace, so both
would have sent a reader to a key that no longer decides anything.
2026-10-05 14:01:54 +02:00
Dai Ha 79423787d0 fleetd #770: lead identity keys on the space, not the tab label
CI / shell-tests (pull_request) Failing after 9s
CI / contract (pull_request) Successful in 54s
CI / build (pull_request) Failing after 1m57s
CI / shell-tests (push) Failing after 9s
CI / contract (push) Successful in 53s
CI / build (push) Failing after 1m52s
The lead tab label becomes a fixed constant (Leader.LEAD_TAB_LABEL = "lead");
fleet.leaders.<name>.tab is now optional legacy, matched case-insensitively
alongside the constant via Leader.acceptedLabels(). The uniqueness boundary
between leads moves from the exact tab text to the workspace: FleetConfig
refuses two leaders that share a workspace, LeadTabScanner indexes lead
labels per space (collaborators stay space-agnostic), and
LeadLauncher.leadNameOf/countLeads require both the accepted label and the
lead's own space to match, so a legacy-labelled tab in the wrong space never
counts and a daemon restart never double-spawns a second lead next to a live
one. Config validation also refuses a fleet.tabLabel template or a
collaborator tab that can render as the fixed lead label.
2026-10-05 13:49:14 +02:00
Dai Ha 364b229db9 fleetd #771 step 1: report each pane's workspace label in fleet_list
CI / shell-tests (pull_request) Failing after 8s
CI / contract (pull_request) Successful in 53s
CI / build (pull_request) Failing after 1m57s
CI / shell-tests (push) Failing after 6s
CI / contract (push) Successful in 54s
CI / build (push) Failing after 1m50s
Adds workspaceLabel next to workspaceId on fleet_list's panes row, read
from herdr's workspace.list via a new PaneLocator.workspaceLabelsByWorkspaceId().
An observer's reduced row still excludes it; a herdr failure or an unknown
workspaceId yields workspaceLabel: null without costing the rest of fleet_list.
2026-10-05 11:37:41 +02:00
Dai Ha ac436efefb fleetd #761: invariant 4 — a lead's own pane needs an empty input box
CI / shell-tests (push) Failing after 11s
CI / contract (push) Successful in 52s
CI / build (push) Failing after 1m58s
2026-10-05 11:17:33 +02:00
Dai Ha 7dad054045 Merge remote-tracking branch 'origin/worker/761-draft-aware-lead-nudge-d05850-4' into vfy/761
CI / shell-tests (push) Failing after 9s
CI / contract (push) Successful in 1m0s
CI / build (push) Failing after 2m16s
2026-10-05 11:09:40 +02:00
Dai Ha 4e414c475f fleetd #761: read the input box marker the live TUI actually draws
CI / shell-tests (pull_request) Failing after 10s
CI / contract (pull_request) Successful in 51s
CI / build (pull_request) Failing after 2m1s
The gate matched the box line as "│ >", which appears zero times on a current
Claude Code pane. EMPTY was unreachable, so every lead nudge was held forever.

Match the box as a line *starting* with "❯" or with "│ >", keeping the older
bordered layout readable. A marker further along a line is transcript text — a
caret the operator quoted — so it no longer counts, and the last matching line
is still the live box because the detection region carries scrollback above it.

Look for the generating marker only from the box line down, for the same
reason: an earlier turn's "esc to interrupt" survives in that scrollback, and
holding on it would be the same unreachable-EMPTY failure by another route.

Fixtures: add IDLE_PROMPT_CARET / DRAFTED_PROMPT_CARET from a live pane and
point every lead-pane fake at them. The bordered constants stay, now covering
the older client. Against the old marker these fixtures fail 55 tests across
PromptBoxTest and the three loop tests, which is the production bug reproduced.

cd fleetd && mvn clean install -> BUILD SUCCESS, MVN_EXIT=0,
Tests run: 2164, Failures: 0, Errors: 0, Skipped: 0 (179 surefire XML files).
2026-10-05 11:07:40 +02:00
Dai Ha 9084667493 fleetd #761: hold a lead nudge while the operator's prompt box holds a draft
CI / shell-tests (pull_request) Failing after 7s
CI / contract (pull_request) Successful in 57s
CI / build (pull_request) Failing after 2m13s
herdr's agent.prompt pastes AND submits in one call, so a nudge arriving
while the operator is mid-sentence submitted their unfinished line with the
nudge glued to it. AgentStatus.injectable() cannot see this: it describes
the agent, and an idle agent reports the same status whether its input box
is empty or holds a half-typed line.

New herdr/PromptBox reads the pane's `detection` region — the same region
StatusRefiner uses, and the one the input box is drawn in — and clears a
delivery only when the box is positively empty. A box with characters, a
pane it cannot recognise, and a failed read all hold, because a held nudge
is recoverable and a submitted half-line is not. Whitespace and a cursor
block count as empty. After 20 consecutive holds for one target it logs one
warning, so a box that never clears is visible rather than silent; the
warning repeats only after the box has cleared again.

Wired into the three paths that nudge a lead's own pane:
  - ReplyPushLoop.decide -> WAIT_BUSY (the pending work is re-read next tick)
  - LeadHeartbeatLoop.tick -> a new DRAFT_HELD action that spends neither the
    quiet budget nor the one context notice per HIGH stretch
  - LeadCoordLoop.tick -> the peer message stays held and unacked

Not wired into inject/Injector: no human types into a spawned member's pane,
so it would buy nothing and cost a herdr agent.read per member poll. The
heartbeat reads the pane only for a tick that would otherwise send.

Tests. FakeHerdr gains detectionText(), because one readText cannot be both
a worker's transcript (what the completion scrape reads) and a lead's empty
prompt; it falls back to readText so no existing fixture changes meaning.
Fixtures for the lead-nudge paths now state what their pane shows, since the
behaviour depends on it. 11 new behavioural tests across the three loops plus
InjectorTest, and 13 for the classifier; all 10 loop tests were run against
the unpatched loops first and fail there.

mvn clean install: BUILD SUCCESS, Tests run: 2161, Failures: 0, Errors: 0,
Skipped: 0 (summed from target/surefire-reports).
2026-10-05 10:49:50 +02:00
Dai Ha 2289e94223 fleetd #759: fix the fleet_reply comment and the hand-copied role list
CI / shell-tests (pull_request) Failing after 7s
CI / contract (pull_request) Successful in 44s
CI / build (pull_request) Failing after 1m59s
CI / shell-tests (push) Failing after 10s
CI / contract (push) Successful in 55s
CI / build (push) Failing after 1m59s
Finding 4: the comment above fleet_reply's handler claimed the authz check
asks whether the caller is a worker at all. It actually checks terminal
ownership (Authz.java REPLY/ASK -> caller.ownsSession), which is why an
observer can reply on its own pane with no role test involved.

Finding 5: fleet_list's tool description hardcoded 'architect/dev/reviewer',
missing hunter. Added MemberRole.wireNames() (pulled out of parse()'s error
message builder, which now calls it too) and used it in the description so
the list can't drift again.
2026-10-05 10:44:55 +02:00
Dai Ha 92adfcfae5 fleetd #756/#758: the canonical block no longer says panes are unlistable
CI / shell-tests (push) Failing after 6s
CI / contract (push) Successful in 57s
CI / build (push) Failing after 2m13s
The table row for an unconfigured pane told every session "Neither
fleet_list nor ListAgents lists these". PR #762 made that false for
fleet_list, and a stale note of this shape is the worst kind: it tells a
future session it cannot do the thing at the moment doing it is the job.

Two edits:

- The observer definition now says where an observer finds a target id,
  which is the one thing it could not learn before.
- The table row names the panes array, says the row is full for a lead, an
  architect or a collaborator and filtered for an observer, and keeps the
  herdr tab list join as the fallback. ListAgents still lists none of them.

wiki/7-Use-Cases.md regenerated from this block in the wiki submodule at
4872227; the sync check prints in sync: True.
2026-10-05 10:39:32 +02:00
Dai Ha 5f7f388e69 Merge remote-tracking branch 'origin/worker/756-758-observer-pane-discovery-7e6ffd-1' into vfy/762
CI / shell-tests (push) Failing after 7s
CI / contract (push) Successful in 54s
CI / build (push) Failing after 1m56s
2026-10-05 10:29:49 +02:00
Dai Ha 39accf73e6 fleetd #759: fix the third copy of the role claim, and drop two references that rot
CI / shell-tests (push) Failing after 8s
CI / contract (push) Successful in 49s
CI / build (push) Failing after 1m58s
ConnectionIdentity's Caller record carried the same wrong rule as the two
places PR #760 fixed: it called the terminal a worker's, and read a null
terminal as the primary. The brief for #760 named only two of the three spots.

MemberPresence pointed at FleetMcp.markTrackedCallerPresent by name inside
{@code}, which the compiler does not check, and inject has no dependency on
mcp so a {@link} would add a cross-package reference. States the principle
instead, which stays true whichever roles qualify. Drops a ticket key.
2026-10-05 10:26:36 +02:00
Dai Ha 5c2f296bc3 fleetd #756/#758: panes array reads the architect-slot gate and widens to observer
CI / shell-tests (pull_request) Failing after 8s
CI / contract (pull_request) Successful in 52s
CI / build (pull_request) Failing after 2m9s
paneRole now reads CallerResolver#boundToArchitectSlot (made public, no
second definition) so a slot-bound pane with no live member reports
"architect", matching what sendableObserverTarget already allowed as a
SEND target.

panesVisibleTo now admits an observer, since an observer holds SEND to
another observer pane. Its panes rows are filtered to
sendableObserverTarget and reduced to sessionId/label/status/role/
deliverable; every other caller's rows are unchanged.
2026-10-05 10:24:00 +02:00
Dai Ha 0f2ec7a6b5 fleetd #759: fix three role-model comments that describe the pre-CB-501 rule
CI / shell-tests (pull_request) Failing after 6s
CI / contract (pull_request) Successful in 54s
CI / build (pull_request) Failing after 2m7s
Role.PRIMARY, ConnectionIdentity (class javadoc + callerTerminal), and
MemberPresence's class javadoc each state a role model the code no longer
implements. Comment-only change.
2026-10-05 10:18:01 +02:00
Dai Ha 02eff4c532 fleetd #743: an observer may now send to another observer
CI / shell-tests (push) Failing after 13s
CI / contract (push) Successful in 51s
CI / build (push) Failing after 2m4s
Three claims in the canonical block went false when the observer SEND grant
merged, and this file is the instruction surface the bridge ships:

  - the observer role definition said "never SEND"
  - invariant 3 listed send as "lead, architect, or collaborator"
  - the hand-opened-pane row said such a pane "cannot fleet_send back"

The grant is narrow. Authz permits an observer's SEND only when
CallerResolver.sendableObserverTarget() classifies the target as an observer
too, so a lead, a collaborator, an architect slot and a spawned member are
each refused. TASK_READ stays denied, so #705 remains closed.

Measured on the merged tree: mvn clean install exit 0, 2137 tests from 178
surefire files, 0 failures. Mutating away the grant's call site
(FleetMcp.java:474) kills exactly FleetMcpObserverSendDeliveryTest, so the
behaviour is pinned and not only the predicate.
2026-10-05 09:42:43 +02:00
Dai Ha 92b6d9406b Merge remote-tracking branch 'origin/worker/743-observer-send-4706db-6' 2026-10-05 09:37:14 +02:00
Dai Ha c4498607e1 Merge remote-tracking branch 'origin/worker/743-pane-discovery-ad5b75-5' 2026-10-05 09:37:14 +02:00
Dai Ha e42eab5b4c fleetd #743: pin GET /agents' tab-label merge with a positive test
CI / shell-tests (pull_request) Failing after 9s
CI / contract (pull_request) Successful in 49s
CI / build (pull_request) Failing after 2m5s
2026-10-05 09:34:15 +02:00
Dai Ha b1d2cb48ac fleetd #743: make the pane-label scan best-effort, trim justification comments
CI / shell-tests (pull_request) Failing after 10s
CI / contract (pull_request) Successful in 51s
CI / build (pull_request) Failing after 1m56s
A workspace.list/tab.list failure in the label scan no longer costs the
caller the agent roster (GET /agents) or the leads/members/capacity/
coordinator rows (fleet_list) that never needed it. Both call sites now
fall back to an empty label map on HerdrException, so a pane row still
renders with label:null instead of the whole response failing.

Also cuts four comments down to the current contract, per the project's
comment rule: dropped the reviewer-facing justification from Fleetd's
deliverableTo javadoc, panesVisibleTo's javadoc, the fleet_list handler's
inline comment, and the panesVisible assembly-gate comment, and removed
the two fragments describing what a test must do.
2026-10-05 09:19:16 +02:00
Dai Ha 001367d82c fleetd #743: drop the redundant source-scrape test for attributeIfObserver
CI / shell-tests (pull_request) Failing after 6s
CI / contract (pull_request) Successful in 57s
CI / build (pull_request) Failing after 2m4s
FleetMcpObserverSendDeliveryTest already kills the same mutation end to
end (it asserts the exact attributed text herdr receives), and the code
quality rule in CLAUDE.md caps new source-text tests in this file at the
existing count.
2026-10-05 09:16:29 +02:00
Dai Ha 9fdcaaa8fd fleetd #743: let one observer pane message another, and nothing else
CI / shell-tests (pull_request) Failing after 8s
CI / contract (pull_request) Successful in 56s
CI / build (pull_request) Failing after 2m0s
Authz.SEND now grants an observer a narrow path: it may reach only a
target that CallerResolver.sendableObserverTarget() would itself
resolve as OBSERVER, never a lead, a collaborator, or a live spawned
member's terminal. This mirrors the existing collaborator SEND clause
rather than adding an unconditional caller.isObserver() grant, which
fleetd #705 already rejected as too broad.

Because the receiving pane cannot otherwise tell an observer's SEND
apart from a human paste, FleetMcp.attributeIfObserver prefixes the
delivered text with the sender's own daemon-resolved terminal on both
the MCP and REST entry paths, for exactly this one new path.

FleetAppAuthTest's start() helper never wired a spawnedMemberRole, so
its "worker" fixture actually resolved as OBSERVER under the real
CallerResolver -- invisible before because OBSERVER and WORKER shared
the same (zero) SEND grant. Granting OBSERVER a real SEND surfaced it:
two SEND-denial tests started passing for the wrong caller. Fixed the
fixture to resolve term_a as a live DEV, matching the helper's own
documented contract.
2026-10-05 09:03:16 +02:00
Dai Ha 459a523e2c fleetd #743: expose herdr pane discovery (tab labels) over REST and MCP
CI / shell-tests (pull_request) Failing after 7s
CI / contract (pull_request) Successful in 1m0s
CI / build (pull_request) Failing after 2m15s
GET /agents now carries each agent's tab label, merged in from the same
herdr daemon(s) the roster is drawn from. fleet_list gains a panes array
with the same label, the sessionId fleet_send takes as a target, the
role the daemon resolves that pane as, and the deliverable gate the
injector itself enforces (Fleetd#deliverableTo, now public). Gated like
leads/collaborators (primary, architect, collaborator), not bare READ,
since a tab label and a member's cwd are not roster facts every READ
caller may see.
2026-10-05 08:58:58 +02:00
Dai Ha 291dc02c77 fleetd #743: document that a hand-opened pane is reachable with no config
The lead's intent->tool table had no row for messaging an unconfigured pane,
and the user-scope instruction file said outright that the fleet has no route
to an interactive session unless an operator registers it as a collaborator.
That claim is false and it is load-bearing: a session reading it concludes the
exchange is impossible and stops, which is what happened here.

Delivery is gated on presence, not on SEND. contextExtractor runs on every MCP
request including initialize, markTrackedCallerPresent enrols an observer into
MemberPresence, and deliverableTo tests presence before the lead and
collaborator maps. So connecting the server is the enrolment, and fleet_reply
is gated on owning your own pane, which every pane does.

Add the table row, and add the enrolment side of the deliverability gate to the
flows page next to the existing "a spawned member is not deliverable until it
has mounted the MCP" bullet, which is the same gate read the other way.

Measured on a real pane, not a fake: trinotes answered with no fleet config, no
restart, and its fleet_* tools still deferred and unloaded.
2026-10-05 08:56:01 +02:00
Dai Ha be835aa259 Merge branch 'worker/749-edge-baseline-28d1a0-3'
CI / shell-tests (push) Failing after 12s
CI / contract (push) Successful in 52s
CI / build (push) Failing after 2m4s
2026-10-05 07:47:18 +02:00
Dai Ha 80506c79d0 fleetd #748: drop a cross-reference the comment cleanup left dangling
errorPatternCoverageLine pointed at exhaustedPatternCoverageLine's javadoc
"for the measured swap mutation this pairing guards against". That narrative
was removed from the destination, so the pointer led nowhere.

The pairing with BUILT_IN_DEFAULT is the contract and it stays. What the
mutation proved belongs in history, not in the comment.
2026-10-05 07:46:59 +02:00
Dai Ha 6677ec8c63 fleetd #749: pin PackageCyclesTest's exceptions to exact edges, not whole packages
CI / shell-tests (pull_request) Failing after 7s
CI / contract (pull_request) Successful in 1m8s
CI / build (pull_request) Failing after 2m25s
ignoreCycle() used to exempt every dependency between two packages, in both
directions, for the whole package. That meant a brand new dependency added
later between an already-excepted pair (auth/mcp, mcp/msg, inject/msg,
metrics/msg, msg/session) was silently exempted too, exactly where the
msg package makes the gate matter most.

Replace the package-wide ignore with a frozen baseline of the 45 exact
origin-class -> target-class edges that exist today between those five
pairs, and ignore only those via SliceRule.ignoreDependency(String, String).
A new dependency between a baselined pair is not in that set, so it is no
longer ignored and the existing beFreeOfCycles() check (or, when the new
edge alone would not form a cycle, a dedicated set-equality check) fails
and names the exact origin class, target class and package pair.

The set-equality check also fails on a baseline entry whose dependency no
longer exists in the code, so a removed edge cannot rot in the baseline
and mask the pair's eligibility for the ticket #131 removal steps. Rewrote
the javadoc to describe only the current contract.
2026-10-05 07:45:51 +02:00
Dai Ha ca0c965932 Merge branch 'worker/748-dead-comment-refs-f42ac5-4' 2026-10-05 07:44:44 +02:00
Dai Ha e8ab933cbc fleetd #748: fix dead test-class references and orphaned javadoc blocks
CI / shell-tests (pull_request) Failing after 8s
CI / contract (pull_request) Successful in 51s
CI / build (pull_request) Failing after 2m1s
Drop 2 dead *WiringTest names from 3 comment sites (renamed to
*AssemblyTest), rewriting each sentence to state the code's guarantee
instead of naming a test class. Reattach 5 javadoc blocks that were
orphaned behind a second /** block to the member they actually
describe, trimming history/evidence text down to the current contract
per the project's comment rule. Comment-only; no production logic
changed.
2026-10-05 07:42:55 +02:00
Dai Ha 2adb950a12 fleetd #748: five code-quality rules enter the repo
Two architects reviewed the codebase independently on separate backends and
both returned MIXED, not "a mess": 0 public mutable fields, 12 extends of
which 8 are exceptions, 120 records, and 2 files over 1000 code lines rather
than the 9 a total-line count suggests. Both rejected a Clean Code section,
a SOLID list, a pattern catalogue, class and method length limits, and a
coverage gate as text that would change no behaviour.

The comment rule they were briefed against was never in this repo. It lives
in the operator's personal global config, so it was not in git, never
reviewed, and not versioned with the code. Its "goes in the ADR" line was
unfollowable because this project has no ADR, which sent load-bearing
knowledge to a destination that does not exist. Rule 2 redirects it to
docs/<subject>.md, which exists.

Rule 4 covers what neither architect ranked first and both described: a
testing problem relieved by reshaping production code. 54 static factories
on Fleetd, 7 volatile race hooks in MessageService, 8 tests asserting on
main source as text, and a 452-line composition root across 8 tickets.

Rule 4's judgement half is marked as having no mechanism. A cap stops a
count growing; no test distinguishes a good decomposition from a bad one.

Verified: canonical block still byte-identical with wiki/7-Use-Cases.md.
2026-10-05 07:36:20 +02:00
Dai Ha 7f0c4a8464 fleetd #737: the handover skill carries the outstanding tickets forward
CI / shell-tests (push) Failing after 12s
CI / contract (push) Successful in 55s
CI / build (push) Failing after 2m7s
fleet_handover{open} now returns outstandingTickets and openAsks. The
successor keeps the authority to poll and answer them but not the ids, and
an uncollected terminal ticket loses its reply at the ticket TTL.
2026-10-05 06:09:26 +02:00
Dai Ha 682991a846 Merge branch 'worker/737-9c61d3-4'
CI / shell-tests (push) Failing after 10s
CI / contract (push) Successful in 50s
CI / build (push) Failing after 1m52s
2026-10-05 06:07:33 +02:00
Dai Ha abe617c48c Merge branch 'worker/737-a263f3-3'
CI / shell-tests (push) Failing after 6s
CI / contract (push) Successful in 1m3s
CI / build (push) Failing after 2m20s
2026-10-05 06:01:49 +02:00
Dai Ha 886ce1521d Merge remote-tracking branch 'origin/main' into worker/737-9c61d3-4
CI / shell-tests (pull_request) Failing after 10s
CI / contract (pull_request) Successful in 55s
CI / build (pull_request) Failing after 1m59s
2026-10-05 06:00:49 +02:00
Dai Ha d41aff4012 fleetd #737 unit 5 correction: outstanding() reports terminal-phase tickets too
A DONE/FAILED ticket nobody has polled yet is destroyed on a timer by
pruneTerminalTickets' completion-based TTL, while a PENDING ticket is not
going anywhere. Drop the !future.isDone() filter so outstanding() reports
every ticket the caller owns that is still in tasks, with its real phase.
2026-10-05 06:00:43 +02:00
Dai Ha 8a1d73b39e fleetd #737 unit 4: key the rollover single-flight claim on the lead's name
CI / shell-tests (pull_request) Failing after 7s
CI / contract (pull_request) Successful in 52s
CI / build (pull_request) Failing after 1m59s
rollingByTerminal keyed the single-flight lock on p.leadTerminal(), the pane
address. A roll replaces the pane, so a second roll of the same lead opened
from the new terminal landed on a different map key and could run concurrent
with the first roll's still-in-flight continuation.

PendingRollover now carries rolloverKey, resolved once in open() from
leadNameForTerminal while the lead is certainly still live, falling back to
the terminal itself when the name resolves null or blank. confirm()'s claim
and both release sites (the continuationRunner-rejection catch and
runRollover's finally) use the carried key instead of recomputing it, since
leadNameForTerminal no longer resolves the old terminal by release time.
NOT_YOUR_ROLLOVER and the pane-teardown calls stay keyed on p.leadTerminal(),
unchanged.

rollingByTerminal is renamed rollingByLead to match.
2026-10-05 05:57:59 +02:00
Dai Ha 70a735b638 fleetd #737 unit 5: fleet_handover{open} reports outstanding tickets and open asks
CI / shell-tests (pull_request) Failing after 9s
CI / contract (pull_request) Successful in 51s
CI / build (pull_request) Failing after 2m10s
MessageService.outstanding(callerOwner) lists the caller's non-terminal
delegations and the subset paused in fleet_ask, filtered by the same
ownsTicket rule poll() already uses. FleetMcp threads the caller's owner
key into handover/handoverOpen and adds outstandingTickets/openAsks to
the open() JSON, alongside the unchanged token/handoverPath/requestedAtMillis.
2026-10-05 05:56:06 +02:00
Dai Ha 803c91ea6c fleetd #737 unit 3: name the resolving accessor in LeadRollover's javadoc
CI / shell-tests (push) Failing after 6s
CI / contract (push) Successful in 1m0s
CI / build (push) Failing after 2m7s
The heartbeat loop now resolves its nudge target through
PrimaryRegistry#currentPrimaryTerminal(), so the sentence describing what a
background loop with no caller uses named the raw accessor instead.
2026-10-05 05:37:32 +02:00
Dai Ha d438a74575 Merge branch 'worker/737-20d1d9-1' 2026-10-05 05:37:32 +02:00
Dai Ha 7467ffa252 fleetd #737 unit 6: drop the stale unnamed-primary claim from the REST status comment
CI / shell-tests (push) Failing after 6s
CI / contract (push) Successful in 55s
CI / build (push) Failing after 2m9s
The comment described the pre-#737 rule, where a null caller key read every
ticket. The pendingAsk gate now matches the unnamed primary's key like any
other, so the exemption it named no longer exists.
2026-10-05 05:32:59 +02:00
Dai Ha f4176ae455 fleetd #737 unit 3: probe the fallback terminal and resolve it by lead name
CI / shell-tests (pull_request) Failing after 10s
CI / contract (pull_request) Successful in 55s
CI / build (pull_request) Failing after 1m55s
ReplyPushLoop.resolveLiveLead dropped a dead per-target delegation and
retried PrimaryRegistry.nudgeTargetFor, but returned that fallback without
checking isLive. The fallback is now probed the same way the first lead is,
and the method returns empty rather than trust a dead terminal.

PrimaryRegistry records a delegating lead's name alongside its learned
terminal and resolves the name back to its current terminal at nudge time,
through a name-to-terminal lookup backed by the live lead-tab scan. A name
with no current match falls back to the terminal that was actually learned,
so an unnamed primary, an off-host lead, or a non-herdr lead keeps working
exactly as before. LeadHeartbeatLoop now reads the resolved current terminal
instead of the raw learned one.
2026-10-05 05:29:06 +02:00
Dai Ha 6ab3a81af7 fleetd #737 unit 6: stop the unnamed primary sharing the internal bypass
CI / shell-tests (pull_request) Failing after 8s
CI / contract (pull_request) Successful in 55s
CI / build (pull_request) Failing after 1m59s
ownsTicket treated a null callerOwner as "read everything", conflating
the internal no-check test seam with a real unnamed primary's owner
key. Give them two different values: a named INTERNAL_NO_OWNER_CHECK
marker for the test-only bypass, and null as just another owner key
that must equal the ticket's recorded creatorOwner (including a
null-to-null match, so an unnamed primary still owns its own tickets).
Applies to both ownsTicket call sites, poll and pendingAsk.
2026-10-05 05:24:37 +02:00
Dai Ha 8fa8d18c97 fleetd #726: rewrite the handover skill for the restart roll
The roll now ends the pane and launches a fresh process instead of typing
/clear, and that jar is deployed, so the skill's self-dated "a change is
coming" note had to go.

Measured on 2026-10-05 before writing:

  grep -c "lead-rollover: rolled" fleetd/fleetd.out   -> 20
  grep -c "lead-rollover:" fleetd/fleetd.out          -> 86 (control)
  <tail from the last "fleetd listening"> | grep -c "lead-rollover:" -> 0
  grep -c 'RELAUNCH_NEVER_READY\|RELAUNCH_NOT_RECOGNISED\|OLD_PANE_NEVER_DIED' -> 0

So all 20 recorded rolls ran under /clear and the restart path has never
executed. The section says that rather than implying the old numbers
describe it.

Also:
- name all eight RollState outcomes, with what each one guarantees
- state that relaunchReadySeconds bounds each of two waits, not the pair
- drop the "never observed as WORKING after 8 consecutive IDLE/DONE polls"
  paragraph: grep finds that wait is deleted, so it cannot appear
- drop the #621 warning: contextNotice now takes requireOperatorConfirm
- split the surprise bullets into their own section
2026-10-05 05:04:52 +02:00
Dai Ha 9d653e86df fleetd #726: name RELAUNCH_NEVER_READY in the IN_PROGRESS terminal-state list
CI / shell-tests (push) Failing after 7s
CI / contract (push) Successful in 57s
CI / build (push) Failing after 2m7s
The javadoc on RollState.IN_PROGRESS enumerates the terminal states the entry
can be overwritten with, and omitted RELAUNCH_NEVER_READY. That state is
reachable at LeadRollover.java:710, so the list told a reader a state could not
occur when it can. Found by a reviewer on PR #742, outside its assigned scope.

Comment only; no behaviour change.
2026-10-04 21:44:03 +02:00
Dai Ha f6d1131d7a Merge remote-tracking branch 'origin/worker/726-unit2-75cb13-4' 2026-10-04 21:43:34 +02:00
Dai Ha a0505dc614 fleetd #726 unit 2: scope the member-daemon assertion to the roll itself
CI / shell-tests (pull_request) Failing after 8s
CI / contract (pull_request) Successful in 52s
CI / build (pull_request) Failing after 2m0s
The assembled daemon's own boot-time orphan-worker reap makes a real call on
the member fake before any roll starts. Clear the member fake's recorded
calls once assembly finishes and before the roll begins, so the assertion
measures calls made since the roll started rather than the whole process's
lifetime, and reword its message to say so.
2026-10-04 21:24:49 +02:00
Dai Ha d2f30f1654 fleetd #726 unit 2: cover the bootstrapText relaunch send with a dedicated test
CI / shell-tests (pull_request) Failing after 9s
CI / contract (pull_request) Successful in 52s
CI / build (pull_request) Failing after 1m46s
LeadRollover already sends every call through the lead-bound AgentControl and
WorkspaceControl it receives at construction, so no production code needed a
routing fix. Add a regression test that drives a full relaunch to the point
where recognition times out and asserts bootstrapText still lands on the lead
daemon and never on the member daemon, the one path the existing assembly test
never reaches.
2026-10-04 21:17:15 +02:00
Dai Ha cd1f04cbb4 Merge remote-tracking branch 'origin/worker/737-owner-key-ff061f-10'
CI / shell-tests (push) Failing after 9s
CI / contract (push) Successful in 53s
CI / build (push) Failing after 1m48s
2026-10-04 21:11:20 +02:00
Dai Ha 73137f198f Merge remote-tracking branch 'origin/main' into worker/726-unit2-75cb13-4 2026-10-04 20:55:29 +02:00
Dai Ha 4ffe49f3bb fleetd #726 unit 2: replace /clear-based lead rollover with a real process restart
CI / shell-tests (pull_request) Failing after 6s
CI / contract (pull_request) Successful in 59s
CI / build (pull_request) Failing after 1m56s
LeadRollover's deferred continuation now ends the old lead's pane, relaunches
a fresh one, and bootstraps it, instead of sending /clear into the same
process. The relaunch step runs two separate bounded waits instead of one
combined check: a readiness wait (the fresh pane reaches a real turn
boundary) is the safety gate and withholds bootstrapText on timeout
(RELAUNCH_NEVER_READY); a recognition wait (the fresh terminal shows up in
the live-lead map) is bookkeeping only, so a timeout there still lets
bootstrapText go out (RELAUNCH_NOT_RECOGNISED). clearSettleSeconds is
retired in favor of relaunchReadySeconds (default 45), which bounds both
waits. Updates FleetConfig/FleetMcp operator-facing text to match.
2026-10-04 20:44:42 +02:00
Dai Ha efd9cdb983 fleetd #737 units 1+2: key tickets and turns on a stable owner, not a terminal
CI / shell-tests (pull_request) Failing after 9s
CI / contract (pull_request) Successful in 57s
CI / build (pull_request) Failing after 2m4s
Add Principal.ownerKey(): role-prefixed, keyed on name for a named lead and
a collaborator (survives a handover's terminal change), on terminal for a
worker, architect and observer, and on a distinct "anonymous" value for an
unauthenticated caller so that case no longer relies on Authz refusing it
first. The unnamed primary keeps a null key, preserving its primary-wide
ticket rule.

Thread that key through Task.creatorOwner, Rendezvous.Owner, poll,
pendingAsk and answer in place of a raw terminal, in both the MCP and REST
surfaces, so a named lead whose terminal changes can still poll and answer
its own delegations while a different lead is refused both.

Mutation evidence (each one-line change killed a named test, then reverted
to green):
- Principal.ownerKey() PRIMARY case made unconditional (dropped the
  null-name guard) -> ownerKeyCoversEveryRole dies:
  "expected: <null> but was: <leader:null>"
- OBSERVER case changed to use the "worker" prefix -> ownerKeyCoversEveryRole
  dies: "expected: <observer:term_observer> but was: <worker:term_observer>"
- prefixed() changed to drop the role prefix entirely -> both
  ownerKeyCoversEveryRole and rolePrefixesKeepLeadAndArchitectKeysDistinct
  die on a lead/architect key collision: "expected: <leader:opus> but was:
  <opus>"
- PRIMARY case changed to key on terminal instead of name ->
  rolePrefixesKeepLeadAndArchitectKeysDistinct and ownerKeyCoversEveryRole
  die: "expected: <leader:opus> but was: <leader:term_lead>"
- ARCHITECT case changed to key on the slot name instead of terminal ->
  same two tests die: "expected: <architect:opus> but was: <architect:design>"

All five mutations were caught by the existing test suite; no test needed
adding.
2026-10-04 20:42:44 +02:00
Dai Ha aabecce901 Merge remote-tracking branch 'origin/worker/736-presence-forget-f35144-9'
CI / shell-tests (push) Failing after 8s
CI / contract (push) Successful in 54s
CI / build (push) Failing after 1m58s
2026-10-04 20:19:52 +02:00
Dai Ha 6754b4edbc fleetd #736: release clears the member's presence entry
CI / shell-tests (pull_request) Failing after 8s
CI / contract (pull_request) Successful in 54s
CI / build (pull_request) Failing after 1m45s
SessionManager.releaseRemoved() tore down a member's registry row and pane
but never cleared it from MemberPresence, so a terminal stayed marked
"present" for the daemon's lifetime after release/idle-reap/shutdown drain.
Clear it in the method's unconditional finally block, alongside the other
must-always-run teardown step, so every release path (explicit release,
the idle reaper's releaseIfCurrent, and a shutdown drain) forgets it the
same way, and a throw from the dirty-worktree check does not skip it.

MemberPresence.forget(null) throws NullPointerException (verified empirically:
ConcurrentHashMap.remove(null) NPEs on key.hashCode()), so the new call guards
on a non-null, non-blank terminal id rather than relying on forget to no-op.
2026-10-04 20:15:34 +02:00
Dai Ha 787ae0ed7a fleetd #726: tell the handover skill which rollover behaviour is live
CI / shell-tests (push) Failing after 10s
CI / contract (push) Successful in 55s
CI / build (push) Failing after 1m59s
Unit 2 replaces the /clear continuation with a real process restart, so every
paragraph in the skill that describes /clear goes false the moment the new jar
is deployed. The code is not merged yet, and a merge is not a deployment, so
rewriting those paragraphs now would hand a lead doing a handover tonight a
document that does not match the daemon it is talking to.

Add a dated note instead. It states that the /clear text stays accurate while
the old jar runs, and gives a test a lead can apply with no shell: the
fleet_handover tool description is served by the running daemon, so if it still
says "clear your pane", the old behaviour is live. It also names the two things
that change, including the one that doubles as a second indicator -- the
"never observed as WORKING after 8 consecutive IDLE/DONE polls" warning cannot
appear once the wait that logs it is deleted. The note names the condition for
deleting itself.

Re-measure the roll evidence while here. The skill recorded four
"lead-rollover: rolled" lines from 2026-09-22; the log now holds 20, against a
control of 86 "lead-rollover:" lines, and "Unknown command" still returns 0.
Add the elapsed spread (median 16507 ms, max 48261 ms, two above 45000 ms) with
the caveat that it times the whole roll and is dominated by the wait for the
calling turn to end, so a slow roll is not a failed one.

Markdown only, no code touched, so no build was run.
2026-10-04 20:09:58 +02:00
Dai Ha e3050efe8b fleetd #705: correct the stale reason on the TASK_READ gate
CI / shell-tests (push) Failing after 8s
CI / contract (push) Successful in 50s
CI / build (push) Failing after 1m48s
The comment said ticket ids are a sequential counter with no owner check, so a
holder could walk every ticket and read another session's reply. PRs #712 and
#716 added that owner check: MessageService.ownsTicket compares a ticket's
creatorTerminal to the caller on every read.

The rule is still right, so only the reason changes. This matters now because
fleetd #737 is deciding ticket ownership across a lead handover, and a reader
who believed the old text could delete the TASK_READ restriction on the grounds
that its stated reason no longer applies.

Comment-only. mvn -o clean install: Tests run: 2083, Failures: 0, Errors: 0,
BUILD SUCCESS. Flagged by the #705 option-1 worker as out of its scope, which
was the right call.
2026-10-04 19:59:45 +02:00
Dai Ha 11998cd626 Merge remote-tracking branch 'origin/worker/705-observer-14c258-6'
CI / shell-tests (push) Failing after 8s
CI / contract (push) Successful in 56s
CI / build (push) Failing after 1m54s
2026-10-04 19:52:34 +02:00
Dai Ha 8e5394f63f fleetd #705 option 1: narrow the unconfigured-pane floor to OBSERVER
CI / shell-tests (pull_request) Failing after 6s
CI / contract (pull_request) Successful in 58s
CI / build (pull_request) Failing after 1m57s
Adds Role.OBSERVER as the bottom rung CallerResolver falls to when a
herdr pane matches no live roster entry, lead, architect slot, or
collaborator tab. An observer may only READ/METRICS and REPLY/ASK on
its own pane. Widens the presence gate so an observer's MCP contact
still marks it deliverable, matching what already happens for a
worker or architect, so a pane that outlives a daemon restart is not
left permanently undeliverable.

Ships as defence in depth alongside the already-merged ticket-owner
check (#712/#716), which closed the reachable exploit this ticket
reported.
2026-10-04 19:46:54 +02:00
Dai Ha 428a12af62 fleetd #722: reconcile presence that arrives before a session's registry entry
CI / shell-tests (pull_request) Failing after 9s
CI / contract (pull_request) Successful in 50s
CI / build (pull_request) Failing after 1m48s
CI / shell-tests (push) Failing after 9s
CI / contract (push) Successful in 47s
CI / build (push) Failing after 1m46s
A member whose MCP contact lands between launcher.spawn() and registry.put()
had its presence marked, but the SPAWNING -> READY transition that markPresent
triggers found no registry entry yet and silently did nothing. The mark then
persisted while registration left the session in SPAWNING, with nothing to
retry the transition. That left the session undeliverable to reclaim/seat
accounting even though it was present and deliverable.

Add SessionManager.reconcilePresence, called right after registry.put in both
the plain-spawn and worktree-spawn paths, to retry the transition for a
terminal already marked present. One private helper serves both call sites.

Tests cover both orderings (contact-then-register and register-then-contact)
for both spawn paths, plus a terminal never marked present staying in
SPAWNING. The contact-then-register tests use a new PresenceRacingLauncher
test double that marks presence from inside spawn(), before acquire()'s own
registry.put runs.
2026-10-04 19:23:02 +02:00
Dai Ha a332dfdb2c Merge remote-tracking branch 'origin/worker/726-ea34a0-2'
CI / shell-tests (push) Failing after 7s
CI / contract (push) Successful in 54s
CI / build (push) Failing after 1m53s
2026-10-04 19:09:03 +02:00
Dai Ha 7b3beaa209 Merge remote-tracking branch 'origin/worker/726-10cbf0-1'
CI / shell-tests (push) Failing after 10s
CI / contract (push) Successful in 49s
CI / build (push) Failing after 1m53s
2026-10-04 19:06:47 +02:00
Dai Ha b3b2bf3da6 fleetd #726 unit 1 review fixes: correct relaunch's javadoc and dedupe its resolve logic
CI / shell-tests (pull_request) Failing after 10s
CI / contract (pull_request) Successful in 57s
CI / build (pull_request) Failing after 1m49s
relaunch's javadoc said it reads the live config; it actually reads the
FleetConfig snapshot this launcher was constructed with (fleet.leaders
is the frozen half), so say that and note a profile/tab edit needs a
daemon restart.

relaunch's recognise-only refusal reused ensureLeads()'s log wording,
which claims the lead 'is not live' — true in ensureLeads()'s context
(reached only after a short live count), false in relaunch's (which
never counts liveness, by design). Dropped that clause.

Pulled the declared/creatable/profile-configured resolution shared by
ensureLeads() and relaunch() into one private resolveLaunchable(name)
helper (returns a new ResolvedLead(lead, profile) record, or null
having logged), so the three refusals and their wording live in one
place instead of two copies that can drift. Behaviour-preserving:
ensureLeads() keeps its own liveness-count logic around the shared
resolve, and the existing 37 LeadLauncherTest cases are unchanged and
still pass.
2026-10-04 18:59:56 +02:00
Dai Ha 8bb2aa0be4 Merge remote-tracking branch 'origin/worker/729-5961c6-3'
CI / shell-tests (push) Failing after 8s
CI / contract (push) Successful in 57s
CI / build (push) Failing after 2m0s
2026-10-04 18:59:54 +02:00
Dai Ha 1fc9e85bf1 fleetd #729: fold a per-boot nonce into every turnId
CI / shell-tests (pull_request) Failing after 6s
CI / contract (pull_request) Successful in 54s
CI / build (pull_request) Failing after 1m57s
askSeq restarts at 0 on every daemon boot, so a turnId (session#n)
minted by one Rendezvous instance could be minted again by a later
instance and resolve to an unrelated ask. Fold a per-instance nonce
into the mint, the same way #719 fixed MessageService's ticket ids.
2026-10-04 18:53:05 +02:00
Dai Ha 38544d467c fleetd #726 unit 1: give LeadLauncher a public single-lead relaunch seam
CI / shell-tests (pull_request) Failing after 8s
CI / contract (pull_request) Successful in 45s
CI / build (pull_request) Failing after 1m48s
Adds LeadLauncher.relaunch(name), which starts exactly the named lead
from the live config, outside of ensureLeads()'s instances bookkeeping.
It retries the whole launch attempt (not just the agent_name_taken/
agent_pane_busy cases ResilientAgentLaunch already retries inside one
agents.start call) up to RELAUNCH_ATTEMPTS times.

launch() now returns the started Agent (null on failure) instead of a
boolean, so relaunch() and ensureLeads() share the same primitive.
2026-10-04 18:52:38 +02:00
70 changed files with 5850 additions and 1791 deletions
+73 -34
View File
@@ -13,7 +13,7 @@ writes a handover file, and the new session reads that file and carries on.
- **By hand.** You write the file, then tell the operator where it is. The operator starts the new - **By hand.** You write the file, then tell the operator where it is. The operator starts the new
session and points it at the file. This always works. session and points it at the file. This always works.
- **With `fleet_handover`** (fleetd #480, merged 2026-09-11). You ask fleetd to do the swap: it - **With `fleet_handover`** (fleetd #480, merged 2026-09-11). You ask fleetd to do the swap: it
checks the file, clears your pane, and tells the fresh session to read it. This needs checks the file, restarts your pane, and tells the fresh session to read it. This needs
`leadRollover:` in `fleetd.yaml`; without it every action answers a clean refusal naming `leadRollover:` in `fleetd.yaml`; without it every action answers a clean refusal naming
`NOT_CONFIGURED`, and you fall back to the manual path. Section 11 below is the procedure. `NOT_CONFIGURED`, and you fall back to the manual path. Section 11 below is the procedure.
@@ -133,8 +133,17 @@ A handover file is a record of state and decisions. It is not a diary.
**Run the three steps in this order. The order is not a style choice — the wrong order is **Run the three steps in this order. The order is not a style choice — the wrong order is
refused.** refused.**
1. **`fleet_handover{action: "open", reason: "<why now>"}`.** It returns a `token` and the 1. **`fleet_handover{action: "open", reason: "<why now>"}`.** It returns a `token`, the
`handoverPath` you must write to. Nothing has happened to your pane yet. `handoverPath` you must write to, and `outstandingTickets` plus `openAsks`. Nothing has happened
to your pane yet.
**Copy `outstandingTickets` and `openAsks` into the handover file.** Your successor keeps the
authority to poll those tickets and answer those asks, because both are gated on the lead's
name, which does not change when your pane does. What it does not keep is the ids — they exist
only in your context and in this response. A ticket already in a terminal phase is the urgent
one: its reply lives only in memory and is deleted once the ticket TTL passes, so an uncollected
report is lost for good. Poll those before you confirm, or name them in the file so your
successor polls them first.
**Write to exactly that path, and do not resolve it yourself.** It is always absolute, even when **Write to exactly that path, and do not resolve it yourself.** It is always absolute, even when
the operator configured a relative `handoverPath`: fleetd resolves a relative one against your the operator configured a relative `handoverPath`: fleetd resolves a relative one against your
@@ -159,15 +168,57 @@ fails.
`{action: "cancel", token}` drops a pending request without rolling. `{action: "cancel", token}` drops a pending request without rolling.
**Things that will surprise you:** ## 12. A roll restarts your process
- **`accepted` does not mean your pane has been cleared.** It means every gate passed and the roll The daemon ends your pane, launches a fresh one, waits for the new terminal to be recognised as a
lead, and only then sends the bootstrap text. Your `claude` process really exits, so a newer CLI on
disk is loaded. It does not type `/clear`.
**The restart path is live in the code but has not run yet.** Measured 2026-10-05: the log holds no
`lead-rollover:` line since the current daemon started, and no occurrence of any new outcome name.
All 20 rolls recorded further down ran under the older `/clear` behaviour, so read them as history
rather than as evidence about your own roll. Re-measure with:
```bash
grep -c "lead-rollover: rolled" fleetd/fleetd.out # successful rolls
grep -c "lead-rollover:" fleetd/fleetd.out # positive control: must be larger
```
Run the control line too. A broken pattern returns a clean `0` that reads exactly like good news.
If the first number has grown past 20, somebody has rolled under the restart path, and this section
should be replaced with what they measured.
**Three separate timeouts bound a roll.** `leadRollover.relaunchReadySeconds` (default 45,
`FleetConfig.java:1483`) bounds **each** of two waits that run after the relaunch, so the worst case
there is about twice that number, not 45 seconds in total. A third bound gives your old pane 10
seconds to die (`LeadRollover.PANE_DEATH_TIMEOUT_SECONDS`).
`fleet_handover{action: "status", token}` answers with one of these:
| Outcome | What it means |
|---|---|
| `IN_PROGRESS` | still running; it always ends on one of the rows below |
| `ROLLED` | the roll succeeded |
| `TURN_NEVER_SETTLED` | your turn ran past `leadRollover.turnSettleSeconds`; nothing was touched |
| `OLD_PANE_NEVER_DIED` | your pane did not exit within the 10-second bound |
| `RELAUNCH_FAILED` | launching the fresh pane failed |
| `RELAUNCH_NEVER_READY` | the fresh pane never became ready within `relaunchReadySeconds` |
| `RELAUNCH_NOT_RECOGNISED` | the fresh terminal never resolved as a lead |
| `FAILED` | the roll threw; `runRollover`'s catch records this rather than leaving it stuck |
Only `TURN_NEVER_SETTLED` guarantees your context is intact. The other failures can leave you
already gone, so you may never read them yourself — they are in the daemon log for whoever looks
next.
## 13. Things that will surprise you
- **`accepted` does not mean your pane has been restarted.** It means every gate passed and the roll
is scheduled to run once your current turn ends. Say your goodbye in the same turn — you will not is scheduled to run once your current turn ends. Say your goodbye in the same turn — you will not
get another one. get another one.
- **If you are still running after that turn, the roll did not happen.** A roll that works clears - **If you are still running after that turn, the roll did not happen.** A roll that works ends your
you, so surviving your own goodbye is itself the signal that it refused. Check with process, so surviving your own goodbye is itself the signal that it refused. Check with
`fleet_handover{action: "status", token}`, using the token you confirmed. `TURN_NEVER_SETTLED` `fleet_handover{action: "status", token}`, using the token you confirmed. `TURN_NEVER_SETTLED`
means your turn ran past `leadRollover.turnSettleSeconds` and **no `/clear` was ever sent**: your means your turn ran past `leadRollover.turnSettleSeconds` and **your pane was never ended**: your
context is intact and nothing was lost. Open a fresh request and retry. Never assume the roll context is intact and nothing was lost. Open a fresh request and retry. Never assume the roll
succeeded because `confirm` answered `accepted` — by the time it refuses, there is no caller left succeeded because `confirm` answered `accepted` — by the time it refuses, there is no caller left
to tell, so this check is the only thing that closes that gap. to tell, so this check is the only thing that closes that gap.
@@ -193,41 +244,29 @@ fails.
**deferred, not hot** — it is read once at boot, so an edit does nothing until the daemon is **deferred, not hot** — it is read once at boot, so an edit does nothing until the daemon is
redeployed. redeployed.
**Until fleetd #621 merges, the nudge text will tell you to ask the operator even where the The context nudge tracks this value, so its text and the config agree (fleetd #621 —
daemon no longer requires it.** `LeadHeartbeatLoop.contextNotice()` hardcodes "ask the operator" `LeadHeartbeatLoop.contextNotice` takes `requireOperatorConfirm`). You still read the config
and takes no config, so it cannot know. Trust the config value over the nudge text. Once #621 is rather than the nudge, because the nudge only reaches you when your context is already high.
merged and deployed, the nudge matches the config and this warning can be deleted.
- **The roll can still refuse after `confirm` returns**, and by then there is no caller to tell. - **The roll can still refuse after `confirm` returns**, and by then there is no caller to tell.
Those outcomes are logged only, as `lead-rollover:` lines in the daemon log. Those outcomes are logged only, as `lead-rollover:` lines in the daemon log.
- **The bootstrap prompt works end to end. Measured 2026-09-22.** This used to say the fix was - **The bootstrap prompt works end to end — measured under the older `/clear` path.** On 2026-09-22
unproven (fleetd #489) and told you to expect a failure. That is no longer true. The daemon log the daemon log held four `lead-rollover: rolled` lines; on 2026-10-04 it held **20**, against a
now holds four `lead-rollover: rolled` lines, and three of them ran on 2026-09-22 at 10:01:43, control of 86 `lead-rollover:` lines. Each roll started a fresh session against the handover file,
10:38:28 and 11:15:47. Each one cleared the old lead and started a fresh session against the with the configured `bootstrapText` arriving as its first message, and no context was lost. So the
handover file, with the configured `bootstrapText` arriving as its first message. No context was bootstrap half of the roll is proven, and that half did not change. Section 12 says how to check
lost. The old `Unknown command: /clearFresh` failure from 2026-09-12 does not appear in the log whether anything has rolled under the restart path since.
at all. Re-measure both numbers with:
```bash 19 of the 20 carry an `elapsedMs`: median 16507 ms, maximum 48261 ms, and two above 45000 ms. That
grep -c "lead-rollover: rolled" fleetd/fleetd.out # successful rolls figure times the **whole** roll, and the wait for your own turn to end dominates it. Expect a roll
grep -c "lead-rollover:" fleetd/fleetd.out # positive control: must be larger to take tens of seconds, and do not treat a slow one as a failed one. The restart path adds a pane
grep -c "Unknown command" fleetd/fleetd.out # the old failure: expect 0 death and a relaunch to that work, so expect it to be slower rather than faster — but nobody has
``` measured it, so do not quote a number for it.
Run the control line too. A broken pattern returns a clean `0` that reads exactly like good news.
If the first number stops growing across rolls, or `Unknown command` returns anything above 0,
the bootstrap has regressed and this paragraph is stale again.
**You still write the file before you confirm, and never the other way round.** That order is not **You still write the file before you confirm, and never the other way round.** That order is not
about the bootstrap being unreliable. It is what the daemon checks: the handover file must have about the bootstrap being unreliable. It is what the daemon checks: the handover file must have
been modified *after* the open request, or `confirm` refuses it as stale. been modified *after* the open request, or `confirm` refuses it as stale.
- **One warning in the log is normal and is not a failure.** Every one of the three rolls above also
logged `/clear on term_… was never observed as WORKING after 8 consecutive IDLE/DONE polls —
releasing rather than wedging the roll`. The daemon could not see the pane go WORKING after
`/clear`, so it released instead of hanging. The roll then succeeded anyway. That is the safe
branch behaving correctly. Do not report it as a broken roll.
## Writing style ## Writing style
Write in plain English. Use everyday words, one idea per sentence, and active voice. Keep every Write in plain English. Use everyday words, one idea per sentence, and active voice. Keep every
+6 -3
View File
@@ -59,9 +59,12 @@ if the script is unavailable or a step fails, this is what it was protecting you
3. **A restart is the only way deferred config keys take effect.** That is usually the reason to do 3. **A restart is the only way deferred config keys take effect.** That is usually the reason to do
it. The startup log names which keys it accepted and which it deferred — read those lines rather it. The startup log names which keys it accepted and which it deferred — read those lines rather
than assuming. than assuming.
4. **Re-check identity afterwards.** Call `fleet_whoami` and confirm it still answers `primary`. The 4. **Re-check identity afterwards.** Call `fleet_whoami` and confirm it still answers `primary`. A
lead is found by its tab label (`fleet.leaders.*.tab`), and a lead whose tab no longer matches is lead is found by two things together: its tab is labelled `lead`, and that tab sits in the space
demoted to worker, which refuses every orchestration call. named by `fleet.leaders.<name>.workspace`. Both must match, so a renamed tab *and* a space whose
label differs from the config each demote the lead to worker, which refuses every orchestration
call. A `tab:` still in config is accepted as a second label for that lead, and the daemon logs
one deprecation warning naming it at startup.
5. **Prove the new jar is the one running.** Confirm a *fresh* `fleetd listening` line at the end of 5. **Prove the new jar is the one running.** Confirm a *fresh* `fleetd listening` line at the end of
`fleetd/fleetd.out`, dated after the restart. An old daemon that never died looks identical from `fleetd/fleetd.out`, dated after the restart. An old daemon that never died looks identical from
the outside. the outside.
+64 -17
View File
@@ -27,26 +27,34 @@ through its `fleet_*` tools. No session addresses a peer, a broker, or the netwo
**Every role reads this file.** A member runs in a git worktree of this same repo, so it inherits **Every role reads this file.** A member runs in a git worktree of this same repo, so it inherits
this `CLAUDE.md` verbatim, and every rule below is role-conditional. this `CLAUDE.md` verbatim, and every rule below is role-conditional.
**Call `fleet_whoami`.** It returns `primary`, `worker`, `architect`, or `collaborator`, resolved by **Call `fleet_whoami`.** It returns `primary`, `worker`, `architect`, `collaborator`, or `observer`,
the daemon from your connection — unforgeable, and the same resolution its authorization gate uses. resolved by the daemon from your connection — unforgeable, and the same resolution its authorization
A worker also carries its `sessionId`, `profile`, `worktree` and `branch`; an architect carries the gate uses. A worker also carries its `sessionId`, `profile`, `worktree` and `branch`; an architect
slot name it was bound to; a collaborator carries its registry name and its own `sessionId`, and carries the slot name it was bound to; a collaborator carries its registry name and its own
**no `leader` key** — a collaborator is a named peer, not a primary. Don't infer what you can ask. `sessionId`, and **no `leader` key** — a collaborator is a named peer, not a primary. An **observer**
carries only its own `sessionId`: a pane the daemon could not place as any of the above, authorized
to `READ`/`METRICS`, to `REPLY`/`ASK` on its own pane, and to `SEND` only to a target that resolves
as an observer too — never to a lead, a collaborator, or a spawned member, and never a ticket. It
finds such a target in `fleet_list`'s `panes` array, which for an observer is filtered to exactly
what it may send to and reduced to `sessionId`, `label`, `status`, `role` and `deliverable`.
Don't infer what you can ask.
Only if that call is unavailable, fall back to these — each is one-way, so keep reading until one Only if that call is unavailable, fall back to these — each is one-way, so keep reading until one
fires: the reply charter in your system prompt (*"You are a spawned member in the fires: the reply charter in your system prompt (*"You are a spawned member in the
claude-bridge fleet"*) ⇒ **spawned member**; fleet tools prefixed `mcp__fleet__*` ⇒ **spawned claude-bridge fleet"*) ⇒ **spawned member**; fleet tools prefixed `mcp__fleet__*` ⇒ **spawned
member** (the launcher fixes that mount name; a primary's mount is named by whoever wrote its member** (the launcher fixes that mount name; a primary's mount is named by whoever wrote its
`.mcp.json`, so it varies — and a member spawned before CB-632 still says `mcp__bridge__*`); `ANTHROPIC_BASE_URL` set ⇒ **spawned member** (Claude-model members run `.mcp.json`, so it varies — and a member spawned before CB-632 still says `mcp__bridge__*`); `ANTHROPIC_BASE_URL` set ⇒ **spawned member** (Claude-model members run
on a clean env, so its *absence* proves nothing). None of these separate a worker from an architect — on a clean env, so its *absence* proves nothing). None of these separate a worker from an architect,
only `fleet_whoami` does. **And none of them fires for a collaborator at all**: every signal in the or a worker from an **observer** — an observer is just as unspawned as a collaborator and carries
ladder detects a *spawned* member, while a collaborator is a tab a person opened by hand, so it has none of these signals either, so only `fleet_whoami` tells the two apart. **And none of them fires
no charter, no fixed mount name and a normal environment. A collaborator that cannot call for a collaborator at all**: every signal in the ladder detects a *spawned* member, while a
`fleet_whoami` therefore falls to the line below and acts as a worker. That is the safe direction — collaborator is a tab a person opened by hand, so it has no charter, no fixed mount name and a
it under-privileges, and the refusals are loud — but it means a collaborator has no way to learn normal environment. A collaborator — or an observer — that cannot call `fleet_whoami` therefore falls
what it is except by asking. **Still unsure ⇒ act as a worker**, the most restricted member role. The to the line below and acts as a worker. That is the safe direction — it under-privileges, and the
two mistakes are not symmetric: a primary acting as a worker is refused by the authorization gate — refusals are loud — but it means a collaborator or an observer has no way to learn what it is except
loud and self-correcting — while a member acting as the primary ends its turn with no `fleet_reply`, by asking. **Still unsure ⇒ act as a worker**, the most restricted member role this ladder can name.
The two mistakes are not symmetric: a primary acting as a worker is refused by the authorization gate
— loud and self-correcting — while a member acting as the primary ends its turn with no `fleet_reply`,
and the sender silently receives nothing. Fail toward the recoverable error. and the sender silently receives nothing. Fail toward the recoverable error.
### Invariants — every role, no exceptions ### Invariants — every role, no exceptions
@@ -57,14 +65,22 @@ and the sender silently receives nothing. Fail toward the recoverable error.
2. **The bridge is the only channel.** Text you print in your terminal reaches nobody — the other 2. **The bridge is the only channel.** Text you print in your terminal reaches nobody — the other
side cannot see your screen. An answer that isn't in a `fleet_*` call is silently discarded. side cannot see your screen. An answer that isn't in a `fleet_*` call is silently discarded.
3. **Identity comes from the connection, never an argument.** Workers never pass a target; you 3. **Identity comes from the connection, never an argument.** Workers never pass a target; you
cannot act as another session. Spawn/stop/drain are lead-only; **send is lead, architect, or cannot act as another session. Spawn/stop/drain are lead-only; **send is lead, architect,
collaborator** — and a collaborator may send only to a lead or another collaborator, never to a collaborator, or observer** — and a collaborator may send only to a lead or another collaborator,
spawned member's terminal; reply/ask are only-as-itself — any peer may answer for its own pane, never to a spawned member's terminal, while an observer may send only to another observer pane;
reply/ask are only-as-itself — any peer may answer for its own pane,
and for no other. A call outside your role is refused, not queued. and for no other. A call outside your role is refused, not queued.
4. **Delivery is status-gated: one message per turn.** Don't busy-poll a peer's terminal and don't 4. **Delivery is status-gated: one message per turn.** Don't busy-poll a peer's terminal and don't
re-send because a call looks slow — the bridge delivers when the peer is `idle`, `blocked` or re-send because a call looks slow — the bridge delivers when the peer is `idle`, `blocked` or
`done`. A spawned member must **also** have mounted the bridge MCP: until it has, it is not `done`. A spawned member must **also** have mounted the bridge MCP: until it has, it is not
deliverable, and a send waits on that gate for ~60s and then fails without ever reaching its pane. deliverable, and a send waits on that gate for ~60s and then fails without ever reaching its pane.
**A lead's own pane has a second gate: its input box must be empty.** The multiplexer pastes and
submits in one step, so a delivery that lands while the operator is typing submits their
half-written line. A heartbeat, a ticket nudge and lead-to-lead mail therefore wait until the box
is clear, and a pane the daemon cannot read as a box waits too. Nothing is lost — every one of
those paths retries — but a lead that leaves text sitting in its box receives nothing until it
clears, and the only sign is one warning in `fleetd.out` after 20 held checks in a row. Delivery
to a *member* is not gated this way, because nobody types in a member's pane.
5. **Never move a fleet session, pane or peer except through the bridge.** The bridge owns policy; 5. **Never move a fleet session, pane or peer except through the bridge.** The bridge owns policy;
the multiplexer owns PTYs. Any route that changes fleet state without the bridge's checks the multiplexer owns PTYs. Any route that changes fleet state without the bridge's checks
bypasses every rule above — the `herdr` CLI and its socket are the usual example. bypasses every rule above — the `herdr` CLI and its socket are the usual example.
@@ -173,6 +189,7 @@ you decide.
| Message a **peer lead** on this host | `fleet_send{sessionId: <their terminal>, content}` — `fleet_list` → `leads` reports it. Coordination only, **never** a task | | Message a **peer lead** on this host | `fleet_send{sessionId: <their terminal>, content}` — `fleet_list` → `leads` reports it. Coordination only, **never** a task |
| Message a **peer lead** on another daemon or host | `fleet_send{coordId: <their coord-id>, content}` — needs a `coordinator:` block; your own coord-id is in `fleet_list`. Coordination only, **never** a task | | Message a **peer lead** on another daemon or host | `fleet_send{coordId: <their coord-id>, content}` — needs a `coordinator:` block; your own coord-id is in `fleet_list`. Coordination only, **never** a task |
| Message a **collaborator** on this host | `fleet_send{sessionId: <their terminal>, content}` — `fleet_list` reports a `collaborators` array, and each row carries that peer's `name` and the `sessionId` you send to. It is visible to you, to an architect and to another collaborator, never to a worker. Coordination only, **never** a task | | Message a **collaborator** on this host | `fleet_send{sessionId: <their terminal>, content}` — `fleet_list` reports a `collaborators` array, and each row carries that peer's `name` and the `sessionId` you send to. It is visible to you, to an architect and to another collaborator, never to a worker. Coordination only, **never** a task |
| Message an **unconfigured pane** — a tab a person opened by hand | `fleet_send{sessionId: <their terminal>, content}` — it needs **no** `fleet.collaborators` entry and no restart, because a pane becomes deliverable the moment its agent connects the bridge MCP. `fleet_list`'s `panes` array reports every such pane with its label and the terminal id to send to — the full row for you, an architect or a collaborator; filtered and reduced for an observer. **`ListAgents` still never lists these**, and joining `herdr tab list` to `GET /agents` on `tab_id` stays the read-only fallback if the array is missing. Such a pane resolves as an `observer`: it can answer you with `fleet_reply`, and it can `fleet_send` to another observer pane, but never to you. Coordination only, **never** a task |
| Answer a peer lead that messaged you | `fleet_send{coordId}` — or `{sessionId}` if they are on this host. **Not** `fleet_reply`: it has no peer route and the publish is refused | | Answer a peer lead that messaged you | `fleet_send{coordId}` — or `{sessionId}` if they are on this host. **Not** `fleet_reply`: it has no peer route and the publish is refused |
| Read your own held lead-to-lead mail (no ack) | `fleet_poll{coordId: <your own coord-id, from fleet_list's coordinator.selfId>}` — primary-only; never acks, so `fleet_list`'s `held[]` still shows it after. `fleet_list`'s `held[]` gives only a truncated preview — this is the only way to read the full body | | Read your own held lead-to-lead mail (no ack) | `fleet_poll{coordId: <your own coord-id, from fleet_list's coordinator.selfId>}` — primary-only; never acks, so `fleet_list`'s `held[]` still shows it after. `fleet_list`'s `held[]` gives only a truncated preview — this is the only way to read the full body |
| Collect a held reply | `fleet_poll{target}` · then `fleet_ack{target, msgId}` | | Collect a held reply | `fleet_poll{target}` · then `fleet_ack{target, msgId}` |
@@ -497,3 +514,33 @@ to replace them.
Prefer the unnamed lambda parameter `_` for required-but-unused params; a non-public Prefer the unnamed lambda parameter `_` for required-but-unused params; a non-public
`static void main(String[])` is valid (JEP 512) and boots via `java -jar`. `static void main(String[])` is valid (JEP 512) and boots via `java -jar`.
## Code quality — five rules, and what each already cost (enforced)
Measured at `7f0c4a8`: 124 main files, 36,278 lines, of which **17,850 are code** — 44% is comment,
and only **two** files exceed 1000 *code* lines. Encapsulation and inheritance are already sound (0
public mutable fields; 12 `extends`, 8 of them exceptions; 120 records). So there is **no Clean Code
section, no SOLID list and no pattern catalogue** here: two architect reviews rejected those
independently as text that would change no behaviour. These five rules are the whole standard.
1. **A comment states the current contract or a current maintainer constraint — nothing else.** No
tickets, history, dates, measurements or review rationale; those go in the commit message or the
MR description. Source code only — this rule never applies to Markdown.
2. **A javadoc block stops at 30 lines.** Longer means it is a design argument, so it moves to
`docs/<subject>.md` and is linked in one line. The longest here is 235 lines
(`config/ConfigRef.java`) and the knowledge in it is load-bearing: **move it, never delete it.**
This project has **no ADR** — subject pages under `docs/` are the destination.
3. **A comment in main source never names a test class.** There are 44 such names in 76 places and
**2 are already dead**, because a name inside `{@code}` is invisible to the compiler and rots in
silence. Say what the code guarantees; the test is found by looking.
4. **Never relieve a testing problem by reshaping production code.** `Fleetd` carries 54 static
factories, `MessageService` carries 7 `volatile` race hooks, and 8 tests assert on main source as
*text*. Make the part injectable instead. `FleetdAssembly.assembleAndStart` is 452 lines and may
not grow; no new source-text test may be added.
5. **No new package cycle, and no widening of a recorded one.** Five pairs are frozen as an exact
edge baseline in `PackageCyclesTest` — four of them involve `msg`.
Rules 1, 2, 3 and 5 have build checks, and Gitea CI runs them on every PR, so they bind members too.
**Rule 4's judgement half has no mechanism**: a cap stops a count growing, but no test tells a good
decomposition from a bad one. That half is a review obligation, and saying so is deliberate — a rule
dressed as a gate it does not have is worse than an honest review item.
+6
View File
@@ -182,6 +182,12 @@ Two consequences a lead feels directly:
is fine; the message simply waits, and then restarts the member when it next goes idle. is fine; the message simply waits, and then restarts the member when it next goes idle.
- **A spawned member is not deliverable until it has mounted the MCP.** Until then a send waits on - **A spawned member is not deliverable until it has mounted the MCP.** Until then a send waits on
that gate for about 60 seconds and then fails without ever reaching the pane. that gate for about 60 seconds and then fails without ever reaching the pane.
- **The same gate is what makes an unconfigured pane deliverable.** `contextExtractor` runs on
every MCP request, `initialize` included, and `markTrackedCallerPresent` enrols a spawned member
*or* an observer into `MemberPresence`; `deliverableTo` then tests presence before the lead and
collaborator maps. So mounting the server is the enrolment, and a tab a person opened by hand can
be sent to with no config and no restart. It answers with `fleet_reply` — it cannot `fleet_send`,
because `Authz` keeps `SEND` to a primary, an architect or a collaborator.
`UNKNOWN` is deliberately neither injectable nor a pickup. A pane whose status cannot be read is `UNKNOWN` is deliberately neither injectable nor a pickup. A pane whose status cannot be read is
not a pane that is safe to write to — see fleetd #176 for what happens when a gate treats an not a pane that is safe to write to — see fleetd #176 for what happens when a gate treats an
+22 -17
View File
@@ -99,17 +99,17 @@ bind:
# contextHighNudge: false # contextHighNudge: false
# Lead rollover (fleetd #480): replace a lead session that has decided it is ready to be replaced, # Lead rollover (fleetd #480): replace a lead session that has decided it is ready to be replaced,
# without an operator doing it by hand. A lead writes a handover file, then asks fleetd to clear its # without an operator doing it by hand. A lead writes a handover file, then asks fleetd to end its
# own pane and bootstrap a fresh session against that file. # own pane, launch a fresh one, and bootstrap that fresh session against the handover file.
# #
# Opt-in on purpose — it clears the lead's own pane on request, so upgrading the daemon must never # Opt-in on purpose — it tears down the lead's own pane on request, so upgrading the daemon must
# acquire that ability for you. Absent block = feature off, and nothing is constructed at all. Even # never acquire that ability for you. Absent block = feature off, and nothing is constructed at all.
# once present, nothing but an explicit confirm() call — one that passes every check — can ever # Even once present, nothing but an explicit confirm() call — one that passes every check — can ever
# cause a /clear: there is no recurring timer, heartbeat or scheduler anywhere in this feature that # tear a pane down: there is no recurring timer, heartbeat or scheduler anywhere in this feature that
# fires one on its own initiative. confirm() itself is called FROM the calling lead's own turn, so # fires one on its own initiative. confirm() itself is called FROM the calling lead's own turn, so it
# it cannot clear the pane inline (that pane is still WORKING); instead it schedules a one-shot # cannot act on the pane inline (that pane is still WORKING); instead it schedules a one-shot
# continuation that waits for the SAME confirm() call's turn to end, then does the actual work. See # continuation that waits for the SAME confirm() call's turn to end, then does the actual work. See
# dev.ltms.fleet.lead.LeadRollover's class javadoc for the exact order (fleetd #480 correction). # dev.ltms.fleet.lead.LeadRollover's class javadoc for the exact order.
# #
# handoverPath: REQUIRED when this block is present — where the handover file a fresh lead session # handoverPath: REQUIRED when this block is present — where the handover file a fresh lead session
# reads must live. No default (an operator-specific path); a present block with no # reads must live. No default (an operator-specific path); a present block with no
@@ -118,25 +118,30 @@ bind:
# working directory when that lead has none configured) — never against whatever # working directory when that lead has none configured) — never against whatever
# directory the daemon process happens to have been started in. An absolute path is # directory the daemon process happens to have been started in. An absolute path is
# used unchanged. Prefer an absolute path if the daemon and the lead's pane might not # used unchanged. Prefer an absolute path if the daemon and the lead's pane might not
# share a working directory (fleetd #480 follow-up). # share a working directory.
# requireOperatorConfirm: true # default true — confirm() refuses unless the caller also passes # requireOperatorConfirm: true # default true — confirm() refuses unless the caller also passes
# # operatorConfirmed: true # # operatorConfirmed: true
# maxDocAgeSeconds: 3600 # default 3600 — refuse a handover file older than this # maxDocAgeSeconds: 3600 # default 3600 — refuse a handover file older than this
# turnSettleSeconds: 20 # default 20 — how long the deferred roll waits for the CALLING # turnSettleSeconds: 20 # default 20 — how long the deferred roll waits for the CALLING
# # lead's own turn to end (its pane to report injectable again) # # lead's own turn to end (its pane to report injectable again)
# # before sending /clear at all. If this elapses, /clear is NEVER # # before tearing the old pane down at all. If this elapses, nothing
# # sent — a lead that never goes idle is still doing real work. # # is torn down — a lead that never goes idle is still doing real
# clearSettleSeconds: 20 # default 20 — how long to wait for the pane to become injectable # # work.
# # again AFTER /clear before giving up (never sends bootstrapText # relaunchReadySeconds: 45 # default 45 — bounds two later waits, after the old pane is gone
# # if this elapses). A separate, second wait from turnSettleSeconds. # # and a fresh one has been launched: first, for the fresh pane to
# # reach a real turn boundary (never sends bootstrapText if THIS one
# # elapses); second, for the new terminal to be recognised as this
# # lead (bootstrapText is sent either way once the first wait
# # passes). A separate, later pair of waits from turnSettleSeconds.
# bootstrapText: "..." # default names the RESOLVED (absolute) handoverPath — sent to # bootstrapText: "..." # default names the RESOLVED (absolute) handoverPath — sent to
# # the lead once its pane settles after /clear # # the freshly relaunched lead's pane once it reaches a real turn
# # boundary
# leadRollover: # leadRollover:
# handoverPath: /path/to/handover.md # handoverPath: /path/to/handover.md
# requireOperatorConfirm: true # requireOperatorConfirm: true
# maxDocAgeSeconds: 3600 # maxDocAgeSeconds: 3600
# turnSettleSeconds: 20 # turnSettleSeconds: 20
# clearSettleSeconds: 20 # relaunchReadySeconds: 45
# bootstrapText: "Fresh lead session: read the handover file and carry on." # bootstrapText: "Fresh lead session: read the handover file and carry on."
# Fleet health detection is dormant unless enabled (CB-573). It reads one whole-fleet agent list # Fleet health detection is dormant unless enabled (CB-573). It reads one whole-fleet agent list
+52 -33
View File
@@ -226,16 +226,15 @@ public final class Fleetd {
* is the same way — a person's own tab, matched to a configured name, never spawned. * is the same way — a person's own tab, matched to a configured name, never spawned.
* *
* <p>Neither a lead nor a collaborator is ever enrolled in {@link MemberPresence} — {@code * <p>Neither a lead nor a collaborator is ever enrolled in {@link MemberPresence} — {@code
* FleetMcp} marks presence for every spawned member (worker and architect), deliberately, since * FleetMcp} marks presence only for a worker, an architect, or the unconfigured-pane floor,
* that map doubles as the member roster's availability signal and a lead or collaborator counted * never for a lead or a collaborator. So without the second and third disjuncts a lead or
* there would show up as an available member. So without the second and third disjuncts a lead or
* collaborator is permanently un-deliverable: every send to one sat on the gate for * collaborator is permanently un-deliverable: every send to one sat on the gate for
* {@code READINESS_GRACE_POLLS} (~60s) and then failed having never been typed into the pane. * {@code READINESS_GRACE_POLLS} (~60s) and then failed having never been typed into the pane.
* *
* <p>Both sets are read through their supplier on each call rather than snapshotted, so a lead or * <p>Both sets are read through their supplier on each call rather than snapshotted, so a lead or
* collaborator discovered by {@code leadScan} after startup becomes deliverable without a restart. * collaborator discovered by {@code leadScan} after startup becomes deliverable without a restart.
*/ */
static Predicate<String> deliverableTo(MemberPresence presence, Supplier<Map<String, String>> leads, public static Predicate<String> deliverableTo(MemberPresence presence, Supplier<Map<String, String>> leads,
Supplier<Map<String, String>> collaborators) { Supplier<Map<String, String>> collaborators) {
return target -> presence.isPresent(target) || leads.get().containsKey(target) return target -> presence.isPresent(target) || leads.get().containsKey(target)
|| collaborators.get().containsKey(target); || collaborators.get().containsKey(target);
@@ -337,22 +336,6 @@ public final class Fleetd {
}, reasonByCredential::get); }, reasonByCredential::get);
} }
/**
* fleetd #415 (review follow-up): package-private factory for the CB-578 stage A {@code
* exhaustedPattern} startup coverage line, paired explicitly with {@link
* CompletionResolver.UnsetMeaning#OFF} — {@code exhaustedPattern} has no fallback, so a
* profile with none configured really does have the classification off.
*
* <p>Extracted out of {@code main} for the same reason {@link #capacitySource} and {@link
* #worktreeBranchLookup} were: {@code coverage()}'s own tests ({@code CompletionResolverTest})
* prove it words {@code OFF} and {@link CompletionResolver.UnsetMeaning#BUILT_IN_DEFAULT}
* correctly when a test supplies the meaning itself — they cannot prove {@code main} pairs the
* right meaning with the right key, which is the actual fleetd #415 defect. <b>Measured:</b>
* swapping the {@code UnsetMeaning} arguments between this method and {@link
* #errorPatternCoverageLine} — recreating #415's defect with the two keys exchanged — compiled
* with 0 errors and left all 1506 existing tests green before {@code
* FleetdPatternCoverageLineTest} was added to catch exactly that swap.
*/
/** /**
* fleetd #446 follow-up: the criterion-2 WARNING text — "name the fix, not just the fact" — * fleetd #446 follow-up: the criterion-2 WARNING text — "name the fix, not just the fact" —
* for a profile whose {@code model:} is configured. Extracted out of the {@code * for a profile whose {@code model:} is configured. Extracted out of the {@code
@@ -387,19 +370,22 @@ public final class Fleetd {
+ "s quarantine above is the only thing keeping new spawns off it for now"; + "s quarantine above is the only thing keeping new spawns off it for now";
} }
/**
* Package-private factory for the {@code exhaustedPattern} startup coverage line, paired
* explicitly with {@link CompletionResolver.UnsetMeaning#OFF} — {@code exhaustedPattern} has
* no fallback, so a profile with none configured really does have the classification off.
*/
static String exhaustedPatternCoverageLine(Set<String> allProfiles, Set<String> configuredProfiles) { static String exhaustedPatternCoverageLine(Set<String> allProfiles, Set<String> configuredProfiles) {
return CompletionResolver.coverage("exhaustedPattern", CompletionResolver.UnsetMeaning.OFF, return CompletionResolver.coverage("exhaustedPattern", CompletionResolver.UnsetMeaning.OFF,
allProfiles, configuredProfiles); allProfiles, configuredProfiles);
} }
/** /**
* fleetd #415 (review follow-up): the {@code errorPattern} counterpart of {@link * The {@code errorPattern} counterpart of {@link #exhaustedPatternCoverageLine}, paired
* #exhaustedPatternCoverageLine}, paired explicitly with {@link * explicitly with {@link CompletionResolver.UnsetMeaning#BUILT_IN_DEFAULT} — an unset
* CompletionResolver.UnsetMeaning#BUILT_IN_DEFAULT} — an unset {@code errorPattern} still runs * {@code errorPattern} still runs backend-error classification against
* backend-error classification against {@code CompletionResolver}'s built-in {@code * {@code CompletionResolver}'s built-in {@code BACKEND_ERROR} pattern, so the empty case is
* BACKEND_ERROR} pattern, so the empty case is not "off". See {@link * not "off".
* #exhaustedPatternCoverageLine}'s javadoc for the measured swap mutation this pairing guards
* against.
*/ */
static String errorPatternCoverageLine(Set<String> allProfiles, Set<String> configuredProfiles) { static String errorPatternCoverageLine(Set<String> allProfiles, Set<String> configuredProfiles) {
return CompletionResolver.coverage("errorPattern", CompletionResolver.UnsetMeaning.BUILT_IN_DEFAULT, return CompletionResolver.coverage("errorPattern", CompletionResolver.UnsetMeaning.BUILT_IN_DEFAULT,
@@ -735,8 +721,8 @@ public final class Fleetd {
* {@code CompletionResolver} constructor call — provably untested wiring, the whole reason * {@code CompletionResolver} constructor call — provably untested wiring, the whole reason
* fleetd #248 exists: dropping that one argument (passing {@code _ -> null} instead) compiled * fleetd #248 exists: dropping that one argument (passing {@code _ -> null} instead) compiled
* clean and left every test green. Extracted here, {@code main} now calls this factory instead * clean and left every test green. Extracted here, {@code main} now calls this factory instead
* of building the lambda inline, and a source assertion on that call site * of building the lambda inline, so the argument reaching the {@code CompletionResolver}
* ({@code FleetdCompletionResolverWiringTest}) proves the argument is still actually passed. * constructor is a named, directly testable call rather than an inline lambda.
* *
* <p>Takes the roster as a plain {@link Supplier} — not a {@link SessionManager} — so this is * <p>Takes the roster as a plain {@link Supplier} — not a {@link SessionManager} — so this is
* directly testable with a hand-built session list; no real {@code SessionManager} (launcher, * directly testable with a hand-built session list; no real {@code SessionManager} (launcher,
@@ -905,6 +891,32 @@ public final class Fleetd {
throw (T) t; throw (T) t;
} }
/**
* The {@link PrimaryRegistry} lookup for "which terminal currently hosts the lead named
* {@code name}" — the inverse of {@code liveLeadTerminals} (terminal id → lead name), read live
* on every call so a lead discovered, rolled, or lost since the last call is reflected without
* a restart. Returns {@code null} when no currently recognised lead carries that name — a name
* that is not a lead at all (an architect slot, a collaborator), or a lead whose tab the scan
* cannot currently place (just rolled, off-host, non-herdr).
*
* @param liveLeadTerminals terminal id → lead name for every CURRENTLY recognised lead, normally
* the same {@code leads} supplier {@code main} already builds for
* {@code HerdrRouter}/{@link #leadSeatLookup}
*/
static Function<String, String> currentTerminalForName(Supplier<Map<String, String>> liveLeadTerminals) {
return name -> {
if (name == null) {
return null;
}
for (var entry : liveLeadTerminals.get().entrySet()) {
if (name.equals(entry.getValue())) {
return entry.getKey();
}
}
return null;
};
}
/** /**
* fleetd #480: construct the {@link LeadRollover} executor only when {@code leadRollover:} is * fleetd #480: construct the {@link LeadRollover} executor only when {@code leadRollover:} is
* present at startup — the same presence gate {@code leadHeartbeat:} uses just above this * present at startup — the same presence gate {@code leadHeartbeat:} uses just above this
@@ -942,21 +954,27 @@ public final class Fleetd {
* cfg.leadHeartbeat()} * cfg.leadHeartbeat()}
* @param leadAgents the {@link AgentControl} instance that reaches the LEAD's pane (not * @param leadAgents the {@link AgentControl} instance that reaches the LEAD's pane (not
* {@code memberAgents}), normally {@code router.leadAgents()} * {@code memberAgents}), normally {@code router.leadAgents()}
* @param leadSpaces the {@link WorkspaceControl} instance that reaches the LEAD's
* workspace, normally {@code router.leadSpaces()} — used to tear down
* a rolled lead's old pane and confirm it is gone
* @param launcher starts the fresh lead a roll relaunches once the old one is gone
* @param config the live {@link ConfigRef}, captured only inside the returned * @param config the live {@link ConfigRef}, captured only inside the returned
* supplier and the workspace lookup — never dereferenced here * supplier and the two lookups below — never dereferenced here
* @param liveLeadTerminals terminal id → lead NAME for every CURRENTLY recognised lead, normally * @param liveLeadTerminals terminal id → lead NAME for every CURRENTLY recognised lead, normally
* the same {@code leads} supplier {@code main} already builds for * the same {@code leads} supplier {@code main} already builds for
* {@code HerdrRouter}/{@link #leadSeatLookup} — never a value snapshot * {@code HerdrRouter}/{@link #leadSeatLookup} — never a value snapshot
* @return a constructed {@link LeadRollover}, or {@code null} when {@code leadRollover:} is * @return a constructed {@link LeadRollover}, or {@code null} when {@code leadRollover:} is
* absent from the startup config * absent from the startup config
*/ */
static LeadRollover leadRollover(FleetConfig cfg, AgentControl leadAgents, ConfigRef config, static LeadRollover leadRollover(FleetConfig cfg, AgentControl leadAgents,
WorkspaceControl leadSpaces, LeadLauncher launcher, ConfigRef config,
Supplier<Map<String, String>> liveLeadTerminals) { Supplier<Map<String, String>> liveLeadTerminals) {
if (cfg.leadRollover() == null) { if (cfg.leadRollover() == null) {
return null; return null;
} }
Function<String, String> leadNameForTerminal = terminal -> liveLeadTerminals.get().get(terminal);
Function<String, String> leadWorkspace = terminal -> { Function<String, String> leadWorkspace = terminal -> {
String leadName = liveLeadTerminals.get().get(terminal); String leadName = leadNameForTerminal.apply(terminal);
if (leadName == null) { if (leadName == null) {
return null; return null;
} }
@@ -964,7 +982,8 @@ public final class Fleetd {
FleetConfig.Leader leader = fleet == null ? null : fleet.leaders().get(leadName); FleetConfig.Leader leader = fleet == null ? null : fleet.leaders().get(leadName);
return leader == null ? null : leader.cwd(); return leader == null ? null : leader.cwd();
}; };
return new LeadRollover(leadAgents, () -> config.get().leadRollover(), leadWorkspace); return new LeadRollover(leadAgents, leadSpaces, launcher, () -> config.get().leadRollover(),
leadWorkspace, leadNameForTerminal, liveLeadTerminals);
} }
/** /**
@@ -254,18 +254,28 @@ final class FleetdAssembly {
if (leadTerminals.size() > 1) { if (leadTerminals.size() > 1) {
log.info("leads: {} panes recognised {}", leadTerminals.size(), leadTerminals.values()); log.info("leads: {} panes recognised {}", leadTerminals.size(), leadTerminals.values());
} }
// CB-531/CB-579: discover leads by the tab labels the operator writes, one scanner per // Discover leads by their tab labels, one scanner per configured lead's own space. fleetd
// configured lead's own exact `tab:` label. fleetd #669: the same scan also recognises a // #669: the same scan also recognises a configured collaborator's tab, so one herdr pass
// configured collaborator's tab, so one herdr pass answers both. // answers both.
final Supplier<Map<String, String>> leads; final Supplier<Map<String, String>> leads;
final Supplier<Map<String, String>> collaboratorTerminals; final Supplier<Map<String, String>> collaboratorTerminals;
var leaders = cfg.fleet().leaders(); var leaders = cfg.fleet().leaders();
var collaboratorsConfig = cfg.fleet().collaborators(); var collaboratorsConfig = cfg.fleet().collaborators();
if (!leaders.isEmpty() || !collaboratorsConfig.isEmpty()) { if (!leaders.isEmpty() || !collaboratorsConfig.isEmpty()) {
Map<String, String> tabToName = new LinkedHashMap<>(); Map<String, Map<String, String>> leadLabelsBySpace = new LinkedHashMap<>();
Map<String, String> spaceByLeadName = new LinkedHashMap<>();
leaders.forEach((name, leader) -> { leaders.forEach((name, leader) -> {
if (leader != null && leader.tab() != null && !leader.tab().isBlank()) { if (leader == null) {
tabToName.put(leader.tab(), name); return;
}
spaceByLeadName.put(name, leader.workspace());
Map<String, String> labelsHere = leadLabelsBySpace
.computeIfAbsent(leader.workspace(), k -> new LinkedHashMap<>());
leader.acceptedLabels().forEach(label -> labelsHere.put(label, name));
if (leader.tab() != null && !leader.tab().isBlank()) {
log.warn("lead '{}' (fleet.leaders.{}) still configures tab: \"{}\" — deprecated, "
+ "the lead tab label is now fixed to '{}'",
name, name, leader.tab(), FleetConfig.Leader.LEAD_TAB_LABEL);
} }
}); });
Map<String, String> collaboratorTabToName = new LinkedHashMap<>(); Map<String, String> collaboratorTabToName = new LinkedHashMap<>();
@@ -283,13 +293,13 @@ final class FleetdAssembly {
? 10 ? 10
: leaders.values().iterator().next().scanIntervalSeconds(); : leaders.values().iterator().next().scanIntervalSeconds();
// This must use the lead daemon: scanning member tabs would demote the lead to a worker. // This must use the lead daemon: scanning member tabs would demote the lead to a worker.
LeadTabScanner scanner = new LeadTabScanner(herdr, tabToName, collaboratorTabToName, Set.of(), LeadTabScanner scanner = new LeadTabScanner(herdr, leadLabelsBySpace, collaboratorTabToName,
TimeUnit.SECONDS.toNanos(scanIntervalSeconds), ports.nanoClock()); Set.of(), TimeUnit.SECONDS.toNanos(scanIntervalSeconds), ports.nanoClock());
leads = scanner; leads = scanner;
collaboratorTerminals = scanner::collaborators; collaboratorTerminals = scanner::collaborators;
log.info("lead/collaborator scan: tabs {} host a lead, tabs {} host a collaborator " log.info("lead/collaborator scan: space per lead {}, tabs {} host a collaborator "
+ "(rescan every {}s, shared fleet space)", + "(rescan every {}s)",
tabToName.keySet(), collaboratorTabToName.keySet(), scanIntervalSeconds); spaceByLeadName, collaboratorTabToName.keySet(), scanIntervalSeconds);
} else { } else {
leads = () -> leadTerminals; leads = () -> leadTerminals;
collaboratorTerminals = Map::of; collaboratorTerminals = Map::of;
@@ -297,11 +307,15 @@ final class FleetdAssembly {
leadsRef.set(leads); leadsRef.set(leads);
collaboratorTerminalsRef.set(collaboratorTerminals); collaboratorTerminalsRef.set(collaboratorTerminals);
// Constructed unconditionally — it is cheap and side-effect free — so a LeadRollover built
// below can relaunch a lead even on a boot where herdr was down for the ensureLeads() call.
LeadLauncher leadLauncher = new LeadLauncher(router.leadAgents(), router.leadSpaces(), cfg);
// CB-558: start any declared lead that is not already running. After the scanner is built, // CB-558: start any declared lead that is not already running. After the scanner is built,
// and only when herdr answered — the launcher's whole safety property is that it can count // and only when herdr answered — the launcher's whole safety property is that it can count
// live leads first, and must never guess and risk a second orchestrator. // live leads first, and must never guess and risk a second orchestrator.
if (herdrUp && !leaders.isEmpty()) { if (herdrUp && !leaders.isEmpty()) {
int launched = new LeadLauncher(router.leadAgents(), router.leadSpaces(), cfg).ensureLeads(); int launched = leadLauncher.ensureLeads();
if (launched > 0) { if (launched > 0) {
log.info("lead auto-launch: {} lead(s) started", launched); log.info("lead auto-launch: {} lead(s) started", launched);
} }
@@ -389,7 +403,8 @@ final class FleetdAssembly {
ports.leadMailboxOpener()); ports.leadMailboxOpener());
// CB-307: learn the primary's terminal from orchestration tool calls (or pin from config). // CB-307: learn the primary's terminal from orchestration tool calls (or pin from config).
String pinnedPrimaryTerminal = cfg.primary() != null ? cfg.primary().terminal() : null; String pinnedPrimaryTerminal = cfg.primary() != null ? cfg.primary().terminal() : null;
PrimaryRegistry primaryRegistry = new PrimaryRegistry(pinnedPrimaryTerminal); PrimaryRegistry primaryRegistry = new PrimaryRegistry(pinnedPrimaryTerminal,
Fleetd.currentTerminalForName(leads));
// CB-532: `primary.terminal` is superseded and no longer needed for either of its jobs. // CB-532: `primary.terminal` is superseded and no longer needed for either of its jobs.
if (pinnedPrimaryTerminal != null && !pinnedPrimaryTerminal.isBlank()) { if (pinnedPrimaryTerminal != null && !pinnedPrimaryTerminal.isBlank()) {
log.warn("primary.terminal is DEPRECATED (CB-532) and can be deleted: identity now comes " log.warn("primary.terminal is DEPRECATED (CB-532) and can be deleted: identity now comes "
@@ -440,7 +455,8 @@ final class FleetdAssembly {
heartbeatScheduler.shutdownNow(); heartbeatScheduler.shutdownNow();
} }
// fleetd #480: lead rollover. Opt-in; absent `leadRollover:` this is never constructed. // fleetd #480: lead rollover. Opt-in; absent `leadRollover:` this is never constructed.
LeadRollover leadRollover = Fleetd.leadRollover(cfg, router.leadAgents(), config, leads); LeadRollover leadRollover = Fleetd.leadRollover(cfg, router.leadAgents(), router.leadSpaces(),
leadLauncher, config, leads);
MessageService messages = new MessageService(router, injector, rendezvous, replyInbox, MessageService messages = new MessageService(router, injector, rendezvous, replyInbox,
pushLoop, metrics); pushLoop, metrics);
@@ -70,33 +70,58 @@ public final class Authz {
*/ */
public static final Predicate<String> NO_KNOWN_LEAD_OR_COLLABORATOR = target -> false; public static final Predicate<String> NO_KNOWN_LEAD_OR_COLLABORATOR = target -> false;
/**
* The fail-closed classifier for an observer's {@code SEND}: answers no for every target, so
* the grant is refused unless a caller supplies a real one. {@code
* CallerResolver#sendableObserverTarget()} is the real one, read from the same maps {@code
* CallerResolver#resolve} consults, so a target that classifier calls known is one {@code
* resolve} would actually resolve as {@link Role#OBSERVER}.
*/
public static final Predicate<String> NO_KNOWN_OBSERVER_TARGET = target -> false;
/** /**
* Convenience form for a caller with no classifier to supply. Fails closed: a collaborator's * Convenience form for a caller with no classifier to supply. Fails closed: a collaborator's
* {@code SEND} is refused, as if no terminal were a configured lead or collaborator — the * or an observer's {@code SEND} is refused, as if no terminal were a configured lead,
* same decision {@link #NO_KNOWN_LEAD_OR_COLLABORATOR} gives explicitly. Every other action's * collaborator, or observer target — the same decision {@link #NO_KNOWN_LEAD_OR_COLLABORATOR}
* result is identical to the four-argument form's, since none of them consult the classifier. * and {@link #NO_KNOWN_OBSERVER_TARGET} give explicitly. Every other action's result is
* identical to the five-argument form's, since none of them consult either classifier.
* *
* <p>Its default classifier denies every collaborator, so a caller enforcing authorization * <p>Its default classifiers deny every collaborator and every observer, so a caller
* must use the four-argument form instead. * enforcing authorization must use the five-argument form instead.
*/ */
public static boolean permits(Principal caller, Action action, String targetSession) { public static boolean permits(Principal caller, Action action, String targetSession) {
return permits(caller, action, targetSession, NO_KNOWN_LEAD_OR_COLLABORATOR); return permits(caller, action, targetSession, NO_KNOWN_LEAD_OR_COLLABORATOR, NO_KNOWN_OBSERVER_TARGET);
}
/**
* As {@link #permits(Principal, Action, String)}, with a real classifier for a collaborator's
* {@code SEND}. An observer's {@code SEND} still fails closed ({@link #NO_KNOWN_OBSERVER_TARGET}) —
* a caller enforcing both grants must use the five-argument form.
*/
public static boolean permits(Principal caller, Action action, String targetSession,
Predicate<String> knownLeadOrCollaborator) {
return permits(caller, action, targetSession, knownLeadOrCollaborator, NO_KNOWN_OBSERVER_TARGET);
} }
/** /**
* Whether {@code caller} may perform {@code action} against {@code targetSession}. * Whether {@code caller} may perform {@code action} against {@code targetSession}.
* *
* @param targetSession the session id in the request path; only consulted for the * @param targetSession the session id in the request path; only consulted for the
* worker-scoped actions ({@code REPLY}, {@code ASK}) and for a * worker-scoped actions ({@code REPLY}, {@code ASK}), for a
* collaborator's {@code SEND}, ignored otherwise, may be * collaborator's {@code SEND}, and for an observer's
* {@code null} * {@code SEND}, ignored otherwise, may be {@code null}
* @param knownLeadOrCollaborator whether a terminal is a configured lead or collaborator — * @param knownLeadOrCollaborator whether a terminal is a configured lead or collaborator —
* consulted only for a collaborator's {@code SEND}, to confine * consulted only for a collaborator's {@code SEND}, to confine
* it to another named peer and never a spawned member's * it to another named peer and never a spawned member's
* terminal * terminal
* @param knownObserverTarget whether a terminal is one this daemon would itself resolve as
* {@link Role#OBSERVER} — consulted only for an observer's
* {@code SEND}, to confine it to another observer pane and never
* a lead, a collaborator, or a spawned member
*/ */
public static boolean permits(Principal caller, Action action, String targetSession, public static boolean permits(Principal caller, Action action, String targetSession,
Predicate<String> knownLeadOrCollaborator) { Predicate<String> knownLeadOrCollaborator,
Predicate<String> knownObserverTarget) {
if (caller == null || caller.isAnonymous()) { if (caller == null || caller.isAnonymous()) {
return false; // authenticated as nothing ⇒ authorized for nothing return false; // authenticated as nothing ⇒ authorized for nothing
} }
@@ -107,13 +132,15 @@ public final class Authz {
// would be a worker escalating into the orchestrator role. // would be a worker escalating into the orchestrator role.
case SPAWN, STOP, DRAIN, HANDOVER -> caller.isPrimary(); case SPAWN, STOP, DRAIN, HANDOVER -> caller.isPrimary();
// Delivering a turn to a local session is open to the primary, the architect, and a // Delivering a turn to a local session is open to the primary and the architect
// collaborator whose target is itself a configured lead or collaborator: the architect // unconditionally. A collaborator may reach only a target that is itself a configured
// delegates to workers (that is the role's point); a collaborator may reach only // lead or collaborator, never a spawned member's terminal. An observer may reach only
// another named peer, never a spawned member's terminal. A worker is excluded — // a target that would itself resolve as an observer, never a lead, a collaborator, or
// sending would be it escalating. // a spawned member. A worker is excluded from every case — sending would be it
// escalating into the orchestrator role.
case SEND -> caller.isPrimary() || caller.isArchitect() case SEND -> caller.isPrimary() || caller.isArchitect()
|| (caller.isCollaborator() && knownLeadOrCollaborator.test(targetSession)); || (caller.isCollaborator() && knownLeadOrCollaborator.test(targetSession))
|| (caller.isObserver() && knownObserverTarget.test(targetSession));
// Resolving a worker's blocked question is part of delegating to it, open to the same // Resolving a worker's blocked question is part of delegating to it, open to the same
// two roles that may stand up that delegation in the first place. Not a collaborator: // two roles that may stand up that delegation in the first place. Not a collaborator:
@@ -138,13 +165,15 @@ public final class Authz {
// fleet_whoami — and carries no secrets: no ticket reply, no pending question, and no // fleet_whoami — and carries no secrets: no ticket reply, no pending question, and no
// other session's turn state. Those live under TASK_READ. METRICS is the separate // other session's turn state. Those live under TASK_READ. METRICS is the separate
// Prometheus scrape. Both are open to every authenticated role, including a // Prometheus scrape. Both are open to every authenticated role, including a
// collaborator. // collaborator and the unconfigured-pane floor.
case READ, METRICS -> caller.isPrimary() || caller.isWorker() || caller.isArchitect() case READ, METRICS -> caller.isPrimary() || caller.isWorker() || caller.isArchitect()
|| caller.isCollaborator(); || caller.isCollaborator() || caller.isObserver();
// Ticket polling and session status, open to every role READ is open to except a // Ticket polling and session status, open to every role READ is open to except a
// collaborator: ticket ids are a sequential counter with no owner check, so a holder // collaborator or an observer. MessageService compares a ticket's creator to the
// could walk every ticket and read another session's delegation reply. // caller on every read as well, so dropping this gate would not expose another
// session's reply — it would move the refusal later and widen what a caller that
// never orchestrates can probe.
case TASK_READ -> caller.isPrimary() || caller.isWorker() || caller.isArchitect(); case TASK_READ -> caller.isPrimary() || caller.isWorker() || caller.isArchitect();
// fleetd #421: reading held lead-to-lead mail is the primary's alone. An architect // fleetd #421: reading held lead-to-lead mail is the primary's alone. An architect
@@ -40,7 +40,7 @@ import java.util.function.Supplier;
* the case the previous step does not catch: a binding with no live spawned-member session.</li> * the case the previous step does not catch: a binding with no live spawned-member session.</li>
* <li>A loopback peer PID that maps to an operator-labelled collaborator tab ⇒ * <li>A loopback peer PID that maps to an operator-labelled collaborator tab ⇒
* {@link Role#COLLABORATOR}, carrying that collaborator's name.</li> * {@link Role#COLLABORATOR}, carrying that collaborator's name.</li>
* <li>A loopback peer PID that maps to any other herdr pane ⇒ {@link Role#WORKER}. This is * <li>A loopback peer PID that maps to any other herdr pane ⇒ {@link Role#OBSERVER}. This is
* unforgeable (the OS reports the PID, herdr owns the PID→pane map) and is honoured * unforgeable (the OS reports the PID, herdr owns the PID→pane map) and is honoured
* regardless of auth mode, so enabling auth never breaks the fleet.</li> * regardless of auth mode, so enabling auth never breaks the fleet.</li>
* <li>Otherwise, under {@code token} mode, a valid bearer token ⇒ {@link Role#PRIMARY}.</li> * <li>Otherwise, under {@code token} mode, a valid bearer token ⇒ {@link Role#PRIMARY}.</li>
@@ -270,6 +270,31 @@ public final class CallerResolver {
|| collaboratorTerminals.get().containsKey(target); || collaboratorTerminals.get().containsKey(target);
} }
/**
* Whether {@code target} names a terminal this resolver would itself resolve as {@link
* Role#OBSERVER} — the classifier an observer's {@code SEND} is checked against, read from the
* same maps and functions {@link #resolve} consults so a target this accepts is exactly one
* {@code resolve} would hand back {@link Role#OBSERVER} for, and the reverse.
*/
public Predicate<String> sendableObserverTarget() {
return target -> target != null
&& spawnedMemberRole.apply(target) == null
&& !leadTerminals.get().containsKey(target)
&& !boundToArchitectSlot(target)
&& !collaboratorTerminals.get().containsKey(target);
}
/**
* Whether {@code terminal} is bound to a configured slot the live roster still confirms as an
* architect — the one classifier {@link #sendableObserverTarget()} and {@code FleetMcp}'s
* {@code panes} row both read, so a pane's reported role and its {@code SEND} reachability can
* never drift apart.
*/
public boolean boundToArchitectSlot(String terminal) {
String slot = architectTerminals.get().get(terminal);
return slot != null && memberSlotRoles.apply(slot) == MemberRole.ARCHITECT;
}
/** /**
* Resolve the caller of a request. * Resolve the caller of a request.
* *
@@ -323,7 +348,7 @@ public final class CallerResolver {
// of the above keeps that stronger role. // of the above keeps that stronger role.
return Principal.collaborator(collaborator, c.terminal(), c.pid()); return Principal.collaborator(collaborator, c.terminal(), c.pid());
} }
return Principal.worker(c.terminal(), c.pid()); // unforgeable; never token-gated return Principal.observer(c.terminal(), c.pid()); // unforgeable; never token-gated
} }
if (tokenMode) { if (tokenMode) {
@@ -88,6 +88,13 @@ public record Principal(Role role, String terminal, long pid, String name) {
return new Principal(Role.COLLABORATOR, terminal, pid, name); return new Principal(Role.COLLABORATOR, terminal, pid, name);
} }
/**
* The unconfigured-pane floor: a loopback caller whose pane matched no other role.
*/
public static Principal observer(String terminal, long pid) {
return new Principal(Role.OBSERVER, terminal, pid);
}
public boolean isPrimary() { public boolean isPrimary() {
return role == Role.PRIMARY; return role == Role.PRIMARY;
} }
@@ -104,11 +111,15 @@ public record Principal(Role role, String terminal, long pid, String name) {
return role == Role.WORKER; return role == Role.WORKER;
} }
public boolean isObserver() {
return role == Role.OBSERVER;
}
/** /**
* Whether this caller is a spawned member with its own pane. * Whether this caller is a spawned member with its own pane.
* *
* <p>Both workers and architects are spawned members. A lead is excluded because recording it * <p>Both workers and architects are spawned members. A lead is not: it is a peer the
* as present would count it as an available member in the roster. * operator started and named, never a pane this daemon spawned.
*/ */
public boolean isSpawnedMember() { public boolean isSpawnedMember() {
return role == Role.WORKER || role == Role.ARCHITECT; return role == Role.WORKER || role == Role.ARCHITECT;
@@ -134,12 +145,33 @@ public record Principal(Role role, String terminal, long pid, String name) {
return terminal != null && terminal.equals(sessionId); return terminal != null && terminal.equals(sessionId);
} }
/**
* Stable identity used to own tickets and open turns. The unnamed primary has no owner key —
* {@code null} — and that is matched against a ticket's recorded owner the same way any other
* key is: it owns a ticket another unnamed primary created, and nothing else.
*/
public String ownerKey() {
return switch (role) {
case PRIMARY -> name == null ? null : prefixed("leader", name);
case WORKER -> prefixed("worker", terminal);
case ARCHITECT -> prefixed("architect", terminal);
case COLLABORATOR -> prefixed("collaborator", name);
case OBSERVER -> prefixed("observer", terminal);
case ANONYMOUS -> "anonymous";
};
}
private static String prefixed(String role, String identity) {
return role + ":" + identity;
}
/** Short, non-sensitive description for audit lines and error details. */ /** Short, non-sensitive description for audit lines and error details. */
public String describe() { public String describe() {
return switch (role) { return switch (role) {
case WORKER -> "worker:" + terminal; case WORKER -> "worker:" + terminal;
case ARCHITECT -> "architect:" + name; case ARCHITECT -> "architect:" + name;
case COLLABORATOR -> "collaborator:" + name; case COLLABORATOR -> "collaborator:" + name;
case OBSERVER -> "observer:" + terminal;
case PRIMARY -> name == null ? "primary" : "leader:" + name; case PRIMARY -> name == null ? "primary" : "leader:" + name;
case ANONYMOUS -> "anonymous"; case ANONYMOUS -> "anonymous";
}; };
@@ -12,9 +12,10 @@ package dev.ltms.fleet.auth;
public enum Role { public enum Role {
/** /**
* The orchestrating session. Established either by being a loopback caller that is not a * The orchestrating session. Established either by being a loopback caller that resolves to
* worker pane (under {@code loopback-trust}) or by presenting a valid bearer token (under * no herdr pane at all (under {@code loopback-trust}) or by presenting a valid bearer token
* {@code token} mode). * (under {@code token} mode). A loopback caller that does own a pane, but matches none of the
* roles below, resolves to {@link #OBSERVER} instead.
*/ */
PRIMARY, PRIMARY,
@@ -45,6 +46,18 @@ public enum Role {
*/ */
COLLABORATOR, COLLABORATOR,
/**
* A loopback pane that resolved to none of the roles above: not a live spawned member, not a
* configured lead, not a bound architect slot, not a configured collaborator tab. Unforgeable
* like a worker's — derived from the connection's pane, never from a request argument, and
* honoured regardless of auth mode. May {@code READ} and {@code METRICS}, {@code REPLY}/
* {@code ASK} only as its own pane, and {@code SEND} only to a target that would itself
* resolve as {@code OBSERVER}; may not {@code SPAWN}/{@code STOP}/{@code DRAIN}/
* {@code HANDOVER}, poll a ticket ({@code TASK_READ}), or reach the coordination broker
* ({@code COORD_SEND}/{@code COORD_READ}).
*/
OBSERVER,
/** Authenticated as nothing. Authorized for nothing but {@code /healthz}. */ /** Authenticated as nothing. Authorized for nothing but {@code /healthz}. */
ANONYMOUS ANONYMOUS
} }
@@ -67,7 +67,7 @@ import java.util.function.Supplier;
* {@code models:} above: {@code dev.ltms.fleet.lead.LeadRollover} holds a * {@code models:} above: {@code dev.ltms.fleet.lead.LeadRollover} holds a
* {@code Supplier<FleetConfig.LeadRollover>} (the same {@code () -> config.get().x()} shape) * {@code Supplier<FleetConfig.LeadRollover>} (the same {@code () -> config.get().x()} shape)
* and reads {@code handoverPath}/{@code requireOperatorConfirm}/{@code maxDocAgeSeconds}/ * and reads {@code handoverPath}/{@code requireOperatorConfirm}/{@code maxDocAgeSeconds}/
* {@code turnSettleSeconds}/{@code clearSettleSeconds}/{@code bootstrapText} fresh on every * {@code turnSettleSeconds}/{@code relaunchReadySeconds}/{@code bootstrapText} fresh on every
* {@code open()}/{@code confirm()} call (and on the deferred post-{@code confirm()} * {@code open()}/{@code confirm()} call (and on the deferred post-{@code confirm()}
* continuation fleetd #480's correction added — see {@code LeadRollover}'s class doc) rather * continuation fleetd #480's correction added — see {@code LeadRollover}'s class doc) rather
* than capturing them into fields at construction — unlike its closest * than capturing them into fields at construction — unlike its closest
@@ -28,6 +28,7 @@ import java.util.Comparator;
import java.util.HashSet; import java.util.HashSet;
import java.util.LinkedHashMap; import java.util.LinkedHashMap;
import java.util.List; import java.util.List;
import java.util.Locale;
import java.util.Map; import java.util.Map;
import java.util.Set; import java.util.Set;
import java.util.regex.Pattern; import java.util.regex.Pattern;
@@ -1092,8 +1093,8 @@ public record FleetConfig(
} }
/** /**
* One entry of the CB-530 {@code leaders:} registry — a pane that orchestrates rather than one * One entry of the {@code leaders:} registry — a pane that orchestrates rather than one that is
* that is orchestrated. * orchestrated.
* *
* <p>Why a registry and not a second {@code primary:}: {@code primary.terminal} is singular by * <p>Why a registry and not a second {@code primary:}: {@code primary.terminal} is singular by
* construction, so a session in any other pane resolves as a worker. That is correct while one * construction, so a session in any other pane resolves as a worker. That is correct while one
@@ -1103,46 +1104,49 @@ public record FleetConfig(
* <p>{@code kind} and {@code model} are descriptive only: they document what runs in the pane * <p>{@code kind} and {@code model} are descriptive only: they document what runs in the pane
* and are reported back by {@code fleet_whoami}. * and are reported back by {@code fleet_whoami}.
* *
* <p><b>A lead is now also creatable (CB-557).</b> Before, nothing spawned one — a lead * <p>A lead with {@code profile} and {@code instances} set may be launched by the daemon when
* pre-existed, which is why it had to be recognised by configuration rather than created. With * none is live; recognition always comes first, so only the shortfall is launched.
* {@code profile} and {@code instances} the daemon may stand one up when none is live, so the
* pane no longer has to exist before the daemon does. Recognition still comes first: a lead
* already running in its configured {@code tab} is adopted, and only the shortfall is launched.
* *
* <p><b>{@code tab} replaced {@code terminal} (CB-579).</b> A herdr {@code terminal_id} changes * <p>Every lead's tab is labelled {@link #LEAD_TAB_LABEL}, a fixed constant — not a per-entry
* every time the lead's session restarts, so pinning one cost a config edit and a daemon restart * config value. {@code workspace} is therefore what tells one lead from another: two leaders
* per restart. A tab is stable: a human opens it once, it holds exactly one pane, and its label * sharing one space would both resolve to the one tab named {@code lead} there, so only one
* survives restarts of the agent inside it — so identity is now the tab label alone. * could ever be found. {@code tab} is a deprecated legacy label, still matched within this
* lead's own space alongside the constant.
* *
* @param profile the {@code profiles:} entry to launch this lead on when one must * @param profile the {@code profiles:} entry to launch this lead on when one must
* be created; {@code null} ⇒ recognise-only, never create. * be created; {@code null} ⇒ recognise-only, never create.
* <p>fleetd #176: also the field {@code Fleetd.leadSeatLookup} reads * <p>Also the field {@code Fleetd.leadSeatLookup} reads to learn
* to learn which account this lead's own live session shares — set it * which account this lead's own live session shares — set it
* (safely, even on an already-running recognise-only lead: naming a * (safely, even on an already-running recognise-only lead: naming a
* profile here never starts anything beyond {@code instances}) so a * profile here never starts anything beyond {@code instances}) so a
* {@code subscription: true} worker profile sharing its * {@code subscription: true} worker profile sharing its
* {@code effectiveCredentialId()} has this lead's seat subtracted from * {@code effectiveCredentialId()} has this lead's seat subtracted from
* {@code fleet_list}'s {@code free}. {@code null} here also means this * {@code fleet_list}'s {@code free}. {@code null} here also means this
* lead's seat cannot be derived and is not counted. * lead's seat cannot be derived and is not counted.
* @param tab the exact tab label hosting this lead, matched case-insensitively; * @param tab deprecated legacy tab label, matched case-insensitively within
* the only field identity depends on. Required — a lead with no * this lead's own space alongside {@link #LEAD_TAB_LABEL}. Optional —
* {@code tab} can never be discovered, launched or not * {@code null}/blank means only the constant is accepted
* @param instances how many of this lead should be live (default 1). The daemon * @param instances how many of this lead should be live (default 1). The daemon
* launches only the shortfall, so a restart adopts rather than doubles * launches only the shortfall, so a restart adopts rather than doubles
* @param tabPrefix lead-tab naming convention checked against member labels. Lead * @param tabPrefix lead-tab naming convention checked against member labels. Lead
* identity uses {@code tab}. Default {@code "lead:"} * identity uses {@link #acceptedLabels()}. Default {@code "lead:"}
* @param scanIntervalSeconds how long a tab scan is cached before herdr is asked again; also the * @param scanIntervalSeconds how long a tab scan is cached before herdr is asked again; also the
* worst case before a newly-labelled tab is recognised. Default 10 * worst case before a newly-labelled tab is recognised. Default 10
* @param kind which agent runs there ({@code claude}, {@code opencode}, …) * @param kind which agent runs there ({@code claude}, {@code opencode}, …)
* @param model the model or selector it runs, for operators reading the roster * @param model the model or selector it runs, for operators reading the roster
* @param workspace the space this lead's tab lives in — the uniqueness boundary
* identity now depends on. Default {@link #DEFAULT_WORKSPACE}
*/ */
@JsonIgnoreProperties(ignoreUnknown = true) @JsonIgnoreProperties(ignoreUnknown = true)
public record Leader(String profile, String tab, Integer instances, String tabPrefix, public record Leader(String profile, String tab, Integer instances, String tabPrefix,
Integer scanIntervalSeconds, String kind, String model, Integer scanIntervalSeconds, String kind, String model,
String workspace, String cwd) { String workspace, String cwd) {
/** The tab label every lead is found by, and an auto-launched instance is created with. */
public static final String LEAD_TAB_LABEL = "lead";
/** /**
* Where an auto-launched lead's tab is created (CB-558). It defaults to the SAME shared * Where an auto-launched lead's tab is created. It defaults to the SAME shared
* {@code "fleet"} space the members use, so the operator sees one "session" with many tabs. * {@code "fleet"} space the members use, so the operator sees one "session" with many tabs.
* The scanner no longer excludes member spaces — it tells a lead from a member by the exact * The scanner no longer excludes member spaces — it tells a lead from a member by the exact
* tab label, so a lead sharing the members' space is still discovered (see LeadLauncher). * tab label, so a lead sharing the members' space is still discovered (see LeadLauncher).
@@ -1170,9 +1174,23 @@ public record FleetConfig(
return profile != null && !profile.isBlank() && instances > 0; return profile != null && !profile.isBlank() && instances > 0;
} }
/** The tab label an auto-launched instance of this lead gets — its configured {@code tab}. */ /** The tab label an auto-launched instance of this lead gets. */
public String tabLabel() { public String tabLabel() {
return tab; return LEAD_TAB_LABEL;
}
/**
* The normalised labels (stripped, lower-cased) a tab in this lead's own space may carry to
* be recognised as this lead: {@link #LEAD_TAB_LABEL} first, plus the deprecated {@code tab}
* when configured and different. Every matcher in this class's callers reads this method —
* none re-derives the set.
*/
public List<String> acceptedLabels() {
String normalizedTab = (tab == null) ? null : tab.toLowerCase(Locale.ROOT);
if (normalizedTab == null || normalizedTab.equals(LEAD_TAB_LABEL)) {
return List.of(LEAD_TAB_LABEL);
}
return List.of(LEAD_TAB_LABEL, normalizedTab);
} }
} }
@@ -1416,17 +1434,16 @@ public record FleetConfig(
* before anything exists to call — the same fact already true of adding a brand-new * before anything exists to call — the same fact already true of adding a brand-new
* {@code profiles:} entry. * {@code profiles:} entry.
* *
* <p><strong>{@code turnSettleSeconds} (fleetd #480 correction):</strong> {@code confirm()} is * <p><strong>{@code turnSettleSeconds}:</strong> {@code confirm()} is called FROM the calling
* called FROM the calling lead's own turn, so its pane is still {@code WORKING} the instant * lead's own turn, so its pane is still {@code WORKING} the instant {@code confirm()} validates
* {@code confirm()} validates every gate and schedules the roll. {@code * every gate and schedules the roll. {@code dev.ltms.fleet.lead.LeadRollover}'s deferred
* dev.ltms.fleet.lead.LeadRollover}'s deferred continuation waits up to this many seconds for * continuation waits up to this many seconds for that SAME pane to report {@code IDLE} or
* that SAME pane to report {@code IDLE} or {@code DONE} — i.e. for the calling turn to actually * {@code DONE} — i.e. for the calling turn to actually end — before it ends the old pane's
* end — before it sends {@code /clear} at all. {@code BLOCKED} does not count: that is a live * process at all. {@code BLOCKED} does not count: that is a live turn merely paused, not one
* turn merely paused, not one that has finished. If that wait times out, no {@code /clear} is * that has finished. If that wait times out, the old pane is never touched: a lead that never
* ever sent: a lead that never goes idle is still doing real work, and clearing it would * goes idle is still doing real work, and the roll ends that pane's whole process — there is no
* destroy live context. This is a separate wait from {@code clearSettleSeconds} below, which * way back from this once it runs, so this wait is the only thing standing between "still
* bounds the SECOND wait, for the pane to reach {@code IDLE} or {@code DONE} again AFTER * working" and "gone".
* {@code /clear} has already gone out.
* *
* @param handoverPath required when this block is present — where the handover file a fresh * @param handoverPath required when this block is present — where the handover file a fresh
* lead session reads must live. There is no sane non-null default for an * lead session reads must live. There is no sane non-null default for an
@@ -1447,36 +1464,49 @@ public record FleetConfig(
* attempt can never be mistaken for a fresh one. * attempt can never be mistaken for a fresh one.
* @param turnSettleSeconds default 300 — bound on how long the deferred roll waits for the * @param turnSettleSeconds default 300 — bound on how long the deferred roll waits for the
* CALLING lead's own turn to end (its pane to report {@code IDLE} or * CALLING lead's own turn to end (its pane to report {@code IDLE} or
* {@code DONE}) before sending {@code /clear} at all. See the paragraph * {@code DONE}) before ending that pane's process at all. See the
* above. * paragraph above.
* @param clearSettleSeconds default 20 — bound on how long to wait for the lead's pane to * @param relaunchReadySeconds default 45 — bound on EACH of two separate waits that run after
* report {@code IDLE} or {@code DONE} again after {@code /clear} before * the old lead's pane has been torn down and a fresh one launched: first,
* giving up. A roll that times out here never sends {@code bootstrapText}. * for the fresh pane itself to reach a real turn boundary ({@code IDLE} or
* {@code DONE}, never merely {@code BLOCKED}) — the safety gate, since
* typing into a pane that has not finished booting loses the keystrokes;
* second, for the fresh terminal to show up as a recognised lead, which is
* bookkeeping rather than a safety gate, so a timeout on this second wait
* does not withhold {@code bootstrapText} — it is sent once the pane is
* ready regardless. Recognition comes from the same periodically-refreshed
* scan {@code LeadTabScanner} already keeps ({@code scanIntervalSeconds},
* 10s live), so a budget has to clear more than one scan interval to leave
* any real margin for the CLI's own boot time; 20 was rejected for exactly
* that reason — at a 10s scan interval it only buys two scans. 45 buys
* roughly four. Only a timeout on the FIRST wait (the pane never becomes
* ready) withholds {@code bootstrapText}.
* @param bootstrapText default a sentence naming the RESOLVED handover path — sent to the * @param bootstrapText default a sentence naming the RESOLVED handover path — sent to the
* lead's pane once it settles after {@code /clear}, telling the fresh * fresh lead's pane once it reaches a real turn boundary after relaunch,
* session where to read the handover and carry on. Left {@code null} here * telling the fresh session where to read the handover and carry on. Left
* when the operator configures none: the default sentence cannot be built * {@code null} here when the operator configures none: the default sentence
* at construction time because it must name the path AFTER {@code * cannot be built at construction time because it must name the path AFTER
* dev.ltms.fleet.lead.LeadRollover#open} has resolved a relative {@code * {@code dev.ltms.fleet.lead.LeadRollover#open} has resolved a relative
* handoverPath} against the calling lead's workspace, which this record has * {@code handoverPath} against the calling lead's workspace, which this
* no way to know — see {@link #bootstrapTextFor(String)}. * record has no way to know — see {@link #bootstrapTextFor(String)}.
*/ */
@JsonIgnoreProperties(ignoreUnknown = true) @JsonIgnoreProperties(ignoreUnknown = true)
public record LeadRollover(String handoverPath, Boolean requireOperatorConfirm, public record LeadRollover(String handoverPath, Boolean requireOperatorConfirm,
Integer maxDocAgeSeconds, Integer turnSettleSeconds, Integer maxDocAgeSeconds, Integer turnSettleSeconds,
Integer clearSettleSeconds, String bootstrapText) { Integer relaunchReadySeconds, String bootstrapText) {
public LeadRollover { public LeadRollover {
requireOperatorConfirm = requireOperatorConfirm == null || requireOperatorConfirm; requireOperatorConfirm = requireOperatorConfirm == null || requireOperatorConfirm;
maxDocAgeSeconds = (maxDocAgeSeconds == null || maxDocAgeSeconds <= 0) ? 3600 : maxDocAgeSeconds; maxDocAgeSeconds = (maxDocAgeSeconds == null || maxDocAgeSeconds <= 0) ? 3600 : maxDocAgeSeconds;
turnSettleSeconds = (turnSettleSeconds == null || turnSettleSeconds <= 0) ? 300 : turnSettleSeconds; turnSettleSeconds = (turnSettleSeconds == null || turnSettleSeconds <= 0) ? 300 : turnSettleSeconds;
clearSettleSeconds = (clearSettleSeconds == null || clearSettleSeconds <= 0) ? 20 : clearSettleSeconds; relaunchReadySeconds = (relaunchReadySeconds == null || relaunchReadySeconds <= 0)
? 45 : relaunchReadySeconds;
bootstrapText = (bootstrapText == null || bootstrapText.isBlank()) ? null : bootstrapText; bootstrapText = (bootstrapText == null || bootstrapText.isBlank()) ? null : bootstrapText;
} }
/** /**
* The text actually sent to the lead's pane once it settles after {@code /clear}: the * The text actually sent to the fresh lead's pane once it reaches a real turn boundary
* operator's configured {@link #bootstrapText} when one is set, otherwise the default * after relaunch: the operator's configured {@link #bootstrapText} when one is set,
* sentence built from {@code resolvedHandoverPath}. * otherwise the default sentence built from {@code resolvedHandoverPath}.
* *
* @param resolvedHandoverPath the ABSOLUTE path {@code dev.ltms.fleet.lead.LeadRollover * @param resolvedHandoverPath the ABSOLUTE path {@code dev.ltms.fleet.lead.LeadRollover
* #open} already resolved — never the raw configured {@link * #open} already resolved — never the raw configured {@link
@@ -1919,6 +1949,7 @@ public record FleetConfig(
rejectNegativeMaxLoad(yaml); rejectNegativeMaxLoad(yaml);
rejectAutoCompactWindowOutOfRange(yaml); rejectAutoCompactWindowOutOfRange(yaml);
warnConflictingAutoCompactWindows(yaml); warnConflictingAutoCompactWindows(yaml);
warnRetiredClearSettleSecondsKey(yaml);
rejectMalformedProfilePatterns(yaml); rejectMalformedProfilePatterns(yaml);
rejectUnknownKind(yaml); rejectUnknownKind(yaml);
rejectUnknownAuthMode(yaml); rejectUnknownAuthMode(yaml);
@@ -2082,13 +2113,6 @@ public record FleetConfig(
} }
} }
/**
* The top-level keys in {@code yaml} that this build does not understand, sorted. Package-private
* so the guardrail is asserted directly rather than through a log appender.
*
* @return empty when everything is known, or when {@code yaml} is not a mapping at all (a
* malformed file is {@code readValue}'s error to report, not this method's)
*/
/** /**
* Top-level keys renamed by the member taxonomy, mapped old → new. * Top-level keys renamed by the member taxonomy, mapped old → new.
* *
@@ -2342,6 +2366,33 @@ public record FleetConfig(
names, String.join(", ", detail)); names, String.join(", ", detail));
} }
/**
* Warn when a {@code leadRollover:} block still sets the retired {@code clearSettleSeconds}
* key. {@link LeadRollover} carries {@code @JsonIgnoreProperties(ignoreUnknown = true)} and no
* longer declares that component, so Jackson drops it with no signal of its own — this raw-YAML
* check is the only place an operator's now-inert setting is reported at all; by the time a
* {@link LeadRollover} instance exists to run a validator against, the key is already gone.
*
* @param yaml the raw config text
*/
static void warnRetiredClearSettleSecondsKey(String yaml) {
Map<?, ?> raw;
try {
raw = YAML.readValue(yaml, Map.class);
} catch (IOException | IllegalArgumentException e) {
return;
}
if (raw == null || !(raw.get("leadRollover") instanceof Map<?, ?> leadRollover)) {
return;
}
if (leadRollover.containsKey("clearSettleSeconds")) {
log.warn("leadRollover.clearSettleSeconds is retired and no longer read. Set "
+ "leadRollover.relaunchReadySeconds instead: it bounds how long to wait, after "
+ "a lead is relaunched, for its pane to become ready and then for it to be "
+ "recognised as a lead. Remove clearSettleSeconds from fleetd.yaml.");
}
}
/** /**
* Reject a profile whose {@code errorPattern} (fleetd #201 Unit 5) or {@code exhaustedPattern} * Reject a profile whose {@code errorPattern} (fleetd #201 Unit 5) or {@code exhaustedPattern}
* (CB-578 stage A) is not a valid Java regex, naming the profile, the key, and the parser's own * (CB-578 stage A) is not a valid Java regex, naming the profile, the key, and the parser's own
@@ -2596,6 +2647,13 @@ public record FleetConfig(
} }
} }
/**
* The top-level keys in {@code yaml} that this build does not understand, sorted. Package-private
* so the guardrail is asserted directly rather than through a log appender.
*
* @return empty when everything is known, or when {@code yaml} is not a mapping at all (a
* malformed file is {@code readValue}'s error to report, not this method's)
*/
static List<String> unknownTopLevelKeys(String yaml) { static List<String> unknownTopLevelKeys(String yaml) {
Map<?, ?> raw; Map<?, ?> raw;
try { try {
@@ -2707,23 +2765,42 @@ public record FleetConfig(
/** /**
* Reject a member tab-label template that could render as a configured lead or collaborator * Reject a member tab-label template that could render as a configured lead or collaborator
* tab or match a lead-tab naming convention, and reject two {@code fleet.leaders} or * tab, as the fixed lead tab label, or that matches a lead-tab naming convention; reject two
* {@code fleet.collaborators} entries — across either registry — that share one exact tab. * {@code fleet.leaders} entries that share one space; reject two {@code fleet.collaborators}
* entries — or a lead and a collaborator — that share one exact tab; and reject a collaborator
* tab equal to the fixed lead tab label.
* *
* <p>{@code fleet.collaborators} has no {@code tabPrefix}: identity is matched on the exact * <p>{@code fleet.collaborators} has no {@code tabPrefix}: identity is matched on the exact
* {@code tab} alone, so only the exact-render check applies there, not the prefix check. * {@code tab} alone, so only the exact-render check applies there, not the prefix check.
* *
* @throws IllegalStateException when the fleet template or a profile {@code tabLabel} override * @throws IllegalStateException when the fleet template or a profile {@code tabLabel} override
* can render as a configured lead or collaborator tab or match a * can render as a configured lead or collaborator tab, as the
* lead-tab prefix, or when two entries — of either registry, or * fixed lead tab label, or match a lead-tab prefix; when two
* one of each — carry the same exact {@code tab} * leaders share one space; when two collaborators (or a lead and
* (case-insensitively) * a collaborator) carry the same exact {@code tab}
* (case-insensitively); or when a collaborator's {@code tab}
* equals the fixed lead tab label
*/ */
public void validateLeadTabPrefixes() { public void validateLeadTabPrefixes() {
if (fleet == null) { if (fleet == null) {
return; return;
} }
List<String> bad = new ArrayList<>(); List<String> bad = new ArrayList<>();
if (templateCanRenderAs(fleet.tabLabel(), Leader.LEAD_TAB_LABEL)) {
bad.add("fleet.tabLabel=\"" + fleet.tabLabel() + "\" can render as \""
+ Leader.LEAD_TAB_LABEL + "\", the fixed lead tab label");
}
profiles().entrySet().stream()
.map(Map.Entry::getKey)
.sorted()
.forEach(p -> {
String label = profiles().get(p).tabLabel();
if (templateCanRenderAs(label, Leader.LEAD_TAB_LABEL)) {
bad.add("profile '" + p + "' overrides tabLabel with \"" + label
+ "\", which can render as \"" + Leader.LEAD_TAB_LABEL
+ "\", the fixed lead tab label");
}
});
fleet.leaders().forEach((leadName, leader) -> { fleet.leaders().forEach((leadName, leader) -> {
if (leader == null) { if (leader == null) {
return; return;
@@ -2758,6 +2835,10 @@ public record FleetConfig(
return; return;
} }
String tab = collaborator.tab(); String tab = collaborator.tab();
if (tab != null && tab.equalsIgnoreCase(Leader.LEAD_TAB_LABEL)) {
bad.add("fleet.collaborators." + collabName + ".tab=\"" + tab + "\" is the fixed "
+ "lead tab label — a collaborator there would shadow a lead");
}
if (templateCanRenderAs(fleet.tabLabel(), tab)) { if (templateCanRenderAs(fleet.tabLabel(), tab)) {
bad.add("fleet.tabLabel=\"" + fleet.tabLabel() + "\" can render as the tab of " bad.add("fleet.tabLabel=\"" + fleet.tabLabel() + "\" can render as the tab of "
+ "collaborator '" + collabName + "' (\"" + tab + "\")"); + "collaborator '" + collabName + "' (\"" + tab + "\")");
@@ -2787,18 +2868,20 @@ public record FleetConfig(
for (int i = 0; i < leadNames.size(); i++) { for (int i = 0; i < leadNames.size(); i++) {
String nameA = leadNames.get(i); String nameA = leadNames.get(i);
Leader a = fleet.leaders().get(nameA); Leader a = fleet.leaders().get(nameA);
if (a == null || a.tab() == null || a.tab().isBlank()) { if (a == null) {
continue; continue;
} }
for (int j = i + 1; j < leadNames.size(); j++) { for (int j = i + 1; j < leadNames.size(); j++) {
String nameB = leadNames.get(j); String nameB = leadNames.get(j);
Leader b = fleet.leaders().get(nameB); Leader b = fleet.leaders().get(nameB);
if (b == null || b.tab() == null || b.tab().isBlank()) { if (b == null) {
continue; continue;
} }
if (a.tab().equalsIgnoreCase(b.tab())) { if (a.workspace().equalsIgnoreCase(b.workspace())) {
collisions.add("lead '" + nameA + "' and lead '" + nameB + "' both use tab \"" collisions.add("lead '" + nameA + "' and lead '" + nameB + "' share workspace \""
+ a.tab() + "\""); + a.workspace() + "\" — both would resolve to the tab named \""
+ Leader.LEAD_TAB_LABEL + "\" in that space, so only one could ever be "
+ "found");
} }
} }
} }
@@ -2842,9 +2925,9 @@ public record FleetConfig(
return; return;
} }
throw new IllegalStateException("refusing to start: " + String.join("; ", collisions) throw new IllegalStateException("refusing to start: " + String.join("; ", collisions)
+ ". Tab identity is matched exactly, so only one of two entries sharing a tab can " + ". Identity is matched exactly, so only one of two entries sharing a space or a "
+ "ever be found — the other is silently unreachable. Give each lead and " + "tab can ever be found — the other is silently unreachable. Give each lead its "
+ "collaborator its own exact tab."); + "own space, and each collaborator its own exact tab.");
} }
private static boolean templateCanRenderAs(String template, String tab) { private static boolean templateCanRenderAs(String template, String tab) {
@@ -2873,30 +2956,32 @@ public record FleetConfig(
} }
/** /**
* Reject a profile that places its members by {@code "pane"} while any {@code fleet.leaders} * Reject a profile that places its members by {@code "pane"} while {@code fleet.leaders} has
* or {@code fleet.collaborators} entry names a {@code tab}. A pane-placed member lands inside * any entry, or any {@code fleet.collaborators} entry names a {@code tab}. A pane-placed
* the focused tab rather than its own, so it can land inside a lead's or collaborator's own * member lands inside the focused tab rather than its own, so it can land inside a lead's or
* labelled tab. {@link dev.ltms.fleet.herdr.LeadTabScanner} identifies a lead or collaborator * collaborator's own labelled tab. {@link dev.ltms.fleet.herdr.LeadTabScanner} identifies a
* purely by that tab's label — it does not exclude the member space — so a member that ends up * lead or collaborator purely by that tab's label — it does not exclude the member space — so
* there, while its pane carries no entry in the spawned-member roster, is read back as that * a member that ends up there, while its pane carries no entry in the spawned-member roster,
* lead or collaborator and granted that identity's authority. * is read back as that lead or collaborator and granted that identity's authority.
* *
* <p>Only an entry with a non-blank {@code tab} is in scope: one with no {@code tab} feeds * <p>Every {@code fleet.leaders} entry is in scope regardless of its own {@code tab} field:
* {@link Leader#acceptedLabels()} always includes {@link Leader#LEAD_TAB_LABEL}. Only a
* collaborator with a non-blank {@code tab} is in scope: one with no {@code tab} feeds
* nothing into {@link dev.ltms.fleet.herdr.LeadTabScanner}, so it creates no hazard here. * nothing into {@link dev.ltms.fleet.herdr.LeadTabScanner}, so it creates no hazard here.
* *
* @throws IllegalStateException when any {@code profiles:} entry is pane-placed while any * @throws IllegalStateException when any {@code profiles:} entry is pane-placed while
* {@code fleet.leaders} or {@code fleet.collaborators} entry * {@code fleet.leaders} is non-empty, or any
* names a non-blank {@code tab} * {@code fleet.collaborators} entry names a non-blank
* {@code tab}
*/ */
public void validatePanePlacementAgainstLeadTabs() { public void validatePanePlacementAgainstLeadTabs() {
if (fleet == null) { if (fleet == null) {
return; return;
} }
boolean anyLeaderHasTab = fleet.leaders().values().stream() boolean anyLead = !fleet.leaders().isEmpty();
.anyMatch(leader -> leader != null && leader.tab() != null && !leader.tab().isBlank());
boolean anyCollaboratorHasTab = fleet.collaborators().values().stream() boolean anyCollaboratorHasTab = fleet.collaborators().values().stream()
.anyMatch(c -> c != null && c.tab() != null && !c.tab().isBlank()); .anyMatch(c -> c != null && c.tab() != null && !c.tab().isBlank());
if (!anyLeaderHasTab && !anyCollaboratorHasTab) { if (!anyLead && !anyCollaboratorHasTab) {
return; return;
} }
List<String> bad = new ArrayList<>(); List<String> bad = new ArrayList<>();
@@ -2913,8 +2998,9 @@ public record FleetConfig(
+ "pane-placed member can land inside that labelled tab, and while its pane " + "pane-placed member can land inside that labelled tab, and while its pane "
+ "carries no entry in the spawned-member roster, it is read back as the lead or " + "carries no entry in the spawned-member roster, it is read back as the lead or "
+ "collaborator and granted that identity's authority. Set placement: tab for " + "collaborator and granted that identity's authority. Set placement: tab for "
+ "each named profile, or remove the tab from every fleet.leaders and " + "each named profile — the only fix when a lead triggered this, since a lead's "
+ "fleet.collaborators entry."); + "tab label is fixed regardless of its own tab: field. A collaborator's tab can "
+ "still be removed instead.");
} }
/** /**
@@ -3021,14 +3107,13 @@ public record FleetConfig(
* so duplicates are unrepresentable by construction once loaded — and {@link #load(Path)} * so duplicates are unrepresentable by construction once loaded — and {@link #load(Path)}
* already rejects a duplicated slot name at parse time, before the map collapses. * already rejects a duplicated slot name at parse time, before the map collapses.
* *
* <p>Also rejects a {@code fleet.collaborators} entry with no (or a blank) {@code tab}. A * <p>A lead's {@code profile} is optional — a {@code profile}-less lead is still useful
* {@code profile}-less lead is still useful recognise-only — {@code tab} is the only field * recognise-only. Also rejects a {@code fleet.collaborators} entry with no (or a blank)
* that matters to it either way. A collaborator carries no other field at all, so a blank * {@code tab}: a collaborator carries no other field at all, so a blank {@code tab} leaves
* {@code tab} leaves nothing for the entry to mean. * nothing for the entry to mean.
* *
* @throws IllegalStateException when a slot names no profile or an unknown one, when a lead * @throws IllegalStateException when a slot or a lead references an unknown profile, or a
* can be neither found nor created, or when a collaborator names * collaborator names no tab, naming the offending entry
* no tab, naming the offending entry
*/ */
public void validateMembers() { public void validateMembers() {
if (fleet == null) { if (fleet == null) {
@@ -3061,10 +3146,6 @@ public record FleetConfig(
+ "', which is not a configured profiles: entry (have: " + profiles.keySet() + "', which is not a configured profiles: entry (have: " + profiles.keySet()
+ ")."); + ").");
} }
if (leader.tab() == null || leader.tab().isBlank()) {
bad.add("fleet.leaders." + name + " has no tab: — a lead is now found (and, if "
+ "auto-launched, labelled) purely by its tab, so every entry must name one.");
}
}); });
fleet.collaborators().forEach((name, collaborator) -> { fleet.collaborators().forEach((name, collaborator) -> {
if (collaborator == null) { if (collaborator == null) {
@@ -25,14 +25,12 @@ import java.util.function.Supplier;
* by first starting the session and asking it. Scanning closes that loop: label the tab, and the * by first starting the session and asking it. Scanning closes that loop: label the tab, and the
* pane is recognised on the next resolve. * pane is recognised on the next resolve.
* *
* <p><strong>CB-579 — matched by name, not prefix.</strong> This used to strip one shared * <p><strong>Matched by label within a space, not by a shared prefix.</strong> Each lead's accepted
* {@code tabPrefix} off a label to derive the lead's name, and merged a config-supplied * labels (the fixed {@code lead} label, plus a deprecated {@code tab} when still configured) are
* {@code terminal_id} pin over every scan result so the pin could never expire. Both are gone: each * matched exactly (case-insensitively) against tabs in that lead's own space only — a tab named
* lead now configures its own exact {@code tab} label ({@code fleet.leaders.<name>.tab}), so this * {@code lead} in one space never resolves to another space's lead. A scan result is the whole
* class is handed a {@code tab → name} map up front and matches labels against it exactly * answer; nothing is merged in from configuration between scans, so a tab that is gone drops out on
* (case-insensitively). There is no merge step — a scan result is the whole answer. That is the * the very next scan instead of lingering forever.
* fix for the bug this replaces: a {@code terminal_id} pin surviving in config after the pane it
* named was gone, so the daemon kept treating a dead session as a live lead forever.
* *
* <p><strong>Direction of trust.</strong> The label names the lead; it never <em>grants</em> * <p><strong>Direction of trust.</strong> The label names the lead; it never <em>grants</em>
* anything a pane could take for itself. Three properties keep that honest: * anything a pane could take for itself. Three properties keep that honest:
@@ -103,7 +101,8 @@ public final class LeadTabScanner implements Supplier<Map<String, String>> {
private record Entry(String name, Kind kind) {} private record Entry(String name, Kind kind) {}
private final HerdrClient herdr; private final HerdrClient herdr;
private final Map<String, Entry> tabToEntry; private final Map<String, Map<String, String>> leadLabelsBySpace;
private final Map<String, String> collaboratorTabToName;
private final Set<String> excludedWorkspaceLabels; private final Set<String> excludedWorkspaceLabels;
private final long ttlNanos; private final long ttlNanos;
private final LongSupplier clock; private final LongSupplier clock;
@@ -124,15 +123,17 @@ public final class LeadTabScanner implements Supplier<Map<String, String>> {
/** /**
* @param herdr the herdr client to query ({@code workspace.list}, * @param herdr the herdr client to query ({@code workspace.list},
* {@code tab.list}, {@code pane.list} — all read-only) * {@code tab.list}, {@code pane.list} — all read-only)
* @param tabToName every configured lead's exact tab label → its name * @param leadLabelsBySpace each configured lead's accepted tab labels, keyed by the
* ({@code fleet.leaders.<name>.tab}), matched case-insensitively * lead's own space label, then by label, to its name — matched
* case-insensitively on both the space and the label. A tab
* matches a lead only within that lead's own space
* @param excludedWorkspaceLabels workspaces never scanned — the configured worker spaces * @param excludedWorkspaceLabels workspaces never scanned — the configured worker spaces
* @param ttlNanos how long a scan result is reused before the next one * @param ttlNanos how long a scan result is reused before the next one
* @param clock nanosecond time source ({@code System::nanoTime} in production) * @param clock nanosecond time source ({@code System::nanoTime} in production)
*/ */
public LeadTabScanner(HerdrClient herdr, Map<String, String> tabToName, public LeadTabScanner(HerdrClient herdr, Map<String, Map<String, String>> leadLabelsBySpace,
Set<String> excludedWorkspaceLabels, long ttlNanos, LongSupplier clock) { Set<String> excludedWorkspaceLabels, long ttlNanos, LongSupplier clock) {
this(herdr, tabToName, Map.of(), excludedWorkspaceLabels, ttlNanos, clock); this(herdr, leadLabelsBySpace, Map.of(), excludedWorkspaceLabels, ttlNanos, clock);
} }
/** /**
@@ -140,14 +141,15 @@ public final class LeadTabScanner implements Supplier<Map<String, String>> {
* for configured collaborator tabs in the same pass. * for configured collaborator tabs in the same pass.
* *
* @param collaboratorTabToName every configured collaborator's exact tab label → its name * @param collaboratorTabToName every configured collaborator's exact tab label → its name
* ({@code fleet.collaborators.<name>.tab}), matched the same way as * ({@code fleet.collaborators.<name>.tab}), matched
* {@code tabToName} * case-insensitively in any space
*/ */
public LeadTabScanner(HerdrClient herdr, Map<String, String> tabToName, public LeadTabScanner(HerdrClient herdr, Map<String, Map<String, String>> leadLabelsBySpace,
Map<String, String> collaboratorTabToName, Map<String, String> collaboratorTabToName,
Set<String> excludedWorkspaceLabels, long ttlNanos, LongSupplier clock) { Set<String> excludedWorkspaceLabels, long ttlNanos, LongSupplier clock) {
this.herdr = herdr; this.herdr = herdr;
this.tabToEntry = buildTabIndex(tabToName, collaboratorTabToName); this.leadLabelsBySpace = buildLeadIndex(leadLabelsBySpace);
this.collaboratorTabToName = normalizedLabelMap(collaboratorTabToName);
this.excludedWorkspaceLabels = excludedWorkspaceLabels == null this.excludedWorkspaceLabels = excludedWorkspaceLabels == null
? Set.of() : Set.copyOf(excludedWorkspaceLabels); ? Set.of() : Set.copyOf(excludedWorkspaceLabels);
this.ttlNanos = ttlNanos; this.ttlNanos = ttlNanos;
@@ -155,30 +157,46 @@ public final class LeadTabScanner implements Supplier<Map<String, String>> {
} }
/** /**
* Keys stripped and lower-cased once, so every lookup is a plain map hit. Leads and * Space and label keys stripped and lower-cased once, so every lookup is a plain map hit. A
* collaborators merge into a single index, so {@link #scan()} matches both kinds in one pass * space with no usable labels is simply absent — {@link #leadLabelsFor} then finds nothing for
* over the tab list; a label naming both a lead and a collaborator takes the lead entry — * it, which is also what a space with a {@code null} label gets.
* leads are put last, so a colliding key's lead entry is the one that overwrites — since a lead
* can already do everything a collaborator can. Config validation already refuses a lead and a
* collaborator sharing one exact tab, so this ordering is defence in depth, not the control.
*/ */
private static Map<String, Entry> buildTabIndex(Map<String, String> tabToName, private static Map<String, Map<String, String>> buildLeadIndex(
Map<String, String> collaboratorTabToName) { Map<String, Map<String, String>> leadLabelsBySpace) {
Map<String, Entry> out = new LinkedHashMap<>(); Map<String, Map<String, String>> out = new LinkedHashMap<>();
putNormalized(out, collaboratorTabToName, Kind.COLLABORATOR); if (leadLabelsBySpace == null) {
putNormalized(out, tabToName, Kind.LEAD); return Map.of();
}
leadLabelsBySpace.forEach((space, labelsToName) -> {
if (space == null || space.isBlank()) {
return;
}
Map<String, String> normalized = normalizedLabelMap(labelsToName);
if (!normalized.isEmpty()) {
out.put(space.strip().toLowerCase(Locale.ROOT), normalized);
}
});
return Collections.unmodifiableMap(out); return Collections.unmodifiableMap(out);
} }
private static void putNormalized(Map<String, Entry> out, Map<String, String> tabToName, Kind kind) { private static Map<String, String> normalizedLabelMap(Map<String, String> labelToName) {
if (tabToName == null) { Map<String, String> out = new LinkedHashMap<>();
return; if (labelToName != null) {
labelToName.forEach((label, name) -> {
if (label != null && !label.isBlank() && name != null && !name.isBlank()) {
out.put(label.strip().toLowerCase(Locale.ROOT), name);
}
});
} }
tabToName.forEach((tab, name) -> { return Collections.unmodifiableMap(out);
if (tab != null && !tab.isBlank() && name != null && !name.isBlank()) { }
out.put(tab.strip().toLowerCase(Locale.ROOT), new Entry(name, kind));
} /** The accepted lead labels configured for {@code spaceLabel}, or an empty map for no match. */
}); private Map<String, String> leadLabelsFor(String spaceLabel) {
if (spaceLabel == null) {
return Map.of();
}
return leadLabelsBySpace.getOrDefault(spaceLabel.strip().toLowerCase(Locale.ROOT), Map.of());
} }
/** /**
@@ -241,9 +259,10 @@ public final class LeadTabScanner implements Supplier<Map<String, String>> {
if (ws.workspaceId() == null || excludedWorkspaceLabels.contains(ws.label())) { if (ws.workspaceId() == null || excludedWorkspaceLabels.contains(ws.label())) {
continue; continue;
} }
Map<String, String> leadLabelsHere = leadLabelsFor(ws.label());
for (JsonNode t : herdr.call("tab.list", Map.of("workspace_id", ws.workspaceId())).path("tabs")) { for (JsonNode t : herdr.call("tab.list", Map.of("workspace_id", ws.workspaceId())).path("tabs")) {
Tab tab = Tab.from(t); Tab tab = Tab.from(t);
Entry entry = entryOf(tab.label()); Entry entry = entryOf(tab.label(), leadLabelsHere);
if (entry != null && tab.tabId() != null) { if (entry != null && tab.tabId() != null) {
entryByTab.put(tab.tabId(), entry); entryByTab.put(tab.tabId(), entry);
} }
@@ -296,19 +315,27 @@ public final class LeadTabScanner implements Supplier<Map<String, String>> {
} }
/** /**
* The entry a tab label declares, or {@code null} if it names neither a configured lead nor a * The entry a tab label declares within one space, or {@code null} if it names neither a lead
* configured collaborator. * accepted in {@code leadLabelsHere} nor a configured collaborator.
* *
* <p>Exact match (case-insensitive, ends stripped) against {@link #tabToEntry} — no prefix * <p>Exact match (case-insensitive, ends stripped) — no prefix stripping, so an operator's
* stripping, so an operator's {@code "lead: something-else"} tab is never mistaken for a * {@code "lead: something-else"} tab is never mistaken for a configured lead just because it
* configured lead just because it shares a prefix. The match strips a trailing * shares a prefix. The match strips a trailing {@link PendingCloseMarker} first, so a tab
* {@link PendingCloseMarker} first, so a tab {@code LeadLauncher} has flagged as maybe-dead but * {@code LeadLauncher} has flagged as maybe-dead but not yet closed keeps resolving normally
* not yet closed keeps resolving normally while that reconcile is pending. * while that reconcile is pending. A lead match wins over a collaborator match for the same
* label — a lead can already do everything a collaborator can, and config validation refuses a
* lead and a collaborator sharing one exact tab in the first place.
*/ */
private Entry entryOf(String label) { private Entry entryOf(String label, Map<String, String> leadLabelsHere) {
if (label == null) { if (label == null) {
return null; return null;
} }
return tabToEntry.get(PendingCloseMarker.strip(label).toLowerCase(Locale.ROOT)); String normalized = PendingCloseMarker.strip(label).toLowerCase(Locale.ROOT);
String leadName = leadLabelsHere.get(normalized);
if (leadName != null) {
return new Entry(leadName, Kind.LEAD);
}
String collaboratorName = collaboratorTabToName.get(normalized);
return collaboratorName == null ? null : new Entry(collaboratorName, Kind.COLLABORATOR);
} }
} }
@@ -4,6 +4,7 @@ import com.fasterxml.jackson.databind.JsonNode;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet; import java.util.LinkedHashSet;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
@@ -145,6 +146,52 @@ public final class PaneLocator {
return ancestry; return ancestry;
} }
/**
* Every tab herdr tracks across every searched daemon, keyed by tab id, to its display label —
* the pane-discovery surface behind {@code GET /agents} and {@code fleet_list}'s {@code panes}
* row. Collapses to one scan in the single-daemon deployment, the same as
* {@link #terminalForPid}. A tab herdr reports with no label maps to a {@code null} value here;
* a tab with no {@code tab_id} is skipped.
*/
public Map<String, String> tabLabelsByTabId() {
Map<String, String> out = new LinkedHashMap<>();
for (HerdrClient herdr : herdrs) {
for (JsonNode w : herdr.call("workspace.list").path("workspaces")) {
String workspaceId = w.path("workspace_id").asText(null);
if (workspaceId == null) {
continue;
}
for (JsonNode t : herdr.call("tab.list", Map.of("workspace_id", workspaceId)).path("tabs")) {
Tab tab = Tab.from(t);
if (tab.tabId() != null) {
out.put(tab.tabId(), tab.label());
}
}
}
}
return out;
}
/**
* Every workspace ("space") herdr tracks across every searched daemon, keyed by workspace id,
* to its display label — the human-readable name behind {@code fleet_list}'s {@code panes} row,
* next to herdr's own internal {@code workspaceId}. Collapses to one scan in the single-daemon
* deployment, the same as {@link #terminalForPid}. A workspace herdr reports with no label maps
* to a {@code null} value here; a workspace with no {@code workspace_id} is skipped.
*/
public Map<String, String> workspaceLabelsByWorkspaceId() {
Map<String, String> out = new LinkedHashMap<>();
for (HerdrClient herdr : herdrs) {
for (JsonNode w : herdr.call("workspace.list").path("workspaces")) {
Workspace workspace = Workspace.from(w);
if (workspace.workspaceId() != null) {
out.put(workspace.workspaceId(), workspace.label());
}
}
}
return out;
}
/** Whether a pane owns one of the scanned pid's ancestors, or the check of it failed outright. */ /** Whether a pane owns one of the scanned pid's ancestors, or the check of it failed outright. */
private enum Ownership { OWNS, DOES_NOT_OWN, UNKNOWN } private enum Ownership { OWNS, DOES_NOT_OWN, UNKNOWN }
@@ -0,0 +1,169 @@
package dev.ltms.fleet.herdr;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
/**
* Whether an agent pane's input box is clear for a delivery.
*
* <p>{@link AgentControl#send} pastes its text and submits it in the same call, so a delivery into a
* pane whose input box already holds characters submits those characters too. {@link
* AgentStatus#injectable()} cannot see that: it describes the agent, and an agent waiting at its
* prompt reports the same status whether its box is empty or holds a half-typed line. This reads the
* box itself.
*
* <p>Only a box that is positively empty clears the gate. A box with content, a pane this cannot
* recognise, and a failed read all hold the delivery, because a held delivery is recoverable and a
* submitted half-line is not. Every caller must therefore be a path that retries.
*
* <p>A pane that holds for {@link #HOLD_WARN_STREAK} consecutive checks gets one warning, so a box
* that never clears is visible instead of silent. The warning repeats only after the box has cleared
* again.
*/
public final class PromptBox {
private static final Logger log = LoggerFactory.getLogger(PromptBox.class);
/**
* herdr {@code agent.read} source. {@code detection} is the region herdr itself uses for status
* detection, so the input box is always drawn in it. It is not a short tail: it carries transcript
* scrollback above the box, including earlier prompts the operator has already submitted, which is
* why only the last box line on it is the live one.
*/
static final String PROBE_SOURCE = "detection";
/** Consecutive holds for one target before one warning is logged. */
static final int HOLD_WARN_STREAK = 20;
/**
* Input box markers, each matched only as a line's first characters: the caret the current TUI
* draws, and the bordered box an older one drew. A marker further along a line is transcript text,
* such as a caret inside something the operator quoted.
*/
private static final List<String> BOX_MARKERS = List.of("❯", "│ >");
/** Marker of a turn that is still generating; a box drawn under it is not a settled prompt. */
private static final String ACTIVE_TURN_MARKER = "esc to interrupt";
/** Block glyphs a terminal capture can leave in an otherwise empty box for the cursor cell. */
private static final String CURSOR_GLYPHS = "█▉▊▋▌▍▎▏";
/** What a box holds: nothing, unsubmitted characters, or a pane this cannot read as a box. */
public enum State { EMPTY, DRAFT, UNREADABLE }
/** A box reading: its state, and how many characters it holds ({@code 0} unless {@code DRAFT}). */
public record Reading(State state, int characters) {
}
private final AgentControl agents;
/** Consecutive holds per target, so a box that never clears can be warned about once. */
private final Map<String, Integer> holdStreaks = new ConcurrentHashMap<>();
public PromptBox(AgentControl agents) {
this.agents = agents;
}
/**
* Whether {@code target}'s input box is empty, so a delivery would submit only its own text.
* {@code false} means hold and come back; it never means the delivery failed.
*/
public boolean clearToSubmit(String target) {
Reading reading = inspect(target);
if (reading.state() == State.EMPTY) {
holdStreaks.remove(target);
return true;
}
int streak = holdStreaks.merge(target, 1, Integer::sum);
if (streak == HOLD_WARN_STREAK) {
log.warn("prompt box of {} has held a delivery {} times in a row ({}, {} character(s) in the box)"
+ " — nothing is lost, delivery resumes once the box is empty",
target, streak, reading.state(), reading.characters());
} else {
log.debug("prompt box of {} is {} ({} character(s)), holding delivery {}",
target, reading.state(), reading.characters(), streak);
}
return false;
}
/** Read and classify {@code target}'s pane. A read failure reads as {@link State#UNREADABLE}. */
private Reading inspect(String target) {
String pane;
try {
pane = agents.read(target, PROBE_SOURCE);
} catch (RuntimeException e) {
log.debug("prompt box read for {} failed, holding delivery: {}", target, e.getMessage());
return new Reading(State.UNREADABLE, 0);
}
return classify(pane);
}
/**
* Classify a Claude Code TUI pane region. Pure, so it is unit-testable without herdr.
*
* <p>{@link State#EMPTY} needs two positive signals: the pane's last box line holds nothing after
* its marker, and nothing below that line says a turn is still generating. Everything else is
* {@link State#UNREADABLE} — a blank capture, or a region with no box line at all — so a pane this
* does not understand holds the delivery rather than guessing it is safe.
*
* <p>The generating marker is looked for only from the box line down. Above it is scrollback, where
* an earlier turn's marker survives; treating that as a live turn would make {@link State#EMPTY}
* unreachable and hold every delivery forever.
*
* <p>Whitespace, a trailing box border and a cursor block count as nothing. Any other character
* counts as the operator's unsubmitted text, including a placeholder hint a future TUI might draw
* there; that direction holds a delivery it could have sent, which {@link #HOLD_WARN_STREAK} makes
* visible.
*/
static Reading classify(String pane) {
if (pane == null || pane.isBlank()) return new Reading(State.UNREADABLE, 0);
int box = lastBoxLineStart(pane);
if (box < 0) return new Reading(State.UNREADABLE, 0);
String fromBox = pane.substring(box);
if (fromBox.toLowerCase().contains(ACTIVE_TURN_MARKER)) return new Reading(State.UNREADABLE, 0);
String content = boxContent(firstLine(fromBox));
return content.isEmpty() ? new Reading(State.EMPTY, 0) : new Reading(State.DRAFT, content.length());
}
/** Offset of the last line starting with a box marker, or {@code -1} if the region has none. */
private static int lastBoxLineStart(String pane) {
int found = -1;
for (int start = 0; start <= pane.length(); ) {
int end = pane.indexOf('\n', start);
String line = pane.substring(start, end < 0 ? pane.length() : end);
if (markerLength(line) > 0) found = start;
if (end < 0) break;
start = end + 1;
}
return found;
}
/** Length of the box marker this line starts with, or {@code 0} if it starts with none. */
private static int markerLength(String line) {
for (String marker : BOX_MARKERS) {
if (line.startsWith(marker)) return marker.length();
}
return 0;
}
private static String firstLine(String text) {
int newline = text.indexOf('\n');
return newline < 0 ? text : text.substring(0, newline);
}
/** The text the box holds: its own line after the marker, stripped of border, padding and cursor. */
private static String boxContent(String boxLine) {
String line = boxLine.substring(markerLength(boxLine)).stripTrailing();
if (line.endsWith("│")) line = line.substring(0, line.length() - 1);
StringBuilder content = new StringBuilder();
for (char c : line.toCharArray()) {
if (Character.isWhitespace(c) || CURSOR_GLYPHS.indexOf(c) >= 0) continue;
content.append(c);
}
return content.toString();
}
}
@@ -4,16 +4,17 @@ import java.util.concurrent.ConcurrentHashMap;
import java.util.Set; import java.util.Set;
/** /**
* Tracks which workers are <em>available</em> — their Claude has booted and connected its MCP client * Tracks which peers are <em>available</em> — their Claude has booted and connected its MCP
* to the bridge (CB-113). This is the reliable readiness signal, unlike herdr's {@code agent_status}, * client to the bridge. For a spawned member this is the reliable readiness signal,
* which reports {@code idle} for a worker whose Claude is still booting. Delivering into that boot * unlike herdr's {@code agent_status}, which reports {@code idle} while its Claude is still
* window pastes into a not-yet-ready TUI (the text is lost) and wedges the worker's delivery state, * booting. Delivering into that boot window pastes into a not-yet-ready TUI (the text is lost)
* so the {@link Injector} holds the first delivery until the worker is present here. * and wedges that member's delivery state, so the {@link Injector} holds a spawned member's
* first delivery until it is present here.
* *
* <p>Populated from the MCP transport: any MCP request whose connection resolves to a worker terminal * <p>Populated from the MCP transport, for the peers whose deliverability rests on proving a live
* marks that worker present (its {@code initialize} is the first such contact). A worker that never * MCP contact rather than on a configured registry entry. A peer that never mounts the bridge MCP
* mounts the bridge MCP is never marked present — its sends stay queued until they time out, which is * is never marked present — its sends stay queued until they time out, which is correct (it could
* correct (it could not have replied anyway). * not have replied anyway).
*/ */
public class MemberPresence { public class MemberPresence {
@@ -18,6 +18,7 @@ import java.util.ArrayList;
import java.util.LinkedHashMap; import java.util.LinkedHashMap;
import java.util.LinkedHashSet; import java.util.LinkedHashSet;
import java.util.List; import java.util.List;
import java.util.Locale;
import java.util.Map; import java.util.Map;
import java.util.Objects; import java.util.Objects;
import java.util.Set; import java.util.Set;
@@ -83,6 +84,9 @@ public final class LeadLauncher {
private static final Logger log = LoggerFactory.getLogger(LeadLauncher.class); private static final Logger log = LoggerFactory.getLogger(LeadLauncher.class);
/** Attempts {@link #relaunch(String)} makes before giving up and returning {@code null}. */
static final int RELAUNCH_ATTEMPTS = 3;
private final AgentControl agents; private final AgentControl agents;
private final WorkspaceControl spaces; private final WorkspaceControl spaces;
private final FleetConfig cfg; private final FleetConfig cfg;
@@ -201,24 +205,14 @@ public final class LeadLauncher {
log.info("lead '{}': {} live, {} wanted — nothing to start", name, running, wanted); log.info("lead '{}': {} live, {} wanted — nothing to start", name, running, wanted);
continue; continue;
} }
if (!lead.isCreatable()) {
// A lead with a `tab:` but no `profile:` is recognise-only by design: the operator
// opens it by hand. Say so once rather than looking like a silent failure.
log.info("lead '{}' is not live, and names no profile — it can be recognised but not "
+ "launched. Add `profile:` under fleet.leaders.{} to have fleetd start it.",
name, name);
continue;
}
FleetConfig.Profile profile = cfg.profiles().get(lead.profile()); ResolvedLead resolved = resolveLaunchable(name);
if (profile == null) { if (resolved == null) {
log.warn("lead '{}' names profile '{}', which is not configured — not launching",
name, lead.profile());
continue; continue;
} }
for (int i = running; i < wanted; i++) { for (int i = running; i < wanted; i++) {
if (launch(name, lead, profile)) { if (launch(name, resolved.lead(), resolved.profile()) != null) {
started++; started++;
} }
} }
@@ -226,17 +220,89 @@ public final class LeadLauncher {
return started; return started;
} }
/** A declared lead paired with the profile it launches on — {@link #resolveLaunchable}'s result. */
private record ResolvedLead(FleetConfig.Leader lead, FleetConfig.Profile profile) {
}
/**
* The declared {@code Leader} and its {@code Profile} for {@code name}, read from the config
* snapshot this launcher was constructed with.
*
* @return the resolved pair, or {@code null} (having logged) if {@code name} is not declared
* under {@code fleet.leaders}, that lead names no {@code profile:} (a {@code tab:}-only,
* recognise-only lead), or its {@code profile:} is not configured. Shared by
* {@link #ensureLeads()} and {@link #relaunch(String)} so the three refusals and their
* wording live in one place.
*/
private ResolvedLead resolveLaunchable(String name) {
FleetConfig.Leader lead = cfg.fleet().leaders().get(name);
if (lead == null) {
log.warn("lead '{}' is not declared under fleet.leaders — not launching", name);
return null;
}
if (!lead.isCreatable()) {
// A lead with a `tab:` but no `profile:` is recognise-only by design: the operator
// opens it by hand. Say so once rather than looking like a silent failure.
log.info("lead '{}' names no profile — it can be recognised but not launched. Add "
+ "`profile:` under fleet.leaders.{} to have fleetd start it.", name, name);
return null;
}
FleetConfig.Profile profile = cfg.profiles().get(lead.profile());
if (profile == null) {
log.warn("lead '{}' names profile '{}', which is not configured — not launching",
name, lead.profile());
return null;
}
return new ResolvedLead(lead, profile);
}
/**
* Start the named lead from the config snapshot this launcher was constructed with — not a
* live read, so a lead's {@code profile:} or {@code tab:} edited in config needs a daemon
* restart to take effect here — outside of {@link #ensureLeads()}'s {@code instances}
* bookkeeping.
*
* @return the started {@link Agent}, or {@code null} if {@code name} is not declared under
* {@code fleet.leaders}, that lead names no {@code profile:} (a {@code tab:}-only,
* recognise-only lead), its {@code profile:} is not configured, or every attempt up to
* {@link #RELAUNCH_ATTEMPTS} failed to start it. Never throws.
*
* <p>Does not count how many instances of this lead are already live. {@link #ensureLeads()}'s
* count exists to avoid starting a second orchestrator; the caller of this method has already
* decided to replace the lead and owns that decision.
*
* <p>Retries the whole launch attempt — not only the {@code agent_name_taken}/
* {@code agent_pane_busy} cases {@link ResilientAgentLaunch} already retries inside one
* {@code agents.start} call — up to {@link #RELAUNCH_ATTEMPTS} times, sleeping via the
* injected sleeper between attempts, and returns the agent from the first attempt that
* succeeds.
*/
public Agent relaunch(String name) {
ResolvedLead resolved = resolveLaunchable(name);
if (resolved == null) {
return null;
}
for (int attempt = 1; attempt <= RELAUNCH_ATTEMPTS; attempt++) {
Agent started = launch(name, resolved.lead(), resolved.profile());
if (started != null) {
return started;
}
if (attempt < RELAUNCH_ATTEMPTS) {
sleeper.run();
}
}
return null;
}
/** /**
* How many live leads exist per configured name, and which of that name's labelled tabs are * How many live leads exist per configured name, and which of that name's labelled tabs are
* <em>not</em> live: a running agent in a tab labelled with that lead's exact {@code tab} * <em>not</em> live: a running agent in a tab, in that lead's own space, carrying one of its
* (CB-579). A member sitting in the same shared workspace is not counted as a lead because its * {@link FleetConfig.Leader#acceptedLabels()}. A member sitting in the same shared workspace is
* tab carries a different label, not because any workspace is excluded from this count. * not counted as a lead because its tab carries a different label, not because any workspace is
* * excluded from this count. A tab matching a lead's label in a <em>different</em> space is not
* <p>There used to be a second path here — a running agent on the terminal a * counted either — space is the uniqueness boundary between leads.
* {@code fleet.leaders.<name>.terminal} pin named, for a lead opened and pinned by hand. That
* pin is retired: {@code tab} is now the only field identity depends on, and {@link Agent}
* already carries {@link Agent#tabId()} directly, so a hand-opened lead is found the same way an
* auto-launched one is — by labelling its tab to match.
* *
* <p>fleetd #359 review finding 1: a labelled tab with nothing running in it is split into * <p>fleetd #359 review finding 1: a labelled tab with nothing running in it is split into
* {@code toClose} (already flagged pending-close by a previous reconcile, and still dead — two * {@code toClose} (already flagged pending-close by a previous reconcile, and still dead — two
@@ -250,12 +316,11 @@ public final class LeadLauncher {
} }
private Map<String, LeadCount> countLeads(Map<String, FleetConfig.Leader> leaders) { private Map<String, LeadCount> countLeads(Map<String, FleetConfig.Leader> leaders) {
// A lead and the members share ONE workspace now (the operator asked for a single "session" // A lead and the members share ONE workspace (the operator asked for a single "session" with
// with many tabs), so a workspace can no longer be excluded wholesale — the lead lives in the // many tabs), so a workspace can no longer be excluded wholesale — the lead lives in the
// member workspace by design. The sole discriminator is the exact tab label: a lead carries // member workspace by design. The discriminator is the tab label together with the space: a
// its configured `fleet.leaders.<name>.tab` ("lead: opus"), while a member carries its // member's tab never carries one of a lead's accepted labels, and a lead's own label only
// profile's `worker: {profile} #{n}` template. These never collide, so an exact-label match // counts within that lead's configured space.
// separates them without needing to know which workspace anyone is in.
Map<String, String> nameByTab = new LinkedHashMap<>(); Map<String, String> nameByTab = new LinkedHashMap<>();
Set<String> flaggedTabIds = new LinkedHashSet<>(); Set<String> flaggedTabIds = new LinkedHashSet<>();
for (Workspace ws : spaces.listWorkspaces()) { for (Workspace ws : spaces.listWorkspaces()) {
@@ -263,7 +328,7 @@ public final class LeadLauncher {
continue; continue;
} }
for (Tab tab : spaces.listTabs(ws.workspaceId())) { for (Tab tab : spaces.listTabs(ws.workspaceId())) {
String declared = leadNameOf(tab.label(), leaders); String declared = leadNameOf(tab.label(), ws.label(), leaders);
if (declared != null && tab.tabId() != null) { if (declared != null && tab.tabId() != null) {
nameByTab.put(tab.tabId(), declared); nameByTab.put(tab.tabId(), declared);
if (PendingCloseMarker.isFlagged(tab.label())) { if (PendingCloseMarker.isFlagged(tab.label())) {
@@ -313,21 +378,24 @@ public final class LeadLauncher {
} }
/** /**
* The configured lead a tab label names, or {@code null} for a label that names none. * The configured lead a tab names, or {@code null} for a label or space that names none.
* *
* <p>Matched exactly (case-insensitively) against each lead's configured {@code tab}, so an * <p>A match requires both: the label (case-insensitively, trailing {@link PendingCloseMarker}
* operator's {@code "lead: something-else"} tab is not mistaken for a configured lead. A * stripped) must be one of the lead's {@link FleetConfig.Leader#acceptedLabels()}, and {@code
* trailing {@link PendingCloseMarker} is stripped first, so a tab this class flagged on a * space} must be that lead's own {@link FleetConfig.Leader#workspace()}. The same label in a
* previous reconcile is still recognised as the same lead's tab on this one. * different space names no lead — space is the uniqueness boundary between leads.
*/ */
private String leadNameOf(String label, Map<String, FleetConfig.Leader> leaders) { private String leadNameOf(String label, String space, Map<String, FleetConfig.Leader> leaders) {
if (label == null) { if (label == null || space == null) {
return null; return null;
} }
String l = PendingCloseMarker.strip(label); String l = PendingCloseMarker.strip(label).toLowerCase(Locale.ROOT);
for (Map.Entry<String, FleetConfig.Leader> e : leaders.entrySet()) { for (Map.Entry<String, FleetConfig.Leader> e : leaders.entrySet()) {
String tab = e.getValue().tabLabel(); FleetConfig.Leader lead = e.getValue();
if (tab != null && l.equalsIgnoreCase(tab.strip())) { if (lead == null || !lead.workspace().equalsIgnoreCase(space)) {
continue;
}
if (lead.acceptedLabels().contains(l)) {
return e.getKey(); return e.getKey();
} }
} }
@@ -335,7 +403,7 @@ public final class LeadLauncher {
} }
/** /**
* Start one lead. Returns false (having logged) rather than throwing on any failure. * Start one lead. Returns null (having logged) rather than throwing on any failure.
* *
* <p>Goes through the same {@link ResilientAgentLaunch} seam every member spawn uses * <p>Goes through the same {@link ResilientAgentLaunch} seam every member spawn uses
* (fleetd #727): the assembled argv is refused outright if it cannot fit the pane line herdr * (fleetd #727): the assembled argv is refused outright if it cannot fit the pane line herdr
@@ -344,7 +412,7 @@ public final class LeadLauncher {
* relaunch, and a seed pane whose shell has not reached its prompt yet ({@code * relaunch, and a seed pane whose shell has not reached its prompt yet ({@code
* agent_pane_busy}) is retried rather than failing on the first miss. * agent_pane_busy}) is retried rather than failing on the first miss.
*/ */
private boolean launch(String name, FleetConfig.Leader lead, FleetConfig.Profile profile) { private Agent launch(String name, FleetConfig.Leader lead, FleetConfig.Profile profile) {
String label = lead.tabLabel(); String label = lead.tabLabel();
String cwd = (lead.cwd() == null || lead.cwd().isBlank()) String cwd = (lead.cwd() == null || lead.cwd().isBlank())
? System.getProperty("user.dir") : lead.cwd(); ? System.getProperty("user.dir") : lead.cwd();
@@ -375,7 +443,7 @@ public final class LeadLauncher {
log.info("lead '{}' launched: profile={} tab={} pane={} terminal={} label='{}' cwd={}", log.info("lead '{}' launched: profile={} tab={} pane={} terminal={} label='{}' cwd={}",
name, profile.profile(), tab.tab().tabId(), started.paneId(), name, profile.profile(), tab.tab().tabId(), started.paneId(),
started.terminalId(), label, cwd); started.terminalId(), label, cwd);
return true; return started;
} catch (RuntimeException e) { } catch (RuntimeException e) {
log.warn("lead '{}' failed to launch on profile '{}': {}", log.warn("lead '{}' failed to launch on profile '{}': {}",
name, profile.profile(), e.getMessage()); name, profile.profile(), e.getMessage());
@@ -387,7 +455,7 @@ public final class LeadLauncher {
tab.tab().tabId(), cleanup.getMessage()); tab.tab().tabId(), cleanup.getMessage());
} }
} }
return false; return null;
} }
} }
@@ -1,8 +1,11 @@
package dev.ltms.fleet.lead; package dev.ltms.fleet.lead;
import dev.ltms.fleet.config.FleetConfig; import dev.ltms.fleet.config.FleetConfig;
import dev.ltms.fleet.herdr.Agent;
import dev.ltms.fleet.herdr.AgentControl; import dev.ltms.fleet.herdr.AgentControl;
import dev.ltms.fleet.herdr.AgentStatus; import dev.ltms.fleet.herdr.AgentStatus;
import dev.ltms.fleet.herdr.HerdrException;
import dev.ltms.fleet.herdr.WorkspaceControl;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
@@ -24,64 +27,73 @@ import java.util.function.Supplier;
/** /**
* fleetd #480: replace a lead session that has decided it is ready to be rolled over, without an * fleetd #480: replace a lead session that has decided it is ready to be rolled over, without an
* operator doing it by hand. A lead writes a handover file, calls {@link #open}, and then — once * operator doing it by hand. A lead writes a handover file, calls {@link #open}, and then — once
* every gate ({@link #confirm}'s own checks) has passed — a deferred, single-shot continuation * every gate ({@link #confirm}'s own checks) has passed — a deferred, single-shot continuation ends
* clears the lead's own pane and bootstraps a fresh session against that file. * the lead's own pane, launches a fresh one, and bootstraps that fresh session against the file.
* *
* <p>This is the executor behind the {@code fleet_handover} MCP tool ({@code * <p>This is the executor behind the {@code fleet_handover} MCP tool ({@code
* dev.ltms.fleet.mcp.FleetMcp#handover}), which drives {@link #open}, {@link #confirm}, {@link * dev.ltms.fleet.mcp.FleetMcp#handover}), which drives {@link #open}, {@link #confirm}, {@link
* #cancel}, and {@link #status} from a tool call — wired in fleetd #480 Unit C. <strong>An earlier * #cancel}, and {@link #status} from a tool call.
* version of this paragraph said nothing called this class at all; that stopped being true once
* that unit landed, and this correction exists so the javadoc does not go on claiming it.</strong>
* *
* <p><strong>{@code confirm()} cannot roll inline — a fleetd #480 correction.</strong> The first * <p><strong>{@code confirm()} cannot roll inline.</strong> {@code confirm()} is called BY the
* version of this class called {@code agents.send(lead, "/clear")} directly from inside {@code * lead, FROM the lead's own turn: the lead's pane is {@code WORKING} for the whole duration of that
* confirm()}, then polled for the pane to become injectable again. That is wrong, because {@code * call and cannot possibly report a real turn boundary until {@code confirm()} itself returns. So
* confirm()} is called BY the lead, FROM the lead's own turn: the lead's pane is {@code WORKING} * {@link #confirm} validates every gate, then does no I/O against the lead's own pane at all — it
* for the whole duration of that call and cannot possibly report injectable until {@code confirm()} * only records that the request is approved and hands a one-shot continuation to {@code
* itself returns. The poll always timed out — but only after the {@code /clear} had already been * continuationRunner} before returning. That continuation is what actually touches the pane, once
* sent and queued in the pane, where it fired the instant the turn ended anyway. The result was the * the calling turn has ended, in this order:
* worst outcome this feature can produce: a silently destroyed lead context with no fresh session
* ever started, and a refusal return value that claimed nothing had happened.
*
* <p>The fix: {@link #confirm} validates every gate, then does no I/O against the lead's own pane
* at all — it only records that the request is approved and hands a one-shot continuation to
* {@code continuationRunner} before returning. That continuation is what actually touches the pane,
* once the calling turn has ended, in this order:
* <ol> * <ol>
* <li>wait for the lead's own pane to report a real turn boundary — {@code IDLE} or {@code * <li>wait for the lead's own pane to report a real turn boundary — {@code IDLE} or {@code
* DONE}, never merely {@code BLOCKED} — i.e. wait for the very {@code confirm()} call that * DONE}, never merely {@code BLOCKED} — i.e. wait for the very {@code confirm()} call that
* approved this roll to finish its turn — bounded by {@code turnSettleSeconds}. <strong>If * approved this roll to finish its turn — bounded by {@code turnSettleSeconds}. <strong>If
* this never happens, nothing else in this list runs: no {@code /clear} is ever sent.</strong> * this never happens, nothing else in this list runs: the old pane is never touched.</strong>
* A lead that never goes idle is a lead still doing real work, and clearing it would throw * A lead that never goes idle is a lead still doing real work, and tearing it down would throw
* away live context — exactly the failure this correction exists to prevent.</li> * away live context.</li>
* <li>{@code agents.send(lead, "/clear")}</li> * <li>capture the old pane id (and, through it, the old tab) from {@link AgentControl#get}, with
* <li>wait for {@code /clear} to be picked up and settle, bounded by {@code clearSettleSeconds} * a bounded retry — the terminal-to-pane lookup it goes through can itself report a genuinely
* (fleetd #489: no longer a plain re-check of the same boundary — {@code /clear} starts no * live agent as not found (see {@code AgentControl#agentCall}'s own re-resolve-once
* turn of its own, so this instead nudges the submit keystroke while no pickup has been seen, * behaviour), and one false negative here must not abort an otherwise-healthy roll. Neither id
* then waits for a real {@code WORKING} → {@code IDLE}/{@code DONE} boundary once one has; * is ever re-resolved from the terminal again after this — once the pane below is closed there
* see {@link #waitForClearPickupAndSettle})</li> * is nothing left to resolve it from.</li>
* <li>{@code agents.send(lead, cfg.bootstrapTextFor(p.handoverPath()))}</li> * <li>resolve the lead's configured name from its terminal, for the relaunch step below.</li>
* <li>end the old session: close the pane (an already-gone pane counts as success; any other
* failure propagates), then close its tab only when the pane was that tab's sole occupant —
* the same pane-then-tab teardown {@code HerdrPeerLauncher#stop} uses for a member.</li>
* <li>confirm the old pane is actually gone by polling {@link
* dev.ltms.fleet.herdr.WorkspaceControl#locatePane} for a {@code null} result — never {@link
* AgentControl#status}, and never the live-lead terminal map, each of which answers a
* different question. <strong>If the old pane is never confirmed gone, no relaunch is
* attempted</strong> — see {@link RollState#OLD_PANE_NEVER_DIED}.</li>
* <li>launch a fresh lead with {@code LeadLauncher#relaunch}. <strong>If every attempt fails,
* {@code bootstrapText} is never sent</strong> — see {@link RollState#RELAUNCH_FAILED}.</li>
* <li>wait for the fresh pane to reach a real turn boundary ({@code IDLE} or {@code DONE},
* never merely {@code BLOCKED}), bounded by {@code relaunchReadySeconds}. This is the
* safety gate: typing into a pane that has not actually finished booting loses the
* keystrokes. <strong>If the pane never becomes ready, {@code bootstrapText} is never
* sent</strong> — see {@link RollState#RELAUNCH_NEVER_READY}.</li>
* <li>wait for the fresh terminal to be recognised as a live lead — present in the live-lead
* terminal map — bounded by {@code relaunchReadySeconds}. This is bookkeeping, not a
* safety gate: {@code bootstrapText} is sent either way once the pane is ready, whether or
* not this wait itself times out — see {@link RollState#RELAUNCH_NOT_RECOGNISED}.</li>
* <li>{@code agents.send(newTerminal, cfg.bootstrapTextFor(p.handoverPath()))} — sent to the
* FRESH terminal, never the one that was just torn down.</li>
* </ol> * </ol>
* A {@link #confirm} that returns {@link RollDecision#approved()} therefore means <em>"every gate * A {@link #confirm} that returns {@link RollDecision#approved()} therefore means <em>"every gate
* passed and the roll is scheduled"</em>, never <em>"the pane has been cleared"</em> — the pane may * passed and the roll is scheduled"</em>, never <em>"the lead has already been replaced"</em> — the
* still be mid-turn, possibly for a long time, when the caller gets that answer back. * old pane may still be mid-turn, possibly for a long time, when the caller gets that answer back.
* *
* <p><strong>The safety invariant survives this change, restated precisely.</strong> The ticket * <p><strong>The safety invariant.</strong> "No timer, no scheduler, no background thread" means
* that first defined this class required "no timer, no scheduler, no background thread" so that * that nothing but an explicit {@link #confirm} call can ever tear a lead's pane down.
* nothing but an explicit {@link #confirm} call could ever cause a {@code /clear}. That invariant * {@code continuationRunner} launches a single-shot task that exists only because one specific,
* is about INITIATIVE, not about synchronicity, and this correction keeps it: {@code
* continuationRunner} launches a single-shot task that exists only because one specific,
* already-approved {@link #confirm} call created it — it is not recurring, it is not started at * already-approved {@link #confirm} call created it — it is not recurring, it is not started at
* construction time or on any schedule, and no two invocations of it ever share state. A recurring * construction time or on any schedule, and no two invocations of it ever share state. A recurring
* heartbeat or timer that could decide on its own initiative to roll a pane is still, and will * heartbeat or timer that could decide on its own initiative to roll a pane is absent from this
* always be, absent from this class. <strong>Nothing but an explicit {@link #confirm} call that * class. <strong>Nothing but an explicit {@link #confirm} call that passes every gate can ever tear
* passes every gate can ever cause a {@code /clear} — that call may simply finish its own work * a pane down — that call may simply finish its own work slightly later than the method return, as
* slightly later than the method return, as a continuation of the same approved request, rather * a continuation of the same approved request, rather than entirely inside the method body.</strong>
* than entirely inside the method body.</strong>
* *
* <p><strong>Identity is resolved by the caller, never looked up here — a second fleetd #480 * <p><strong>Identity is resolved by the caller, never looked up here — a second fleetd #480
* correction.</strong> The first version resolved the pane to clear via {@code * correction.</strong> The first version resolved the pane to clear via {@code
* PrimaryRegistry#primaryTerminal()}. That is correct for a background loop with no caller (see * PrimaryRegistry#currentPrimaryTerminal()}. That is correct for a background loop with no caller (see
* {@code dev.ltms.fleet.msg.LeadHeartbeatLoop}), but wrong here and a violation of this project's * {@code dev.ltms.fleet.msg.LeadHeartbeatLoop}), but wrong here and a violation of this project's
* own charter invariant 3 — "identity comes from the connection, never an argument." This daemon * own charter invariant 3 — "identity comes from the connection, never an argument." This daemon
* can hold more than one labelled lead tab (see {@code LeadLauncher}'s fleetd #359 two-reading * can hold more than one labelled lead tab (see {@code LeadLauncher}'s fleetd #359 two-reading
@@ -115,19 +127,17 @@ public final class LeadRollover {
private static final Logger log = LoggerFactory.getLogger(LeadRollover.class); private static final Logger log = LoggerFactory.getLogger(LeadRollover.class);
/** Poll interval while waiting for the lead's pane to settle after {@code /clear}. */ /** Poll interval shared by every bounded wait in this class. */
static final long SETTLE_POLL_MS = 250; static final long POLL_INTERVAL_MS = 250;
/** /**
* How many consecutive not-yet-picked-up polls {@link #waitForClearPickupAndSettle} allows * How long {@link #waitUntilPaneGone} polls {@link WorkspaceControl#locatePane} before giving
* before releasing rather than wedging the roll — the same constant and the same * up on ever seeing the old pane disappear. Not configurable: once {@link #endOldSession} has
* release-not-wedge choice {@link dev.ltms.fleet.inject.Injector} already makes for its own * closed the pane (and, usually, its tab), herdr dropping the pane from its own bookkeeping is
* post-turn {@code /clear} housekeeping (fleetd #306). <strong>This bounds the number of * expected to show up within one or two polls, not on an operator-tunable timescale the way a
* consecutive polls, not the number of nudges:</strong> the first {@code PICKUP_GRACE_POLLS - 1} * CLI boot is.
* of those polls each send a nudge, and the {@code PICKUP_GRACE_POLLS}th releases instead of
* nudging again — so 8 polls produce 7 nudges, not 8.
*/ */
static final int PICKUP_GRACE_POLLS = 8; static final int PANE_DEATH_TIMEOUT_SECONDS = 10;
/** /**
* One request opened by {@link #open}, pending its {@link #confirm} (or {@link #cancel}). * One request opened by {@link #open}, pending its {@link #confirm} (or {@link #cancel}).
@@ -139,9 +149,17 @@ public final class LeadRollover {
* calling lead's workspace, before storing it here; see this class's * calling lead's workspace, before storing it here; see this class's
* javadoc. This is the value the MCP layer hands back to the lead as * javadoc. This is the value the MCP layer hands back to the lead as
* "write your file here", so callers may rely on it always being absolute. * "write your file here", so callers may rely on it always being absolute.
* @param rolloverKey the key {@link #confirm}'s single-flight claim is taken under. {@link
* #open} resolves this once, here, from {@code leadTerminal} while the
* calling lead is certainly still live: the lead's configured name, or
* {@code leadTerminal} itself when no name resolves. Carried rather than
* recomputed at release time, because by the time a roll's continuation
* releases its claim the OLD terminal may no longer resolve to any name at
* all — recomputing there would release a different key than the one the
* claim was taken under.
*/ */
public record PendingRollover(String token, String leadTerminal, String handoverPath, public record PendingRollover(String token, String leadTerminal, String handoverPath,
long requestedAtMillis) {} long requestedAtMillis, String rolloverKey) {}
/** Which check refused a {@link #confirm} call, named so a caller can act on it. */ /** Which check refused a {@link #confirm} call, named so a caller can act on it. */
public enum RefusalReason { public enum RefusalReason {
@@ -166,19 +184,19 @@ public final class LeadRollover {
*/ */
HANDOVER_STALE, HANDOVER_STALE,
/** /**
* This lead terminal already has a roll running: an earlier {@link #confirm} call claimed * This lead already has a roll running: an earlier {@link #confirm} call claimed its
* it and that roll's continuation has not released it yet. {@code detail} names the lead * single-flight key (see {@link PendingRollover#rolloverKey}) and that roll's continuation
* terminal and the token that holds the claim. * has not released it yet. {@code detail} names the key and the token that holds the claim.
*/ */
ROLL_ALREADY_RUNNING ROLL_ALREADY_RUNNING
} }
/** /**
* The outcome of a {@link #confirm} call. {@link #approved()} means every gate passed and the * The outcome of a {@link #confirm} call. {@link #approved()} means every gate passed and the
* roll has been handed to a one-shot continuation — <strong>not</strong> that the pane has been * roll has been handed to a one-shot continuation — <strong>not</strong> that the lead has
* cleared; the continuation may still be waiting for the calling turn to end when this returns. * already been replaced; the continuation may still be waiting for the calling turn to end when
* Whether the deferred roll itself later goes on to clear the pane, refuse for never going * this returns. Whether the deferred roll itself later goes on to tear the old pane down and
* idle, or refuse for never re-settling after {@code /clear} is logged only (see this class's * relaunch the lead, or refuses at any of its own steps, is logged only (see this class's
* javadoc) — there is deliberately no synchronous caller left by that point to hand a result to. * javadoc) — there is deliberately no synchronous caller left by that point to hand a result to.
*/ */
public record RollDecision(boolean accepted, RefusalReason reason, String detail) { public record RollDecision(boolean accepted, RefusalReason reason, String detail) {
@@ -214,7 +232,7 @@ public final class LeadRollover {
/** /**
* What is known about one token, right now — the answer {@link #status} gives. Distinguishes * What is known about one token, right now — the answer {@link #status} gives. Distinguishes
* three terminal outcomes an approved roll can finish with, one in-flight outcome for a roll * five terminal outcomes an approved roll can finish with, one in-flight outcome for a roll
* that has been approved but has not finished yet, and two answers for a token that names no * that has been approved but has not finished yet, and two answers for a token that names no
* active work at all: still pending confirmation, or nothing known about this token at all. * active work at all: still pending confirmation, or nothing known about this token at all.
*/ */
@@ -237,41 +255,64 @@ public final class LeadRollover {
* #status} could wrongly answer {@link #UNKNOWN} ("nothing was ever requested") for a roll * #status} could wrongly answer {@link #UNKNOWN} ("nothing was ever requested") for a roll
* that is, in fact, actively running. This is not sticky: the deferred continuation * that is, in fact, actively running. This is not sticky: the deferred continuation
* overwrites this same entry with a terminal state ({@link #ROLLED}, {@link * overwrites this same entry with a terminal state ({@link #ROLLED}, {@link
* #TURN_NEVER_SETTLED}, {@link #CLEAR_NEVER_SETTLED}, or {@link #FAILED}) once it finishes * #TURN_NEVER_SETTLED}, {@link #OLD_PANE_NEVER_DIED}, {@link #RELAUNCH_FAILED}, {@link
* — including by throwing, which fleetd #615's catch in {@link #runRollover} now turns into * #RELAUNCH_NEVER_READY}, {@link #RELAUNCH_NOT_RECOGNISED}, or {@link #FAILED}) once it
* {@link #FAILED} instead of leaving this entry stuck forever. * finishes — including by throwing, which {@link #runRollover}'s catch turns into {@link
* #FAILED} instead of leaving this entry stuck forever.
*/ */
IN_PROGRESS, IN_PROGRESS,
/** /**
* {@link #confirm} was approved and the deferred continuation completed the entire roll: * {@link #confirm} was approved and the deferred continuation completed the entire roll: the
* the calling lead's turn settled, {@code /clear} was sent and settled, and {@code * calling lead's turn settled, the old pane was torn down and confirmed gone, a fresh lead
* bootstrapText} was sent. * was launched and recognised, and {@code bootstrapText} was sent to it.
*/ */
ROLLED, ROLLED,
/** /**
* {@link #confirm} was approved, but the calling lead's own turn never reached a boundary * {@link #confirm} was approved, but the calling lead's own turn never reached a boundary
* (IDLE or DONE) within {@code turnSettleSeconds} — no {@code /clear} was ever sent, at * (IDLE or DONE) within {@code turnSettleSeconds} — the old pane was never touched at all.
* all. This is the branch the fleetd #480 correction exists to make safe, and the one this * This is the state that makes a lead's own stuck turn VISIBLE: without it, a lead that hit
* status exists to make VISIBLE: before this, a lead that hit this case had no way to find * this case would have no way to find out, and would carry on believing it was about to be
* out, and would carry on believing it was about to be replaced. See this class's javadoc. * replaced. See this class's javadoc.
*/ */
TURN_NEVER_SETTLED, TURN_NEVER_SETTLED,
/** /**
* {@link #confirm} was approved and {@code /clear} was sent, but the pane never re-settled * {@link #confirm} was approved and the calling lead's turn settled, the old pane was closed
* within {@code clearSettleSeconds} — {@code bootstrapText} was never sent. * (and its tab, if it was the sole occupant), but {@link
* dev.ltms.fleet.herdr.WorkspaceControl#locatePane} kept reporting it as still present for
* the whole pane-death timeout. No relaunch was ever attempted, and {@code bootstrapText}
* was never sent.
*/ */
CLEAR_NEVER_SETTLED, OLD_PANE_NEVER_DIED,
/** /**
* fleetd #615: the deferred continuation threw a {@link RuntimeException} — most likely a * The old pane was confirmed gone, but {@code LeadLauncher#relaunch} returned {@code null}
* {@link dev.ltms.fleet.herdr.HerdrException} out of one of the two unwrapped {@code * — every launch attempt failed. {@code bootstrapText} was never sent, and no fresh terminal
* agents.send} calls in {@link #runRollover} — and the continuation thread died with it. * exists for this roll to have recognised.
* Before this state existed, that throw left {@link #outcomes} holding {@link #IN_PROGRESS} */
* forever, because the production {@code continuationRunner} is a bare virtual thread with RELAUNCH_FAILED,
* no uncaught-exception handler and nothing downstream of the throw ever ran to write a /**
* terminal outcome. {@code detail} names the exception, so a reader has something to act on * A fresh lead was launched, but its pane never reached a real turn boundary ({@code IDLE}
* — the same diagnostic style as {@link #TURN_NEVER_SETTLED} and {@link * or {@code DONE}, never merely {@code BLOCKED}) within {@code relaunchReadySeconds} — the
* #CLEAR_NEVER_SETTLED}. The roll is dead at this point and does not retry itself; a stuck * CLI never finished booting, or it stayed paused on a startup prompt. {@code bootstrapText}
* lead must {@link #open} a fresh request. * was never sent: typing into a pane that is not actually ready to accept input loses the
* keystrokes.
*/
RELAUNCH_NEVER_READY,
/**
* A fresh lead was launched and its pane reached a real turn boundary, so {@code
* bootstrapText} WAS sent to it, but the terminal was never recognised as a live lead —
* present in the live-lead terminal map — within {@code relaunchReadySeconds}. The session
* itself is alive and bootstrapped; only the daemon's own bookkeeping has not caught up, and
* an operator should check why the tab was not recognised.
*/
RELAUNCH_NOT_RECOGNISED,
/**
* The deferred continuation threw a {@link RuntimeException} and the continuation thread
* died with it. Without this state, that throw would leave {@link #outcomes} holding {@link
* #IN_PROGRESS} forever, because the production {@code continuationRunner} is a bare virtual
* thread with no uncaught-exception handler and nothing downstream of the throw ever runs to
* write a terminal outcome. {@code detail} names the exception, so a reader has something to
* act on. The roll is dead at this point and does not retry itself; a stuck lead must
* {@link #open} a fresh request.
*/ */
FAILED, FAILED,
/** /**
@@ -292,6 +333,10 @@ public final class LeadRollover {
public record RollStatus(RollState state, String detail) {} public record RollStatus(RollState state, String detail) {}
private final AgentControl agents; private final AgentControl agents;
/** Workspace/tab/pane control — used to tear down the old pane and confirm it is gone. */
private final WorkspaceControl spaces;
/** Starts the fresh lead that replaces the one this roll tears down. */
private final LeadLauncher launcher;
private final Supplier<FleetConfig.LeadRollover> configSupplier; private final Supplier<FleetConfig.LeadRollover> configSupplier;
/** /**
* Terminal id → that lead's configured workspace directory (their {@code * Terminal id → that lead's configured workspace directory (their {@code
@@ -301,8 +346,24 @@ public final class LeadRollover {
* daemon-cwd bug this parameter exists to fix. * daemon-cwd bug this parameter exists to fix.
*/ */
private final Function<String, String> leadWorkspace; private final Function<String, String> leadWorkspace;
/**
* Terminal id → that lead's configured name under {@code fleet.leaders}, or {@code null} when
* the terminal names no currently-recognised lead. {@link #open} calls this on the calling
* lead's own terminal, while it is certainly still live, to resolve {@link
* PendingRollover#rolloverKey}. The deferred continuation also calls this, on the OLD terminal,
* before tearing it down, so it knows which lead to pass to {@link LeadLauncher#relaunch} — by
* that point the live roster may no longer contain the old terminal, so this lookup can return
* {@code null} here even though {@link #open}'s earlier call against the same terminal did not.
*/
private final Function<String, String> leadNameForTerminal;
/**
* The daemon's current terminal id → lead name map, read fresh on every poll. The deferred
* continuation polls this for the FRESH terminal {@link LeadLauncher#relaunch} returns, to
* learn when that terminal has been recognised as a live lead — see this class's javadoc.
*/
private final Supplier<Map<String, String>> liveLeadTerminals;
private final LongSupplier nowMillis; private final LongSupplier nowMillis;
private final Runnable settleSleeper; private final Runnable pollSleeper;
/** /**
* Launches the post-{@code confirm()} continuation. Production uses a single unstarted virtual * Launches the post-{@code confirm()} continuation. Production uses a single unstarted virtual
* thread per confirmed request — see this class's javadoc for why that is a single-shot task, * thread per confirmed request — see this class's javadoc for why that is a single-shot task,
@@ -312,14 +373,14 @@ public final class LeadRollover {
private final Consumer<Runnable> continuationRunner; private final Consumer<Runnable> continuationRunner;
private final Map<String, PendingRollover> pending = new ConcurrentHashMap<>(); private final Map<String, PendingRollover> pending = new ConcurrentHashMap<>();
/** /**
* Lead terminal → the token of the roll currently holding that terminal exclusive, for * {@link PendingRollover#rolloverKey} → the token of the roll currently holding that key
* {@link #confirm}'s single-flight claim. {@link #confirm} claims an entry here with an * exclusive, for {@link #confirm}'s single-flight claim. {@link #confirm} claims an entry here
* atomic put-if-absent once every other gate has passed, refusing with {@link * with an atomic put-if-absent once every other gate has passed, refusing with {@link
* RefusalReason#ROLL_ALREADY_RUNNING} when a claim is already held; {@link #runRollover} * RefusalReason#ROLL_ALREADY_RUNNING} when a claim is already held; {@link #runRollover}
* releases it in a {@code finally}, on both the success and the thrown-exception path. A * releases it in a {@code finally}, on both the success and the thrown-exception path. A key
* terminal absent from this map has no roll currently in flight for it. * absent from this map has no roll currently in flight for it.
*/ */
private final Map<String, String> rollingByTerminal = new ConcurrentHashMap<>(); private final Map<String, String> rollingByLead = new ConcurrentHashMap<>();
/** /**
* Finished tokens → what actually happened, for {@link #status}. Bounded by {@link * Finished tokens → what actually happened, for {@link #status}. Bounded by {@link
* #OUTCOME_HISTORY_CAP}, oldest evicted first ({@code removeEldestEntry} on an insertion-order * #OUTCOME_HISTORY_CAP}, oldest evicted first ({@code removeEldestEntry} on an insertion-order
@@ -338,29 +399,40 @@ public final class LeadRollover {
} }
}); });
/** Production constructor — wall clock, real sleep between settle polls, a real virtual thread. */ /** Production constructor — wall clock, real sleep between polls, a real virtual thread. */
public LeadRollover(AgentControl agents, Supplier<FleetConfig.LeadRollover> configSupplier, public LeadRollover(AgentControl agents, WorkspaceControl spaces, LeadLauncher launcher,
Function<String, String> leadWorkspace) { Supplier<FleetConfig.LeadRollover> configSupplier,
this(agents, configSupplier, leadWorkspace, System::currentTimeMillis, Function<String, String> leadWorkspace,
() -> sleepUninterruptibly(SETTLE_POLL_MS), Function<String, String> leadNameForTerminal,
Supplier<Map<String, String>> liveLeadTerminals) {
this(agents, spaces, launcher, configSupplier, leadWorkspace, leadNameForTerminal,
liveLeadTerminals, System::currentTimeMillis,
() -> sleepUninterruptibly(POLL_INTERVAL_MS),
r -> Thread.ofVirtual().name("lead-rollover-continuation-").start(r)); r -> Thread.ofVirtual().name("lead-rollover-continuation-").start(r));
} }
/** /**
* Full constructor — an injectable wall-clock supplier, settle-poll sleeper, and continuation * Full constructor — an injectable wall-clock supplier, poll sleeper, and continuation runner,
* runner, for tests. {@code nowMillis} MUST be a wall-clock source (e.g. {@code * for tests. {@code nowMillis} MUST be a wall-clock source (e.g. {@code
* System.currentTimeMillis()}), never {@code System.nanoTime()}: the freshness check compares * System.currentTimeMillis()}), never {@code System.nanoTime()}: the freshness check compares
* against a file's modified time, which only a wall clock is comparable to, and {@code * against a file's modified time, which only a wall clock is comparable to, and {@code
* nanoTime} freezes while the host sleeps (fleetd #386). * nanoTime} freezes while the host sleeps.
*/ */
LeadRollover(AgentControl agents, Supplier<FleetConfig.LeadRollover> configSupplier, LeadRollover(AgentControl agents, WorkspaceControl spaces, LeadLauncher launcher,
Function<String, String> leadWorkspace, LongSupplier nowMillis, Supplier<FleetConfig.LeadRollover> configSupplier,
Runnable settleSleeper, Consumer<Runnable> continuationRunner) { Function<String, String> leadWorkspace,
Function<String, String> leadNameForTerminal,
Supplier<Map<String, String>> liveLeadTerminals,
LongSupplier nowMillis, Runnable pollSleeper, Consumer<Runnable> continuationRunner) {
this.agents = agents; this.agents = agents;
this.spaces = spaces;
this.launcher = launcher;
this.configSupplier = configSupplier; this.configSupplier = configSupplier;
this.leadWorkspace = leadWorkspace; this.leadWorkspace = leadWorkspace;
this.leadNameForTerminal = leadNameForTerminal;
this.liveLeadTerminals = liveLeadTerminals;
this.nowMillis = nowMillis; this.nowMillis = nowMillis;
this.settleSleeper = settleSleeper; this.pollSleeper = pollSleeper;
this.continuationRunner = continuationRunner; this.continuationRunner = continuationRunner;
} }
@@ -377,8 +449,9 @@ public final class LeadRollover {
/** /**
* The lead says it is ready to be replaced. Generates a token and records the resolved * The lead says it is ready to be replaced. Generates a token and records the resolved
* handover path, this moment's wall-clock timestamp (the baseline {@link #confirm} checks the * handover path, this moment's wall-clock timestamp (the baseline {@link #confirm} checks the
* handover file's modified time against), and {@code leadTerminal} — only that exact terminal * handover file's modified time against), {@code leadTerminal} — only that exact terminal may
* may later {@link #confirm} this token. * later {@link #confirm} this token — and {@link PendingRollover#rolloverKey}, resolved here
* from {@code leadTerminal} while the calling lead is certainly still live.
* *
* @param leadTerminal the calling lead's terminal id, resolved by the MCP layer from the * @param leadTerminal the calling lead's terminal id, resolved by the MCP layer from the
* connection (see this class's javadoc) — never a client-supplied value * connection (see this class's javadoc) — never a client-supplied value
@@ -398,7 +471,9 @@ public final class LeadRollover {
String token = UUID.randomUUID().toString(); String token = UUID.randomUUID().toString();
long requestedAt = nowMillis.getAsLong(); long requestedAt = nowMillis.getAsLong();
String resolvedPath = resolveHandoverPath(cfg.handoverPath(), leadTerminal); String resolvedPath = resolveHandoverPath(cfg.handoverPath(), leadTerminal);
PendingRollover p = new PendingRollover(token, leadTerminal, resolvedPath, requestedAt); String leadName = leadNameForTerminal.apply(leadTerminal);
String rolloverKey = (leadName == null || leadName.isBlank()) ? leadTerminal : leadName;
PendingRollover p = new PendingRollover(token, leadTerminal, resolvedPath, requestedAt, rolloverKey);
pending.put(token, p); pending.put(token, p);
if (resolvedPath.equals(cfg.handoverPath())) { if (resolvedPath.equals(cfg.handoverPath())) {
log.info("lead-rollover: open token={} lead={} handoverPath={} reason={}", log.info("lead-rollover: open token={} lead={} handoverPath={} reason={}",
@@ -504,12 +579,11 @@ public final class LeadRollover {
// Single-flight claim: atomic put-if-absent, taken only after every other gate has // Single-flight claim: atomic put-if-absent, taken only after every other gate has
// passed, so a refused confirm() never takes it. A non-null previous value means a // passed, so a refused confirm() never takes it. A non-null previous value means a
// different, still-running roll already holds this lead terminal. // different, still-running roll already holds this lead's claim.
String holder = rollingByTerminal.putIfAbsent(p.leadTerminal(), token); String holder = rollingByLead.putIfAbsent(p.rolloverKey(), token);
if (holder != null) { if (holder != null) {
return RollDecision.refused(RefusalReason.ROLL_ALREADY_RUNNING, return RollDecision.refused(RefusalReason.ROLL_ALREADY_RUNNING,
"lead terminal " + p.leadTerminal() + " already has a roll running under token " "lead '" + p.rolloverKey() + "' already has a roll running under token " + holder);
+ holder);
} }
// Record IN_PROGRESS BEFORE removing from `pending` — see RollState#IN_PROGRESS and // Record IN_PROGRESS BEFORE removing from `pending` — see RollState#IN_PROGRESS and
@@ -519,8 +593,9 @@ public final class LeadRollover {
// remove-then-put ordering would leave in which the token is in neither map. // remove-then-put ordering would leave in which the token is in neither map.
outcomes.put(token, new RollStatus(RollState.IN_PROGRESS, outcomes.put(token, new RollStatus(RollState.IN_PROGRESS,
"confirm() approved this roll and handed it to the deferred continuation; it has " "confirm() approved this roll and handed it to the deferred continuation; it has "
+ "not finished yet — still waiting for the calling turn to settle, for " + "not finished yet — still waiting for the calling turn to settle, for the "
+ "/clear to be sent and settle, or for bootstrapText to be sent")); + "old pane to be torn down and confirmed gone, for the fresh lead to be "
+ "recognised, or for bootstrapText to be sent"));
pending.remove(token); pending.remove(token);
log.info("lead-rollover: confirmed token={} lead={} — roll scheduled once the calling turn ends", log.info("lead-rollover: confirmed token={} lead={} — roll scheduled once the calling turn ends",
token, callerTerminal); token, callerTerminal);
@@ -529,14 +604,14 @@ public final class LeadRollover {
} catch (RuntimeException e) { } catch (RuntimeException e) {
// continuationRunner can reject the hand-off itself (e.g. a bounded executor's // continuationRunner can reject the hand-off itself (e.g. a bounded executor's
// RejectedExecutionException) before runRollover ever starts, so runRollover's own // RejectedExecutionException) before runRollover ever starts, so runRollover's own
// finally — the only other place that releases rollingByTerminal — never runs either. // finally — the only other place that releases rollingByLead — never runs either.
// Release the claim here and overwrite the IN_PROGRESS entry with a terminal outcome, // Release the claim here and overwrite the IN_PROGRESS entry with a terminal outcome,
// or this lead terminal could never be rolled again and status() would report // or this lead could never be rolled again and status() would report IN_PROGRESS
// IN_PROGRESS forever for a roll that in fact never started. // forever for a roll that in fact never started.
log.warn("lead-rollover: continuationRunner rejected token={} lead={}: {} — the roll " log.warn("lead-rollover: continuationRunner rejected token={} lead={}: {} — the roll "
+ "never started; releasing its claim and reporting it as FAILED", + "never started; releasing its claim and reporting it as FAILED",
token, callerTerminal, e.toString(), e); token, callerTerminal, e.toString(), e);
rollingByTerminal.remove(p.leadTerminal(), token); rollingByLead.remove(p.rolloverKey(), token);
outcomes.put(token, new RollStatus(RollState.FAILED, outcomes.put(token, new RollStatus(RollState.FAILED,
"continuationRunner rejected this roll before it ever started: " + e.toString() "continuationRunner rejected this roll before it ever started: " + e.toString()
+ " — the roll never ran; open() a fresh rollover request")); + " — the roll never ran; open() a fresh rollover request"));
@@ -547,22 +622,19 @@ public final class LeadRollover {
/** /**
* The single-shot continuation {@link #confirm} hands to {@code continuationRunner}. Runs * The single-shot continuation {@link #confirm} hands to {@code continuationRunner}. Runs
* entirely after {@link #confirm} has returned to its caller — see this class's javadoc for the * entirely after {@link #confirm} has returned to its caller — see this class's javadoc for the
* four-step order. There is no result to return to by this point, so every outcome is logged * full order. There is no result to return to by this point, so every outcome is logged only.
* only.
* *
* <p><strong>fleetd #615 — the whole body is wrapped in one {@code try}.</strong> The two {@code * <p><strong>The whole body is wrapped in one {@code try}.</strong> Several calls below —
* agents.send} calls below are not wrapped individually: {@code send} → {@code agentCall} → * {@code agents.get}, {@code agents.close}, {@code agents.send} — can throw an unchecked {@link
* {@code herdr.call} can throw an unchecked {@link dev.ltms.fleet.herdr.HerdrException} (see * dev.ltms.fleet.herdr.HerdrException} (see {@code AgentControl.java}), and the production
* {@code AgentControl.java}), and the production {@code continuationRunner} is a bare virtual * {@code continuationRunner} is a bare virtual thread with no uncaught-exception handler (see
* thread with no uncaught-exception handler (see this class's public constructor). Before this * this class's public constructor). An uncaught throw would kill the continuation thread
* fix, either throw killed the continuation thread silently, leaving the {@link * silently, leaving the {@link RollState#IN_PROGRESS} entry {@link #confirm} wrote at hand-off
* RollState#IN_PROGRESS} entry {@link #confirm} wrote at hand-off stuck forever — {@link * stuck forever — {@link #status} would have no way to tell a dead roll from one still
* #status} had no way to tell a dead roll from one still genuinely running. The {@code catch} * genuinely running. The {@code catch} below is scoped to the method body rather than to each
* below is scoped to the method body rather than to each {@code send} call individually, so it * call individually, so it also covers every call in this continuation, not a fixed list of
* also covers anything else added to this continuation later, not just today's two call sites — * call sites — the same reasoning that put the write-a-terminal-outcome step at each of this
* the same reasoning that put the write-a-terminal-outcome step at each of this method's other * method's other exits rather than inside the helpers that detect them.</p>
* exits (see the {@link RollState#TURN_NEVER_SETTLED} and {@link RollState#CLEAR_NEVER_SETTLED}
* branches below) rather than inside the helpers that detect them.</p>
* *
* <p>Only {@link RuntimeException} is caught, matching the local convention {@link * <p>Only {@link RuntimeException} is caught, matching the local convention {@link
* #waitUntilAtTurnBoundary} already set around its own {@code agents.status} call — not the * #waitUntilAtTurnBoundary} already set around its own {@code agents.status} call — not the
@@ -582,10 +654,10 @@ public final class LeadRollover {
+ "a fresh rollover request")); + "a fresh rollover request"));
} finally { } finally {
// Release the single-flight claim on both the normal return and the thrown-exception // Release the single-flight claim on both the normal return and the thrown-exception
// path above — a release only on success would leave this lead terminal unrollable // path above — a release only on success would leave this lead unrollable forever
// forever after one failure. The conditional two-argument remove only clears the // after one failure. The conditional two-argument remove only clears the entry this
// entry this roll itself holds, never a different roll's claim on the same terminal. // roll itself holds, never a different roll's claim on the same key.
rollingByTerminal.remove(p.leadTerminal(), p.token()); rollingByLead.remove(p.rolloverKey(), p.token());
} }
} }
@@ -593,53 +665,237 @@ public final class LeadRollover {
private void runRolloverUnguarded(PendingRollover p, FleetConfig.LeadRollover cfg) { private void runRolloverUnguarded(PendingRollover p, FleetConfig.LeadRollover cfg) {
String lead = p.leadTerminal(); String lead = p.leadTerminal();
long rollStartMillis = nowMillis.getAsLong(); long rollStartMillis = nowMillis.getAsLong();
TurnSettleResult turnResult = waitUntilAtTurnBoundary(lead, cfg.turnSettleSeconds()); TurnSettleResult turnResult = waitUntilAtTurnBoundary(lead, cfg.turnSettleSeconds());
if (!turnResult.settled()) { if (!turnResult.settled()) {
// fleetd #494 follow-up: this line had the SAME defect as the /clear-timeout line below
// — cfg.turnSettleSeconds() is the CONFIGURED budget, not how long this wait actually
// ran. Print the measured elapsed time alongside it, labelled, exactly like the /clear
// path already does.
log.warn("lead-rollover: pane {} did not reach a turn boundary (IDLE or DONE) after " log.warn("lead-rollover: pane {} did not reach a turn boundary (IDLE or DONE) after "
+ "confirm() — refusing to send /clear at all; the calling lead's own " + "confirm() — the old pane is never touched; the calling lead's own "
+ "turn is still live and clearing it now would destroy live context " + "turn is still live and tearing it down now would destroy live context "
+ "(token={}, configured={}s elapsed={}ms)", + "(token={}, configured={}s elapsed={}ms)",
lead, p.token(), cfg.turnSettleSeconds(), turnResult.elapsedMillis()); lead, p.token(), cfg.turnSettleSeconds(), turnResult.elapsedMillis());
outcomes.put(p.token(), new RollStatus(RollState.TURN_NEVER_SETTLED, outcomes.put(p.token(), new RollStatus(RollState.TURN_NEVER_SETTLED,
"the calling lead's own turn never reached a boundary (IDLE or DONE) within " "the calling lead's own turn never reached a boundary (IDLE or DONE) within "
+ "turnSettleSeconds=" + cfg.turnSettleSeconds() + "s (measured elapsed=" + "turnSettleSeconds=" + cfg.turnSettleSeconds() + "s (measured elapsed="
+ turnResult.elapsedMillis() + "ms) — no /clear was ever sent. If this " + turnResult.elapsedMillis() + "ms) — the old pane was never touched. If "
+ "keeps happening, raise turnSettleSeconds in fleetd.yaml")); + "this keeps happening, raise turnSettleSeconds in fleetd.yaml"));
return; return;
} }
// This deliberately bypasses Injector, exactly like ClaudeCodeLauncher#clearContext: // Captured once, here, and never re-resolved from `lead` again below: once the pane is
// /clear is housekeeping, not a delegated turn, and routing it through Injector wedges the // closed there is nothing left for a terminal lookup to find.
// pane forever (see this class's javadoc). Agent oldAgent = captureAgentWithRetry(lead);
agents.send(lead, "/clear"); String oldPaneId = oldAgent.paneId();
ClearSettleResult clearResult = waitForClearPickupAndSettle(lead, cfg.clearSettleSeconds()); String leadName = leadNameForTerminal.apply(lead);
if (!clearResult.settled()) {
// fleetd #494: cfg.clearSettleSeconds() is the CONFIGURED budget, not how long the wait endOldSession(oldPaneId);
// actually ran — an operator reading only that number wrongly believes it is a measured DeathResult deathResult = waitUntilPaneGone(oldPaneId);
// duration. Print the measured elapsed time and nudge count alongside it, each labelled, if (!deathResult.gone()) {
// so the two can be compared at a glance. log.warn("lead-rollover: old pane {} for lead {} was never confirmed gone after being "
log.warn("lead-rollover: pane {} did not reach a turn boundary (IDLE or DONE) after " + "closed — not attempting a relaunch (token={}, timeout={}s "
+ "/clear — NOT sending bootstrapText (token={}, configured={}s " + "elapsed={}ms)",
+ "elapsed={}ms nudges={})", oldPaneId, lead, p.token(), PANE_DEATH_TIMEOUT_SECONDS, deathResult.elapsedMillis());
lead, p.token(), cfg.clearSettleSeconds(), clearResult.elapsedMillis(), outcomes.put(p.token(), new RollStatus(RollState.OLD_PANE_NEVER_DIED,
clearResult.nudges()); "the old pane was closed, but locatePane kept reporting it as still present "
outcomes.put(p.token(), new RollStatus(RollState.CLEAR_NEVER_SETTLED, + "after a pane-death timeout=" + PANE_DEATH_TIMEOUT_SECONDS
"/clear was sent, but the pane never re-settled within clearSettleSeconds=" + "s (measured elapsed=" + deathResult.elapsedMillis() + "ms) — no "
+ cfg.clearSettleSeconds() + "s (measured elapsed=" + clearResult.elapsedMillis() + "relaunch was attempted"));
+ "ms, nudges=" + clearResult.nudges() + ") — bootstrapText was never sent"));
return; return;
} }
agents.send(lead, cfg.bootstrapTextFor(p.handoverPath()));
Agent newAgent = launcher.relaunch(leadName);
if (newAgent == null) {
log.warn("lead-rollover: relaunch of lead '{}' (old terminal {}) failed every attempt "
+ "— bootstrapText was never sent (token={})", leadName, lead, p.token());
outcomes.put(p.token(), new RollStatus(RollState.RELAUNCH_FAILED,
"lead '" + leadName + "' could not be relaunched — every attempt failed; "
+ "bootstrapText was never sent"));
return;
}
ReadinessResult readinessResult = waitUntilPaneReady(newAgent.terminalId(),
cfg.relaunchReadySeconds());
if (!readinessResult.ready()) {
log.warn("lead-rollover: fresh pane for lead '{}' (terminal {}) never reached a real "
+ "turn boundary — bootstrapText was never sent (token={}, configured={}s "
+ "elapsed={}ms)",
leadName, newAgent.terminalId(), p.token(), cfg.relaunchReadySeconds(),
readinessResult.elapsedMillis());
outcomes.put(p.token(), new RollStatus(RollState.RELAUNCH_NEVER_READY,
"fresh terminal " + newAgent.terminalId() + " never reached a real turn "
+ "boundary (IDLE or DONE) within relaunchReadySeconds="
+ cfg.relaunchReadySeconds() + "s (measured elapsed="
+ readinessResult.elapsedMillis() + "ms) — bootstrapText was never "
+ "sent"));
return;
}
IdentityResult identityResult = waitUntilRecognisedAsLead(newAgent.terminalId(),
cfg.relaunchReadySeconds());
agents.send(newAgent.terminalId(), cfg.bootstrapTextFor(p.handoverPath()));
if (!identityResult.ready()) {
log.warn("lead-rollover: fresh terminal {} for lead '{}' is alive and bootstrapped, but "
+ "was never recognised as a live lead — an operator should check why "
+ "the tab was not recognised (token={}, configured={}s elapsed={}ms)",
newAgent.terminalId(), leadName, p.token(), cfg.relaunchReadySeconds(),
identityResult.elapsedMillis());
outcomes.put(p.token(), new RollStatus(RollState.RELAUNCH_NOT_RECOGNISED,
"bootstrapText was sent to fresh terminal " + newAgent.terminalId() + ", but "
+ "it was never recognised as a live lead within relaunchReadySeconds="
+ cfg.relaunchReadySeconds() + "s (measured elapsed="
+ identityResult.elapsedMillis() + "ms) — check why the tab was not "
+ "recognised"));
return;
}
long rollElapsedMillis = nowMillis.getAsLong() - rollStartMillis; long rollElapsedMillis = nowMillis.getAsLong() - rollStartMillis;
log.info("lead-rollover: rolled token={} lead={} elapsedMs={}", p.token(), lead, rollElapsedMillis); log.info("lead-rollover: rolled token={} oldLead={} newTerminal={} elapsedMs={}",
p.token(), lead, newAgent.terminalId(), rollElapsedMillis);
outcomes.put(p.token(), new RollStatus(RollState.ROLLED, outcomes.put(p.token(), new RollStatus(RollState.ROLLED,
"rolled successfully in " + rollElapsedMillis + "ms")); "rolled successfully in " + rollElapsedMillis + "ms; new terminal="
+ newAgent.terminalId()));
} }
/** Attempts {@link #captureAgentWithRetry} makes before letting the failure propagate. */
static final int CAPTURE_RETRIES = 3;
/**
* {@link AgentControl#get} for {@code lead}, retried up to {@link #CAPTURE_RETRIES} times. The
* terminal-to-pane lookup it goes through can report a genuinely live agent as not found (see
* {@code AgentControl#agentCall}'s own re-resolve-once behaviour), and one such false negative
* must not abort an otherwise-healthy roll. The result is captured once by the caller and never
* looked up again — see this class's javadoc.
*
* @throws RuntimeException the last failure, if every attempt fails — {@link #runRollover}'s
* catch turns that into {@link RollState#FAILED}
*/
private Agent captureAgentWithRetry(String lead) {
RuntimeException last = null;
for (int attempt = 1; attempt <= CAPTURE_RETRIES; attempt++) {
try {
return agents.get(lead);
} catch (RuntimeException e) {
last = e;
log.debug("lead-rollover: agents.get({}) failed on attempt {}/{}: {}",
lead, attempt, CAPTURE_RETRIES, e.toString());
if (attempt < CAPTURE_RETRIES) {
pollSleeper.run();
}
}
}
throw last;
}
/**
* End the old lead's session: close its pane, then close its tab only when the pane was that
* tab's sole occupant — the same pane-then-tab teardown {@code HerdrPeerLauncher#stop} uses for
* a member. An already-gone pane counts as success; any other {@code agents.close} failure
* propagates, so a genuinely failed teardown is never reported as done. A failing
* {@code spaces.closeTab} never propagates — by the time it runs the pane is already closed, so
* it is cosmetic tidying, not a real teardown failure.
*/
private void endOldSession(String paneId) {
WorkspaceControl.PaneLocation loc = spaces.locatePane(paneId);
try {
agents.close(paneId);
} catch (HerdrException e) {
if (!isAlreadyGone(e)) {
throw e;
}
log.debug("lead-rollover: pane.close({}) ignored — already gone: {}", paneId, e.getMessage());
}
if (loc != null && loc.tabPaneCount() == 1) {
try {
spaces.closeTab(loc.tabId());
} catch (RuntimeException e) {
log.warn("lead-rollover: tab.close({}) failed — the pane is already torn down, so "
+ "continuing; the tab may need manual cleanup: {}", loc.tabId(), e.getMessage());
}
} else if (loc != null) {
log.debug("lead-rollover: not closing tab {} — it holds {} panes (not a dedicated lead "
+ "tab)", loc.tabId(), loc.tabPaneCount());
}
}
/** True when a herdr error means the target is already gone (safe to treat as done). */
private static boolean isAlreadyGone(HerdrException e) {
return e.code() != null && e.code().endsWith("_not_found");
}
/**
* Poll {@link WorkspaceControl#locatePane} for {@code paneId} until it reports {@code null}
* (the pane is gone) or {@link #PANE_DEATH_TIMEOUT_SECONDS} elapses. Deliberately never calls
* {@link AgentControl#status} and never reads the live-lead terminal map — both answer a
* different question (whether an AGENT is live, not whether this PANE still exists) and
* {@code locatePane} alone catches a {@link HerdrException} from the underlying {@code
* pane.get} and turns it into {@code null} — see this class's javadoc.
*/
private DeathResult waitUntilPaneGone(String paneId) {
long startMillis = nowMillis.getAsLong();
long deadline = startMillis + TimeUnit.SECONDS.toMillis(PANE_DEATH_TIMEOUT_SECONDS);
while (nowMillis.getAsLong() < deadline) {
if (spaces.locatePane(paneId) == null) {
return new DeathResult(true, nowMillis.getAsLong() - startMillis);
}
pollSleeper.run();
}
return new DeathResult(false, nowMillis.getAsLong() - startMillis);
}
/** The measured outcome of {@link #waitUntilPaneGone}. */
private record DeathResult(boolean gone, long elapsedMillis) {}
/**
* Poll until {@code newTerminal}'s own pane reaches a real turn boundary ({@link
* AgentStatus#IDLE} or {@link AgentStatus#DONE}, never merely {@link AgentStatus#BLOCKED}) —
* the same exclusion {@link #waitUntilAtTurnBoundary} applies to the calling lead's own turn,
* applied here to the fresh one, so {@code bootstrapText} is never typed into a pane that has
* not actually finished booting — or {@code readySeconds} elapses. A failed status read
* degrades to "not yet ready" and is retried on the next poll.
*/
private ReadinessResult waitUntilPaneReady(String newTerminal, int readySeconds) {
long startMillis = nowMillis.getAsLong();
long deadline = startMillis + TimeUnit.SECONDS.toMillis(readySeconds);
while (nowMillis.getAsLong() < deadline) {
AgentStatus status;
try {
status = agents.status(newTerminal);
} catch (RuntimeException e) {
log.debug("lead-rollover: status check failed while waiting for {} to be ready: {}",
newTerminal, e.toString());
status = null;
}
if (status == AgentStatus.IDLE || status == AgentStatus.DONE) {
return new ReadinessResult(true, nowMillis.getAsLong() - startMillis);
}
pollSleeper.run();
}
return new ReadinessResult(false, nowMillis.getAsLong() - startMillis);
}
/** The measured outcome of {@link #waitUntilPaneReady}. */
private record ReadinessResult(boolean ready, long elapsedMillis) {}
/**
* Poll until {@code newTerminal} is present in {@link #liveLeadTerminals} or {@code
* readySeconds} elapses. This is bookkeeping, not a safety gate: the pane's own readiness (see
* {@link #waitUntilPaneReady}) is what decides whether {@code bootstrapText} is safe to send —
* a timeout here only means the daemon's own lead-discovery scan has not caught up yet.
*/
private IdentityResult waitUntilRecognisedAsLead(String newTerminal, int readySeconds) {
long startMillis = nowMillis.getAsLong();
long deadline = startMillis + TimeUnit.SECONDS.toMillis(readySeconds);
while (nowMillis.getAsLong() < deadline) {
if (liveLeadTerminals.get().containsKey(newTerminal)) {
return new IdentityResult(true, nowMillis.getAsLong() - startMillis);
}
pollSleeper.run();
}
return new IdentityResult(false, nowMillis.getAsLong() - startMillis);
}
/** The measured outcome of {@link #waitUntilRecognisedAsLead}. */
private record IdentityResult(boolean ready, long elapsedMillis) {}
/** Drop a pending request without rolling. @return whether a pending request existed for {@code token} */ /** Drop a pending request without rolling. @return whether a pending request existed for {@code token} */
public boolean cancel(String token) { public boolean cancel(String token) {
return pending.remove(token) != null; return pending.remove(token) != null;
@@ -729,15 +985,12 @@ public final class LeadRollover {
/** /**
* Poll {@link AgentControl#status} until {@code target} reports a real turn boundary — {@link * Poll {@link AgentControl#status} until {@code target} reports a real turn boundary — {@link
* AgentStatus#IDLE} or {@link AgentStatus#DONE} — bounded by {@code settleSeconds}. Used once by * AgentStatus#IDLE} or {@link AgentStatus#DONE} — bounded by {@code settleSeconds}. Used by
* {@link #runRollover}, to wait for the CALLING turn's own pane to settle before {@code /clear} * {@link #runRollover} to wait for the CALLING turn's own pane to settle before the old pane is
* is ever sent at all — the {@code turnSettleSeconds} gate that makes this correction safe. The * touched at all — the {@code turnSettleSeconds} gate that makes tearing it down safe. A failed
* SECOND wait, after {@code /clear}, is {@link #waitForClearPickupAndSettle} instead (fleetd * status read degrades to "not yet settled" and is retried on the next poll, the same posture
* #489) — a plain boundary check is not enough there, because {@code /clear} starts no turn of * {@code LeadHeartbeatLoop} and {@code HerdrPeerLauncher}'s readiness gate already take toward
* its own, so this method would (wrongly) report "settled" on its very first poll whether or not * an unreadable status.
* {@code /clear} was actually picked up. A failed status read degrades to "not yet settled" and
* is retried on the next poll, the same posture {@code LeadHeartbeatLoop} and {@code
* HerdrPeerLauncher}'s readiness gate already take toward an unreadable status.
* *
* <p><strong>Deliberately not {@link AgentStatus#injectable()}.</strong> {@code injectable()} * <p><strong>Deliberately not {@link AgentStatus#injectable()}.</strong> {@code injectable()}
* answers the {@code Injector}'s question — "may I deliver a message without stepping on a live * answers the {@code Injector}'s question — "may I deliver a message without stepping on a live
@@ -745,16 +998,16 @@ public final class LeadRollover {
* an approval prompt is safe to queue a message behind. This class asks a stricter question — * an approval prompt is safe to queue a message behind. This class asks a stricter question —
* "has the turn actually ended" — and {@code BLOCKED} answers no: it is a live turn that is * "has the turn actually ended" — and {@code BLOCKED} answers no: it is a live turn that is
* merely paused, not one that has finished. Reusing {@code injectable()} here would let this * merely paused, not one that has finished. Reusing {@code injectable()} here would let this
* wait fire {@code /clear} while the lead's own {@code confirm()}-calling turn is still live and * wait tear the old pane down while the lead's own {@code confirm()}-calling turn is still live
* paused on a prompt — exactly the live-context-destroying failure the {@code turnSettleSeconds} * and paused on a prompt — exactly the live-context-destroying failure {@code turnSettleSeconds}
* gate exists to prevent. Do not "simplify" this back to {@code injectable()}. ({@link * exists to prevent. Do not "simplify" this back to {@code injectable()}. ({@link
* #waitForClearPickupAndSettle} keeps the same exclusion of {@code BLOCKED}, for the same * #waitUntilPaneReady} applies the same exclusion of {@code BLOCKED} to the fresh lead's own
* reason, on the second wait.) * turn.)
* *
* @return a {@link TurnSettleResult} whose {@code settled()} is {@code true} once a real * @return a {@link TurnSettleResult} whose {@code settled()} is {@code true} once a real
* boundary was observed, {@code false} if {@code settleSeconds} elapses first. * boundary was observed, {@code false} if {@code settleSeconds} elapses first.
* {@code elapsedMillis()} is a MEASURED value from the injected {@link #nowMillis} * {@code elapsedMillis()} is a MEASURED value from the injected {@link #nowMillis}
* clock, never the configured {@code settleSeconds} budget (fleetd #494 follow-up). * clock, never the configured {@code settleSeconds} budget.
*/ */
private TurnSettleResult waitUntilAtTurnBoundary(String target, int settleSeconds) { private TurnSettleResult waitUntilAtTurnBoundary(String target, int settleSeconds) {
long startMillis = nowMillis.getAsLong(); long startMillis = nowMillis.getAsLong();
@@ -771,142 +1024,11 @@ public final class LeadRollover {
if (status == AgentStatus.IDLE || status == AgentStatus.DONE) { if (status == AgentStatus.IDLE || status == AgentStatus.DONE) {
return new TurnSettleResult(true, nowMillis.getAsLong() - startMillis); return new TurnSettleResult(true, nowMillis.getAsLong() - startMillis);
} }
settleSleeper.run(); pollSleeper.run();
} }
return new TurnSettleResult(false, nowMillis.getAsLong() - startMillis); return new TurnSettleResult(false, nowMillis.getAsLong() - startMillis);
} }
/** /** The measured outcome of {@link #waitUntilAtTurnBoundary}. */
* The measured outcome of {@link #waitUntilAtTurnBoundary} — fleetd #494 follow-up. The sibling
* of {@link ClearSettleResult} for the FIRST wait, which never nudges, so it carries no nudge
* count.
*/
private record TurnSettleResult(boolean settled, long elapsedMillis) {} private record TurnSettleResult(boolean settled, long elapsedMillis) {}
/**
* The SECOND wait in {@link #runRollover} — after {@code /clear} has been sent, waits for it to
* settle, bounded by {@code settleSeconds}. <strong>fleetd #489 — the paste-race fix.</strong>
* {@code /clear} does not start a real turn of its own, so a pane with no submit race simply
* stays {@link AgentStatus#IDLE} the whole time: {@link #waitUntilAtTurnBoundary} would (wrongly)
* call that "settled" on its very first poll, whether or not the {@code /clear} Enter actually
* landed. That was Fault 1, measured live on 2026-09-12 — the second gate was a no-op, so a
* {@code bootstrapText} send followed immediately, racing Fault 2: {@link AgentControl#submit}'s
* own javadoc already records that the submit accompanying a delivery "can race the paste —
* especially right as the worker's TUI becomes interactive — leaving the text unsubmitted"
* (CB-113). Because {@code runRollover} deliberately bypasses {@code Injector} for {@code
* /clear} (see this class's javadoc), it inherited none of {@code Injector}'s nudging — so the
* lost {@code /clear} Enter sat in the input box and {@code bootstrapText} was typed right after
* it, landing as one concatenated line.
*
* <p>This method copies the pickup-nudge pattern {@link dev.ltms.fleet.inject.Injector} already
* ships for exactly this, on its own post-turn {@code /clear} housekeeping (fleetd #306; see
* {@code Injector.java:288-340} and {@code Injector.java:437-442}):
* <ul>
* <li>an {@link AgentStatus#WORKING} sample means {@code /clear} was picked up as a real
* turn;</li>
* <li>until that happens, each poll that still reports {@link AgentStatus#IDLE} or {@link
* AgentStatus#DONE} re-sends the submit keystroke ({@link AgentControl#submit}) to nudge
* the raced Enter — for the first {@code PICKUP_GRACE_POLLS - 1} of {@link
* #PICKUP_GRACE_POLLS} consecutive such polls (i.e. {@code PICKUP_GRACE_POLLS - 1}
* nudges: 7, not 8, given {@code PICKUP_GRACE_POLLS = 8}). A second Enter on an empty
* Claude Code prompt is a no-op, so repeating it is safe;</li>
* <li>the {@code PICKUP_GRACE_POLLS}th consecutive such poll, with {@code WORKING} still never
* observed, releases rather than wedges the roll instead of nudging again — the same
* choice {@code Injector} makes — and returns {@code settled() == true} anyway, logged at
* {@code warn} with the measured elapsed time (fleetd #494) so an operator can see which
* path ran and how long it actually took;</li>
* <li>once {@code WORKING} has been observed, nudging stops and this instead waits for a real
* {@code working → IDLE/DONE} completion boundary before returning {@code true}.</li>
* </ul>
*
* <p><strong>{@link AgentStatus#BLOCKED} is deliberately excluded from both the nudge and the
* boundary check</strong> — the same reasoning as {@link #waitUntilAtTurnBoundary}'s own
* javadoc: a paused live turn is not a settled one, and re-sending Enter into an open approval
* prompt could wrongly answer it. A {@code BLOCKED} sample (or an unreadable/{@link
* AgentStatus#UNKNOWN} one) simply keeps this polling, with no nudge and no release, until either
* a real boundary is reached or {@code settleSeconds} runs out.
*
* <p>{@link AgentControl#submit} can itself throw; a {@link RuntimeException} from it is
* swallowed and logged at {@code debug}, exactly like {@code Injector.java:437-442} — a failed
* nudge must not abort the roll.
*
* @return a {@link ClearSettleResult} whose {@code settled()} is {@code true} once {@code
* /clear} has settled, or once the nudge budget was exhausted with no pickup ever
* observed (released rather than wedged); {@code false} if {@code settleSeconds} elapses
* first — the caller must NOT send {@code bootstrapText} in that case, exactly as before
* this fix. {@code elapsedMillis()} and {@code nudges()} are MEASURED values (from the
* injected {@link #nowMillis} clock and an actual nudge count), never the configured
* {@code settleSeconds} budget (fleetd #494).
*/
private ClearSettleResult waitForClearPickupAndSettle(String target, int settleSeconds) {
long startMillis = nowMillis.getAsLong();
long deadline = startMillis + TimeUnit.SECONDS.toMillis(settleSeconds);
boolean pickedUp = false; // a WORKING sample has been observed since /clear was sent
int idlePollsAwaitingPickup = 0;
int nudges = 0;
while (nowMillis.getAsLong() < deadline) {
AgentStatus status;
try {
status = agents.status(target);
} catch (RuntimeException e) {
log.debug("lead-rollover: status check failed while waiting for {} to settle after "
+ "/clear: {}", target, e.toString());
status = null;
}
if (status == AgentStatus.WORKING) {
pickedUp = true;
} else if (status == AgentStatus.IDLE || status == AgentStatus.DONE) {
if (pickedUp) {
// a real WORKING -> IDLE/DONE completion boundary
return new ClearSettleResult(true, nowMillis.getAsLong() - startMillis, nudges);
}
if (++idlePollsAwaitingPickup >= PICKUP_GRACE_POLLS) {
long elapsedMillis = nowMillis.getAsLong() - startMillis;
// fleetd #494: this release trades a possibly-unsubmitted /clear for progress
// instead of wedging the roll — that trade is deliberate and stays. But it is
// also exactly the case that reported false success in the real incident (the
// whole roll "succeeded" after 438ms of a 20s budget), so raise it to WARN and
// print the MEASURED elapsed time next to the target pane, not just the count.
//
// fleetd #494 follow-up (2nd pass): BOTH numbers in this line must come from
// the loop's own counters, never from the PICKUP_GRACE_POLLS constant.
// `idlePollsAwaitingPickup` and `nudges` each have exactly one write site in
// this loop, on the same branch, so on this branch they cannot differ from
// PICKUP_GRACE_POLLS / PICKUP_GRACE_POLLS - 1 today — no test can prove the
// difference on this line, and printing the counters does not change that.
// What it does buy: one source of truth instead of two, so a later change to
// the loop (an early return, a second increment site, a different exit
// condition) cannot leave this message reporting a number the loop no longer
// produces. The place where `nudges` genuinely varies with the run — and is
// covered by a test that can tell it apart from a constant — is the
// /clear-timeout warn in runRollover, which prints clearResult.nudges().
log.warn("lead-rollover: /clear on {} was never observed as WORKING after {} "
+ "consecutive IDLE/DONE polls ({} of those were nudged) — "
+ "releasing rather than wedging the roll (elapsed={}ms)",
target, idlePollsAwaitingPickup, nudges, elapsedMillis);
return new ClearSettleResult(true, elapsedMillis, nudges);
}
try {
agents.submit(target); // nudge a raced Enter (CB-113) so /clear actually submits
} catch (RuntimeException e) {
log.debug("lead-rollover: resubmit to {} failed (will retry next poll): {}",
target, e.getMessage());
} finally {
nudges++; // an attempted nudge, whether or not the submit call itself threw
}
}
// AgentStatus.BLOCKED or UNKNOWN (or an unreadable status, above): neither a pickup
// signal nor a boundary — keep polling without nudging or releasing.
settleSleeper.run();
}
return new ClearSettleResult(false, nowMillis.getAsLong() - startMillis, nudges);
}
/**
* The measured outcome of {@link #waitForClearPickupAndSettle} — fleetd #494. Carries the
* MEASURED elapsed time (from the injected {@link #nowMillis} clock) and nudge count alongside
* the settle/timeout decision, so callers can log them instead of the configured budget, which
* is not how long the wait actually ran.
*/
private record ClearSettleResult(boolean settled, long elapsedMillis, int nudges) {}
} }
@@ -5,13 +5,13 @@ import dev.ltms.fleet.herdr.PaneLocator;
/** /**
* Resolves <em>who is calling</em> an MCP tool from the connection alone — the anti-spoofing * Resolves <em>who is calling</em> an MCP tool from the connection alone — the anti-spoofing
* identity model of the MCP contract. It ties the connection's loopback peer PID (from the OS) * identity model of the MCP contract. It ties the connection's loopback peer PID (from the OS)
* to a herdr agent pane (from herdr), yielding the caller's worker {@code terminal_id}. A caller * to a herdr agent pane (from herdr), yielding that pane's {@code terminal_id}. A connection that
* that maps to no worker pane — the primary, or an off-host client — resolves to {@code null}. * maps to no pane resolves to {@code null}; this class assigns no role to either outcome — {@link
* dev.ltms.fleet.auth.CallerResolver} does that.
* *
* <p>Both sources are authoritative and unforgeable: the OS reports the real connecting PID, and * <p>Both sources are authoritative and unforgeable: the OS reports the real connecting PID, and
* herdr owns the PID→pane mapping. A worker cannot claim to be another worker, nor the primary. * herdr owns the PID→pane mapping, so a caller cannot claim to be at another pane. Single-host
* Single-host only (the herd shares the {@code fleetd} host); the token path is the split-host * only (the herd shares the {@code fleetd} host); the token path is the split-host fallback.
* fallback.
*/ */
public final class ConnectionIdentity { public final class ConnectionIdentity {
@@ -46,9 +46,10 @@ public final class ConnectionIdentity {
} }
/** /**
* The caller resolved from the connection: its worker {@code terminal} (or {@code null} for the * The caller resolved from the connection: the {@code terminal} of the pane it connects from
* primary / an off-host client), its {@code pid} (or {@code -1} if not resolvable), and whether * (or {@code null} when the connection maps to no pane), its {@code pid} (or {@code -1} if not
* the pane scan behind {@code terminal} ran to completion ({@link #scanComplete}). * resolvable), and whether the pane scan behind {@code terminal} ran to completion
* ({@link #scanComplete}).
*/ */
public record Caller(String terminal, long pid, boolean scanComplete) { public record Caller(String terminal, long pid, boolean scanComplete) {
@@ -87,8 +88,8 @@ public final class ConnectionIdentity {
} }
/** /**
* The calling worker's {@code terminal_id}, or {@code null} if the caller is not a known * The terminal id of the pane the caller connects from, or {@code null} if the connection
* on-host worker (treat as the primary). * maps to no pane.
*/ */
public String callerTerminal(String remoteAddr, int remotePort) { public String callerTerminal(String remoteAddr, int remotePort) {
return resolve(remoteAddr, remotePort).terminal(); return resolve(remoteAddr, remotePort).terminal();
@@ -1,5 +1,6 @@
package dev.ltms.fleet.mcp; package dev.ltms.fleet.mcp;
import dev.ltms.fleet.Fleetd;
import dev.ltms.fleet.auth.AuditLog; import dev.ltms.fleet.auth.AuditLog;
import dev.ltms.fleet.auth.Authz; import dev.ltms.fleet.auth.Authz;
import dev.ltms.fleet.auth.CallerResolver; import dev.ltms.fleet.auth.CallerResolver;
@@ -41,6 +42,7 @@ import com.fasterxml.jackson.databind.ObjectMapper;
import jakarta.servlet.http.HttpServlet; import jakarta.servlet.http.HttpServlet;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Comparator;
import java.util.LinkedHashMap; import java.util.LinkedHashMap;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
@@ -52,6 +54,7 @@ import java.util.concurrent.TimeUnit;
import java.util.function.BiFunction; import java.util.function.BiFunction;
import java.util.function.Function; import java.util.function.Function;
import java.util.function.LongSupplier; import java.util.function.LongSupplier;
import java.util.function.Predicate;
import java.util.function.Supplier; import java.util.function.Supplier;
import java.util.stream.Collectors; import java.util.stream.Collectors;
@@ -116,9 +119,10 @@ public final class FleetMcp {
private final ConnectionIdentity identity; private final ConnectionIdentity identity;
/** /**
* Kept as a field (rather than only captured by the {@code contextExtractor} closure) so * Kept as a field (rather than only captured by the {@code contextExtractor} closure) so
* {@link #denyFor} can read {@link CallerResolver#knownLeadOrCollaborator()} — the classifier a * {@link #denyFor} can read {@link CallerResolver#knownLeadOrCollaborator()} and {@link
* collaborator's {@code SEND} is checked against, built from the same lead and collaborator * CallerResolver#sendableObserverTarget()} — the classifiers a collaborator's and an
* maps {@link #identity}-based resolution reads. * observer's {@code SEND} are each checked against, built from the same maps {@link #identity}-
* based resolution reads.
*/ */
private final CallerResolver callers; private final CallerResolver callers;
private final Metrics metrics; // CB-502: null → auth failures not counted private final Metrics metrics; // CB-502: null → auth failures not counted
@@ -302,6 +306,34 @@ public final class FleetMcp {
public static LeadConfigDirSource none() { return new LeadConfigDirSource(_ -> null, _ -> null); } public static LeadConfigDirSource none() { return new LeadConfigDirSource(_ -> null, _ -> null); }
} }
/**
* Pane-discovery facts for {@code fleet_list}'s {@code panes} row — every herdr tab's display
* label, the one deliverability gate the status-gated injector itself reads, whether a
* terminal is bound to a configured architect slot, and whether an observer caller may
* {@code SEND} to it.
*
* @param tabLabels tab id → its display label, read lazily (only once the row is actually
* assembled) since it costs a herdr {@code workspace.list}/{@code tab.list}
* scan; a tab herdr reports with no label maps to a {@code null} value
* @param workspaceLabels workspace id → its display label (the herdr "space" name), read lazily
* the same way as {@code tabLabels}; a workspace herdr reports with no label,
* or one the lookup cannot find, maps to a {@code null} value
* @param deliverable the same gate {@link dev.ltms.fleet.Fleetd#deliverableTo} builds for the
* injector, keyed by terminal id — never a second, separately-derived check
* @param architectSlot the same classifier {@link CallerResolver#boundToArchitectSlot} resolves
* a caller against — never a second, separately-derived check
* @param sendableToObserver the same predicate {@link CallerResolver#sendableObserverTarget}
* builds for the {@code SEND} gate — never a second, separately-derived
* check
*/
public record PaneSource(Supplier<Map<String, String>> tabLabels, Supplier<Map<String, String>> workspaceLabels,
Predicate<String> deliverable, Predicate<String> architectSlot, Predicate<String> sendableToObserver) {
/** Inert source — no labels, no deliverable targets, no architect slots, nothing sendable. */
public static PaneSource none() {
return new PaneSource(Map::of, Map::of, _ -> false, _ -> false, _ -> false);
}
}
/** /**
* fleetd #361: peer-visibility facts for {@code fleet_list}'s {@code coordinator} row — this * fleetd #361: peer-visibility facts for {@code fleet_list}'s {@code coordinator} row — this
* daemon's own {@link LeadChannel} (for its self mailbox state and held messages) plus the * daemon's own {@link LeadChannel} (for its self mailbox state and held messages) plus the
@@ -334,7 +366,7 @@ public final class FleetMcp {
* caller explicitly saying so — never by omitting a {@link CallerResolver} the way the old * caller explicitly saying so — never by omitting a {@link CallerResolver} the way the old
* {@code callers == null} idiom allowed. {@code callers} itself is required either way: even * {@code callers == null} idiom allowed. {@code callers} itself is required either way: even
* under {@link #UNENFORCED}, the one real {@link CallerResolver} still resolves every caller's * under {@link #UNENFORCED}, the one real {@link CallerResolver} still resolves every caller's
* {@link Principal} (so {@code markSpawnedMemberPresent}/{@code recordPrimarySingleton} see a * {@link Principal} (so {@code markTrackedCallerPresent}/{@code recordPrimarySingleton} see a
* real identity), and {@link #denyFor} is the only thing that changes. * real identity), and {@link #denyFor} is the only thing that changes.
*/ */
public enum AuthorizationMode { ENFORCED, UNENFORCED } public enum AuthorizationMode { ENFORCED, UNENFORCED }
@@ -349,11 +381,9 @@ public final class FleetMcp {
* overload instead of failing to compile, and every existing test — none of which exercises * overload instead of failing to compile, and every existing test — none of which exercises
* {@code Fleetd.main} itself — stays green while the live daemon quietly answers * {@code Fleetd.main} itself — stays green while the live daemon quietly answers
* {@code NOT_CONFIGURED} to {@code fleet_handover} forever. Collapsing every overload into one * {@code NOT_CONFIGURED} to {@code fleet_handover} forever. Collapsing every overload into one
* required-everything constructor turns that mistake into a compile error instead: this * required-everything constructor turns that mistake into a compile error instead, the same
* project's own antidote for a defaulted parameter surviving as an untested decision (see * defaulted-parameter guard this codebase applies to other required wiring. A caller that
* {@code FleetdCompletionResolverWiringTest} / {@code FleetdLeadRolloverWiringTest}'s own * genuinely wants a feature off must now say so explicitly at the call site — {@code null},
* javadoc for the same lesson applied to a different seam). A caller that genuinely wants a
* feature off must now say so explicitly at the call site — {@code null},
* {@link OutageSource#none()}, {@link LeadSeatSource#none()}, {@code List.of()} are all still * {@link OutageSource#none()}, {@link LeadSeatSource#none()}, {@code List.of()} are all still
* perfectly fine values, just never an implicit default reached by omission. * perfectly fine values, just never an implicit default reached by omission.
* *
@@ -442,10 +472,10 @@ public final class FleetMcp {
// fall back to here — AuthorizationMode governs enforcement, not identity. // fall back to here — AuthorizationMode governs enforcement, not identity.
Principal p = callers.resolve(req.getRemoteAddr(), req.getRemotePort(), Principal p = callers.resolve(req.getRemoteAddr(), req.getRemotePort(),
req.getHeader("Authorization")); req.getHeader("Authorization"));
// CB-532: guard on the ROLE, not on the terminal being null. This excludes a // Guards on the ROLE, not on the terminal being null — this marks presence for
// lead, which carries its pane too, while including every spawned member role. // a worker, an architect, or the unconfigured-pane floor, and excludes a lead
// Enrolling a lead would count it as an available member in the roster. // or a collaborator even though each carries its own pane too.
markSpawnedMemberPresent(p, presence); markTrackedCallerPresent(p, presence);
return McpTransportContext.create(Map.of( return McpTransportContext.create(Map.of(
CALLER_TERMINAL, orEmpty(p.terminal()), CALLER_TERMINAL, orEmpty(p.terminal()),
CALLER_PID, Long.toString(p.pid()), CALLER_PID, Long.toString(p.pid()),
@@ -459,15 +489,20 @@ public final class FleetMcp {
McpSchema.CallToolResult denied = deny(exchange, toolAction("fleet_send", req.arguments()), McpSchema.CallToolResult denied = deny(exchange, toolAction("fleet_send", req.arguments()),
str(req.arguments(), "sessionId")); str(req.arguments(), "sessionId"));
if (denied != null) return denied; if (denied != null) return denied;
String caller = callerTerminal(exchange); Principal caller = principal(exchange);
String callerTerminal = caller.terminal();
String callerOwner = caller.ownerKey();
// CB-548: only a PRIMARY caller may claim the legacy singleton "primary" fallback. // CB-548: only a PRIMARY caller may claim the legacy singleton "primary" fallback.
// An architect delegates as its own pane but must never become the fallback that // An architect delegates as its own pane but must never become the fallback that
// no-delegation inbox nudges target as if it were the primary (the per-target // no-delegation inbox nudges target as if it were the primary (the per-target
// delegation map does not cure the singleton). // delegation map does not cure the singleton).
recordPrimarySingleton(primaryRegistry, caller, principal(exchange)); recordPrimarySingleton(primaryRegistry, callerTerminal, caller);
Map<String, Object> a = req.arguments(); Map<String, Object> a = req.arguments();
String target = str(a, "sessionId"); String target = str(a, "sessionId");
String content = str(a, "content"); // An observer's SEND reaches a pane that cannot otherwise distinguish this
// from a human paste (see attributeIfObserver); every other caller's content
// passes through unchanged.
String content = attributeIfObserver(caller, str(a, "content"));
String turnId = str(a, "turnId"); String turnId = str(a, "turnId");
String coordId = str(a, "coordId"); String coordId = str(a, "coordId");
if (coordId != null && !coordId.isBlank()) { if (coordId != null && !coordId.isBlank()) {
@@ -480,21 +515,28 @@ public final class FleetMcp {
// Answering a worker's fleet_ask (CB-205): resolve its blocked question and // Answering a worker's fleet_ask (CB-205): resolve its blocked question and
// block for the worker's reply as it resumes the same turn. This is the same // block for the worker's reply as it resumes the same turn. This is the same
// delegation, so ownership is left untouched (CB-548) — never re-recorded. // delegation, so ownership is left untouched (CB-548) — never re-recorded.
return answer(messages, turnId, content, timeoutMs(a), caller); return answer(messages, turnId, content, timeoutMs(a), callerOwner);
} }
// CB-548: delegator ownership (which lead's reply nudge this worker routes to, // CB-548: delegator ownership (which lead's reply nudge this worker routes to,
// CB-532) is recorded only once the send is ACCEPTED — MessageService has won the // CB-532) is recorded only once the send is ACCEPTED — MessageService has won the
// session lock and queued delivery — via the accepted-delivery callback, never at // session lock and queued delivery — via the accepted-delivery callback, never at
// request time. A concurrent sender that times out BUSY therefore cannot steal a // request time. A concurrent sender that times out BUSY therefore cannot steal a
// live turn's reply routing without ever owning the turn. // live turn's reply routing without ever owning the turn.
Runnable onAccepted = () -> primaryRegistry.recordDelegation(target, caller); // Only a lead's name is ever resolvable back to a current terminal (PrimaryRegistry
// only looks it up among currently recognised leads) — an architect or collaborator
// name would never match there anyway, but passing null for them keeps the intent
// explicit rather than relying on that lookup to filter it out.
String delegatorName = caller.isPrimary() ? caller.name() : null;
Runnable onAccepted = () ->
primaryRegistry.recordDelegation(target, callerTerminal, delegatorName);
// wait defaults to true (block for the reply); wait:false is fire-and-poll. // wait defaults to true (block for the reply); wait:false is fire-and-poll.
return Boolean.FALSE.equals(a.get("wait")) return Boolean.FALSE.equals(a.get("wait"))
? sendAsync(messages, target, content, onAccepted, workers.profiles(), caller) ? sendAsync(messages, target, content, onAccepted, workers.profiles(), caller)
: send(messages, target, content, timeoutMs(a), onAccepted, workers.profiles(), caller); : send(messages, target, content, timeoutMs(a), onAccepted, workers.profiles(), callerOwner);
}; };
// fleet_reply's identity is the CONNECTION, never an argument — so the authz check // fleet_reply's identity is the CONNECTION, never an argument. The authz check is
// is "is this caller a worker at all", and it can only ever reply as itself. // terminal ownership, not a role test: the caller may reply only for its own pane,
// which is why no role appears in the check at all.
BiFunction<McpSyncServerExchange, McpSchema.CallToolRequest, McpSchema.CallToolResult> replyHandler = BiFunction<McpSyncServerExchange, McpSchema.CallToolRequest, McpSchema.CallToolResult> replyHandler =
(exchange, req) -> { (exchange, req) -> {
String self = callerTerminal(exchange); String self = callerTerminal(exchange);
@@ -514,7 +556,7 @@ public final class FleetMcp {
(exchange, req) -> { (exchange, req) -> {
McpSchema.CallToolResult denied = deny(exchange, toolAction("fleet_status", req.arguments()), null); McpSchema.CallToolResult denied = deny(exchange, toolAction("fleet_status", req.arguments()), null);
if (denied != null) return denied; if (denied != null) return denied;
return status(messages, str(req.arguments(), "sessionId"), callerTerminal(exchange)); return status(messages, str(req.arguments(), "sessionId"), principal(exchange).ownerKey());
}; };
BiFunction<McpSyncServerExchange, McpSchema.CallToolRequest, McpSchema.CallToolResult> pollHandler = BiFunction<McpSyncServerExchange, McpSchema.CallToolRequest, McpSchema.CallToolResult> pollHandler =
(exchange, req) -> { (exchange, req) -> {
@@ -524,7 +566,8 @@ public final class FleetMcp {
// The action depends on the ARGUMENTS, not on the tool name -- see pollAction. // The action depends on the ARGUMENTS, not on the tool name -- see pollAction.
McpSchema.CallToolResult denied = deny(exchange, toolAction("fleet_poll", a), target); McpSchema.CallToolResult denied = deny(exchange, toolAction("fleet_poll", a), target);
if (denied != null) return denied; if (denied != null) return denied;
return poll(messages, leadChannel, str(a, "ticket"), target, coordId, callerTerminal(exchange)); return poll(messages, leadChannel, str(a, "ticket"), target, coordId,
principal(exchange).ownerKey());
}; };
// CB-307 Increment 3: per-msgId ack (not needed in v1 but supported by the inbox). // CB-307 Increment 3: per-msgId ack (not needed in v1 but supported by the inbox).
// Acking removes a reply from the inbox, so it is a drain, not a read. // Acking removes a reply from the inbox, so it is a drain, not a read.
@@ -557,13 +600,20 @@ public final class FleetMcp {
(exchange, _) -> { (exchange, _) -> {
McpSchema.CallToolResult denied = deny(exchange, toolAction("fleet_list", Map.of()), null); McpSchema.CallToolResult denied = deny(exchange, toolAction("fleet_list", Map.of()), null);
if (denied != null) return denied; if (denied != null) return denied;
// A Supplier: the label lookup costs a herdr scan, and must stay behind
// panesVisible so it only runs for a caller that receives the row at all.
PaneSource panes = new PaneSource(() -> identity.panes().tabLabelsByTabId(),
() -> identity.panes().workspaceLabelsByWorkspaceId(),
Fleetd.deliverableTo(presence, callers::leads, callers::collaborators),
callers::boundToArchitectSlot, callers.sendableObserverTarget());
return listFleet(workers, sessions, messages, capacity, healthCoverage, loopHealth, quarantine, outage, return listFleet(workers, sessions, messages, capacity, healthCoverage, loopHealth, quarantine, outage,
leadSeats, leadContextGauge, leadConfigDirs, callers.leads(), leadSeats, leadContextGauge, leadConfigDirs, callers.leads(),
callerTerminal(exchange), callerTerminal(exchange),
callers.collaborators(), collaboratorsVisibleTo(principal(exchange)), callers.collaborators(), collaboratorsVisibleTo(principal(exchange)),
new CoordinationSource(leadChannel, peers), new CoordinationSource(leadChannel, peers),
coordinatorVisibleTo(principal(exchange)), coordinatorVisibleTo(principal(exchange)),
leadsVisibleTo(principal(exchange)), membersVisibleTo(principal(exchange))); leadsVisibleTo(principal(exchange)), membersVisibleTo(principal(exchange)),
panes, panesVisibleTo(principal(exchange)), principal(exchange).isObserver());
}; };
BiFunction<McpSyncServerExchange, McpSchema.CallToolRequest, McpSchema.CallToolResult> stopHandler = BiFunction<McpSyncServerExchange, McpSchema.CallToolRequest, McpSchema.CallToolResult> stopHandler =
(exchange, req) -> { (exchange, req) -> {
@@ -591,7 +641,8 @@ public final class FleetMcp {
(exchange, req) -> { (exchange, req) -> {
McpSchema.CallToolResult denied = deny(exchange, toolAction("fleet_handover", req.arguments()), null); McpSchema.CallToolResult denied = deny(exchange, toolAction("fleet_handover", req.arguments()), null);
if (denied != null) return denied; if (denied != null) return denied;
return handover(leadRollover, callerTerminal(exchange), req.arguments()); Principal caller = principal(exchange);
return handover(leadRollover, messages, caller.terminal(), caller.ownerKey(), req.arguments());
}; };
McpSchema.Tool fleetSend = sendTool(); McpSchema.Tool fleetSend = sendTool();
@@ -699,7 +750,8 @@ public final class FleetMcp {
if (!authorizationEnforced) { if (!authorizationEnforced) {
return null; // AuthorizationMode.UNENFORCED: authorization not enforced (fleetd #518) return null; // AuthorizationMode.UNENFORCED: authorization not enforced (fleetd #518)
} }
if (Authz.permits(caller, action, target, callers.knownLeadOrCollaborator())) { if (Authz.permits(caller, action, target, callers.knownLeadOrCollaborator(),
callers.sendableObserverTarget())) {
if (action != Authz.Action.READ && action != Authz.Action.TASK_READ) { if (action != Authz.Action.READ && action != Authz.Action.TASK_READ) {
AuditLog.allowed(caller, action, target); // reads would drown the trail AuditLog.allowed(caller, action, target); // reads would drown the trail
} }
@@ -727,7 +779,7 @@ public final class FleetMcp {
*/ */
static void recordPrimarySingleton(PrimaryRegistry registry, String callerTerminal, Principal caller) { static void recordPrimarySingleton(PrimaryRegistry registry, String callerTerminal, Principal caller) {
if (caller != null && caller.isPrimary()) { if (caller != null && caller.isPrimary()) {
registry.record(callerTerminal); registry.record(callerTerminal, caller.name());
} }
} }
@@ -787,6 +839,17 @@ public final class FleetMcp {
return caller.isPrimary() || caller.isArchitect(); return caller.isPrimary() || caller.isArchitect();
} }
/**
* Who may see {@code fleet_list}'s {@code panes} array — every role that may
* {@link Authz.Action#SEND} to some other pane. A plain worker holds {@code READ} but never
* {@code SEND}, so it still does not see this array. An observer does hold {@code SEND}, to
* another observer pane only, so it sees the array too — but {@code listFleet} filters its rows
* to {@link CallerResolver#sendableObserverTarget} and reduces each one; see {@code paneRows}.
*/
static boolean panesVisibleTo(Principal caller) {
return caller.isPrimary() || caller.isArchitect() || caller.isCollaborator() || caller.isObserver();
}
/** /**
* The terminal of the caller on this call's connection, or {@code null} when that caller carries * The terminal of the caller on this call's connection, or {@code null} when that caller carries
* no terminal, which is only the unnamed primary. A named lead, an architect and a worker each * no terminal, which is only the unnamed primary. A named lead, an architect and a worker each
@@ -833,9 +896,14 @@ public final class FleetMcp {
return identity; return identity;
} }
/** Mark a connected spawned member available for the injector readiness gate. */ /**
static void markSpawnedMemberPresent(Principal caller, MemberPresence presence) { * Mark a caller present for the injector readiness gate, when its deliverability depends on
if (caller.isSpawnedMember()) { * proving a live MCP contact: a worker, an architect, or the unconfigured-pane floor. A lead
* or a collaborator is excluded — each is already deliverable through its own named-registry
* entry.
*/
static void markTrackedCallerPresent(Principal caller, MemberPresence presence) {
if (caller.isSpawnedMember() || caller.isObserver()) {
presence.markPresent(caller.terminal()); presence.markPresent(caller.terminal());
} }
} }
@@ -891,8 +959,8 @@ public final class FleetMcp {
* fleetd #612 B3 — as {@link #quarantineSource()}, {@code public} for the same cross-package * fleetd #612 B3 — as {@link #quarantineSource()}, {@code public} for the same cross-package
* reason, for the real {@link LeadRollover} (or {@code null}) this daemon was assembled with. * reason, for the real {@link LeadRollover} (or {@code null}) this daemon was assembled with.
* {@code FleetdLeadRolloverAssemblyTest} drives {@code open}/{@code confirm} on this exact * {@code FleetdLeadRolloverAssemblyTest} drives {@code open}/{@code confirm} on this exact
* instance and waits for the real continuation to send {@code /clear} and {@code bootstrapText} * instance and waits for the real continuation to end the old pane, relaunch a fresh one, and
* through the real {@code router.leadAgents()}. * send {@code bootstrapText} through the real {@code router.leadAgents()}.
*/ */
public LeadRollover leadRollover() { public LeadRollover leadRollover() {
return leadRollover; return leadRollover;
@@ -900,6 +968,17 @@ public final class FleetMcp {
// --- tool logic (thin adapters over the services; unit-testable) --------------------------- // --- tool logic (thin adapters over the services; unit-testable) ---------------------------
/**
* The text an observer's {@code SEND} actually delivers: prefixed with the sender's own
* connection-resolved terminal, which the receiving pane cannot otherwise tell apart from a
* human paste. Every other caller's content passes through unchanged. Shared with {@code
* FleetApp}'s REST entry path so both surfaces attribute identically.
*/
public static String attributeIfObserver(Principal caller, String content) {
return caller != null && caller.isObserver()
? "[fleet_send from observer " + caller.terminal() + "]\n" + content : content;
}
/** /**
* {@code fleet_send}: delegate {@code content} to a worker session and block for its reply. * {@code fleet_send}: delegate {@code content} to a worker session and block for its reply.
* The configured profiles are required so a profile name can never bypass target validation. * The configured profiles are required so a profile name can never bypass target validation.
@@ -909,7 +988,7 @@ public final class FleetMcp {
*/ */
static McpSchema.CallToolResult send(MessageService messages, String sessionId, String content, static McpSchema.CallToolResult send(MessageService messages, String sessionId, String content,
Long timeoutMs, Runnable onAccepted, Set<String> profiles, Long timeoutMs, Runnable onAccepted, Set<String> profiles,
String callerTerminal) { String callerOwner) {
if (isBlank(sessionId) || isBlank(content)) { if (isBlank(sessionId) || isBlank(content)) {
return error("sessionId and content are required"); return error("sessionId and content are required");
} }
@@ -919,7 +998,7 @@ public final class FleetMcp {
} }
long timeout = clamp(timeoutMs == null ? DEFAULT_TIMEOUT_MS : timeoutMs); long timeout = clamp(timeoutMs == null ? DEFAULT_TIMEOUT_MS : timeoutMs);
try { try {
return formatReply(messages.send(sessionId, content, timeout, onAccepted, callerTerminal), timeout); return formatReply(messages.send(sessionId, content, timeout, onAccepted, callerOwner), timeout);
} catch (HerdrException e) { } catch (HerdrException e) {
return error("herdr error contacting session " + sessionId + ": " + e.getMessage()); return error("herdr error contacting session " + sessionId + ": " + e.getMessage());
} }
@@ -929,15 +1008,15 @@ public final class FleetMcp {
* {@code fleet_send} carrying a {@code turnId}: the primary's answer to a worker's * {@code fleet_send} carrying a {@code turnId}: the primary's answer to a worker's
* {@code fleet_ask} (CB-205). Resolves the worker's blocked question and blocks for its reply as * {@code fleet_ask} (CB-205). Resolves the worker's blocked question and blocks for its reply as
* it resumes the same turn — surfaced to the primary identically to a normal send. * it resumes the same turn — surfaced to the primary identically to a normal send.
* {@code callerTerminal} must match the turn's recorded owner or this is refused. * {@code callerOwner} must match the turn's recorded owner or this is refused.
*/ */
static McpSchema.CallToolResult answer(MessageService messages, String turnId, String content, Long timeoutMs, static McpSchema.CallToolResult answer(MessageService messages, String turnId, String content, Long timeoutMs,
String callerTerminal) { String callerOwner) {
if (isBlank(turnId) || isBlank(content)) { if (isBlank(turnId) || isBlank(content)) {
return error("turnId and content are required to answer a worker's question"); return error("turnId and content are required to answer a worker's question");
} }
long timeout = clamp(timeoutMs == null ? DEFAULT_TIMEOUT_MS : timeoutMs); long timeout = clamp(timeoutMs == null ? DEFAULT_TIMEOUT_MS : timeoutMs);
return formatReply(messages.answer(turnId, content, timeout, callerTerminal), timeout); return formatReply(messages.answer(turnId, content, timeout, callerOwner), timeout);
} }
/** /**
@@ -1013,13 +1092,12 @@ public final class FleetMcp {
} }
/** /**
* As above, recording {@code creatorTerminal} as this ticket's owner (the caller's own * As above, recording {@code creator}'s owner key so a later {@code fleet_poll{ticket}} only
* terminal, resolved from the connection) so a later {@code fleet_poll{ticket}} only hands the * hands the result back to the same caller — see {@link MessageService#poll(String, String)}.
* result back to that same caller — see {@link MessageService#poll(String, String)}.
*/ */
static McpSchema.CallToolResult sendAsync(MessageService messages, String sessionId, String content, static McpSchema.CallToolResult sendAsync(MessageService messages, String sessionId, String content,
Runnable onAccepted, Set<String> profiles, Runnable onAccepted, Set<String> profiles,
String creatorTerminal) { Principal creator) {
if (isBlank(sessionId) || isBlank(content)) { if (isBlank(sessionId) || isBlank(content)) {
return error("sessionId and content are required"); return error("sessionId and content are required");
} }
@@ -1027,7 +1105,7 @@ public final class FleetMcp {
if (targetError != null) { if (targetError != null) {
return targetError; return targetError;
} }
String ticket = messages.sendAsync(sessionId, content, onAccepted, creatorTerminal); String ticket = messages.sendAsync(sessionId, content, onAccepted, creator);
return text("accepted — task delegated. Poll fleet_poll with ticket=" + ticket); return text("accepted — task delegated. Poll fleet_poll with ticket=" + ticket);
} }
@@ -1217,13 +1295,12 @@ public final class FleetMcp {
} }
/** /**
* As above, refusing a ticket lookup whose caller's terminal differs from the terminal that * As above, refusing a ticket lookup whose caller owner key differs from the key that created it
* created it — see {@link MessageService#poll(String, String)}. {@code callerTerminal} is the * — see {@link MessageService#poll(String, String)}. {@code callerOwner} comes from the calling
* CALLING session's terminal id, resolved by the MCP layer from the connection, never a * connection's resolved principal, never a client-supplied value.
* client-supplied value.
*/ */
static McpSchema.CallToolResult poll(MessageService messages, LeadChannel leadChannel, String ticket, static McpSchema.CallToolResult poll(MessageService messages, LeadChannel leadChannel, String ticket,
String target, String coordId, String callerTerminal) { String target, String coordId, String callerOwner) {
if (!isBlank(coordId)) { if (!isBlank(coordId)) {
return pollHeldPeerMail(leadChannel, coordId); return pollHeldPeerMail(leadChannel, coordId);
} }
@@ -1237,7 +1314,7 @@ public final class FleetMcp {
if (isBlank(ticket)) { if (isBlank(ticket)) {
return error("ticket (or target) is required"); return error("ticket (or target) is required");
} }
MessageService.TaskView v = messages.poll(ticket, callerTerminal); MessageService.TaskView v = messages.poll(ticket, callerOwner);
if (v == null) { if (v == null) {
return error("unknown ticket: " + ticket + " (never issued, or expired)"); return error("unknown ticket: " + ticket + " (never issued, or expired)");
} }
@@ -1346,18 +1423,18 @@ public final class FleetMcp {
* {@code fleet_status}: the live lifecycle status of a worker session, plus — when the worker * {@code fleet_status}: the live lifecycle status of a worker session, plus — when the worker
* is paused mid-turn in an async {@code fleet_ask} — the open question and how to answer it, so * is paused mid-turn in an async {@code fleet_ask} — the open question and how to answer it, so
* a lead on its normal poll cadence does not need the ticket to notice. The question, its * a lead on its normal poll cadence does not need the ticket to notice. The question, its
* {@code turnId} and its ticket id are shown only to the caller whose terminal created that * {@code turnId} and its ticket id are shown only to the caller whose owner key matches the
* delegation, or to a caller with no terminal at all (the unnamed primary); any other caller * delegation's creator — the unnamed primary matches only a delegation another unnamed
* still sees the base status. {@code callerTerminal} is the CALLING session's terminal id, * primary created; any other caller still sees the base status. {@code callerOwner} comes
* resolved by the MCP layer from the connection, never a client-supplied value. * from the calling connection's resolved principal.
*/ */
static McpSchema.CallToolResult status(MessageService messages, String sessionId, String callerTerminal) { static McpSchema.CallToolResult status(MessageService messages, String sessionId, String callerOwner) {
if (isBlank(sessionId)) { if (isBlank(sessionId)) {
return error("sessionId is required"); return error("sessionId is required");
} }
try { try {
String base = messages.status(sessionId).name().toLowerCase(); String base = messages.status(sessionId).name().toLowerCase();
MessageService.PendingAsk ask = messages.pendingAsk(sessionId, callerTerminal); MessageService.PendingAsk ask = messages.pendingAsk(sessionId, callerOwner);
if (ask == null) { if (ask == null) {
return text(base); return text(base);
} }
@@ -1412,6 +1489,14 @@ public final class FleetMcp {
} }
return text(json(m)); return text(json(m));
} }
if (caller.isObserver()) {
// No architect slot, collaborator name, or lead name to report — only the pane itself,
// so a peer that already knows this terminal can still address it.
if (caller.terminal() != null) {
m.put("sessionId", caller.terminal());
}
return text(json(m));
}
if (!caller.isWorker()) { if (!caller.isWorker()) {
// CB-530: which lead, once more than one pane is configured as one. `role` deliberately // CB-530: which lead, once more than one pane is configured as one. `role` deliberately
// still reads "primary" — the fallback ladder in CLAUDE.md keys on it, and a lead IS a // still reads "primary" — the fallback ladder in CLAUDE.md keys on it, and a lead IS a
@@ -1462,14 +1547,15 @@ public final class FleetMcp {
* #474 charter tool-surface gate), so every action here must degrade to a clean, structured * #474 charter tool-surface gate), so every action here must degrade to a clean, structured
* refusal naming {@code NOT_CONFIGURED} rather than ever throwing. * refusal naming {@code NOT_CONFIGURED} rather than ever throwing.
*/ */
static McpSchema.CallToolResult handover(LeadRollover leadRollover, String callerTerminal, static McpSchema.CallToolResult handover(LeadRollover leadRollover, MessageService messages,
String callerTerminal, String callerOwner,
Map<String, Object> args) { Map<String, Object> args) {
String action = str(args, "action"); String action = str(args, "action");
if (isBlank(action)) { if (isBlank(action)) {
return error("action is required: \"open\", \"confirm\", \"cancel\" or \"status\""); return error("action is required: \"open\", \"confirm\", \"cancel\" or \"status\"");
} }
return switch (action) { return switch (action) {
case "open" -> handoverOpen(leadRollover, callerTerminal, str(args, "reason")); case "open" -> handoverOpen(leadRollover, messages, callerTerminal, callerOwner, str(args, "reason"));
case "confirm" -> handoverConfirm(leadRollover, callerTerminal, str(args, "token"), case "confirm" -> handoverConfirm(leadRollover, callerTerminal, str(args, "token"),
truthy(args, "operatorConfirmed")); truthy(args, "operatorConfirmed"));
case "cancel" -> handoverCancel(leadRollover, str(args, "token")); case "cancel" -> handoverCancel(leadRollover, str(args, "token"));
@@ -1485,14 +1571,15 @@ public final class FleetMcp {
* IllegalStateException} for that), both degrade to the same clean {@code NOT_CONFIGURED} * IllegalStateException} for that), both degrade to the same clean {@code NOT_CONFIGURED}
* refusal — never an escaping exception. * refusal — never an escaping exception.
*/ */
private static McpSchema.CallToolResult handoverOpen(LeadRollover leadRollover, String callerTerminal, private static McpSchema.CallToolResult handoverOpen(LeadRollover leadRollover, MessageService messages,
String callerTerminal, String callerOwner,
String reason) { String reason) {
if (leadRollover == null) { if (leadRollover == null) {
return notConfigured(); return notConfigured();
} }
if (isBlank(callerTerminal)) { if (isBlank(callerTerminal)) {
// An unnamed primary (token/loopback path, no resolved pane) has nowhere for the // An unnamed primary (token/loopback path, no resolved pane) has nowhere for the
// eventual /clear + bootstrap to land — LeadRollover#open would throw // eventual relaunch + bootstrap to land — LeadRollover#open would throw
// IllegalArgumentException for the same reason; refuse cleanly here instead. // IllegalArgumentException for the same reason; refuse cleanly here instead.
return error("fleet_handover requires a named lead pane (a resolved connection terminal) " return error("fleet_handover requires a named lead pane (a resolved connection terminal) "
+ "to open a rollover request against — an unnamed primary has none"); + "to open a rollover request against — an unnamed primary has none");
@@ -1503,6 +1590,9 @@ public final class FleetMcp {
m.put("token", p.token()); m.put("token", p.token());
m.put("handoverPath", p.handoverPath()); m.put("handoverPath", p.handoverPath());
m.put("requestedAtMillis", p.requestedAtMillis()); m.put("requestedAtMillis", p.requestedAtMillis());
MessageService.Outstanding outstanding = messages.outstanding(callerOwner);
m.put("outstandingTickets", outstanding.tickets());
m.put("openAsks", outstanding.asks());
return text(json(m)); return text(json(m));
} catch (IllegalStateException e) { } catch (IllegalStateException e) {
// leadRollover: was removed from config by a hot reload since this FleetMcp was // leadRollover: was removed from config by a hot reload since this FleetMcp was
@@ -1942,6 +2032,25 @@ public final class FleetMcp {
Map.of(), false, coordination, callerIsPrimary, leadsVisible, membersVisible); Map.of(), false, coordination, callerIsPrimary, leadsVisible, membersVisible);
} }
/**
* As below, with no pane discovery — {@code panes} is {@link PaneSource#none()} and
* {@code panesVisible} is {@code false}.
*/
static McpSchema.CallToolResult listFleet(PeerLauncher workers, SessionManager sessions, MessageService messages,
CapacitySource capacity, HealthCoverageSource healthCoverage,
LoopHealthSource loopHealth,
QuarantineSource quarantine, OutageSource outage,
LeadSeatSource leadSeats, LeadContextGauge contextGauge,
LeadConfigDirSource leadConfigDirs,
Map<String, String> leads, String selfTerm,
Map<String, String> collaborators, boolean collaboratorsVisible,
CoordinationSource coordination, boolean callerIsPrimary,
boolean leadsVisible, boolean membersVisible) {
return listFleet(workers, sessions, messages, capacity, healthCoverage, loopHealth, quarantine, outage,
leadSeats, contextGauge, leadConfigDirs, leads, selfTerm, collaborators, collaboratorsVisible,
coordination, callerIsPrimary, leadsVisible, membersVisible, PaneSource.none(), false);
}
/** /**
* The canonical implementation. {@code contextGauge} is the "lead context gauge" (see * The canonical implementation. {@code contextGauge} is the "lead context gauge" (see
* {@link LeadContextGauge}) — every wrapper overload above passes a freshly constructed one, * {@link LeadContextGauge}) — every wrapper overload above passes a freshly constructed one,
@@ -1960,6 +2069,11 @@ public final class FleetMcp {
* {@link #leadsVisibleTo}) * {@link #leadsVisibleTo})
* @param membersVisible whether this caller may see the {@code members} array (see * @param membersVisible whether this caller may see the {@code members} array (see
* {@link #membersVisibleTo}) * {@link #membersVisibleTo})
* @param panes pane-discovery facts — labels and the deliverable gate for the
* {@code panes} row; {@link PaneSource#none()} for a caller that
* does not want the row
* @param panesVisible whether this caller may see the {@code panes} array (see
* {@link #panesVisibleTo})
*/ */
static McpSchema.CallToolResult listFleet(PeerLauncher workers, SessionManager sessions, MessageService messages, static McpSchema.CallToolResult listFleet(PeerLauncher workers, SessionManager sessions, MessageService messages,
CapacitySource capacity, HealthCoverageSource healthCoverage, CapacitySource capacity, HealthCoverageSource healthCoverage,
@@ -1970,11 +2084,38 @@ public final class FleetMcp {
Map<String, String> leads, String selfTerm, Map<String, String> leads, String selfTerm,
Map<String, String> collaborators, boolean collaboratorsVisible, Map<String, String> collaborators, boolean collaboratorsVisible,
CoordinationSource coordination, boolean callerIsPrimary, CoordinationSource coordination, boolean callerIsPrimary,
boolean leadsVisible, boolean membersVisible) { boolean leadsVisible, boolean membersVisible,
PaneSource panes, boolean panesVisible) {
return listFleet(workers, sessions, messages, capacity, healthCoverage, loopHealth, quarantine, outage,
leadSeats, contextGauge, leadConfigDirs, leads, selfTerm, collaborators, collaboratorsVisible,
coordination, callerIsPrimary, leadsVisible, membersVisible, panes, panesVisible, false);
}
/**
* As above, plus fleetd #758: an observer sees the {@code panes} array too, but filtered to
* {@link CallerResolver#sendableObserverTarget} and each row reduced to the five fields an
* observer may learn — see {@code paneRows}/{@code paneRow}.
*
* @param callerIsObserver whether the {@code fleet_list} caller is an observer; every wrapper
* overload above passes {@code false}, so a test that wants the
* filtered, reduced view must call this overload with an explicit
* {@code true}
*/
static McpSchema.CallToolResult listFleet(PeerLauncher workers, SessionManager sessions, MessageService messages,
CapacitySource capacity, HealthCoverageSource healthCoverage,
LoopHealthSource loopHealth,
QuarantineSource quarantine, OutageSource outage,
LeadSeatSource leadSeats, LeadContextGauge contextGauge,
LeadConfigDirSource leadConfigDirs,
Map<String, String> leads, String selfTerm,
Map<String, String> collaborators, boolean collaboratorsVisible,
CoordinationSource coordination, boolean callerIsPrimary,
boolean leadsVisible, boolean membersVisible,
PaneSource panes, boolean panesVisible, boolean callerIsObserver) {
try { try {
// Neither row's assembly (leadView/memberCapacityView probing herdr for live status) // Neither row's assembly (leadView/memberCapacityView probing herdr for live status)
// runs unless at least one of them needs the live-agent lookup backing it. // runs unless at least one of them needs the live-agent lookup backing it.
Map<String, Agent> live = (leadsVisible || membersVisible) Map<String, Agent> live = (leadsVisible || membersVisible || panesVisible)
? workers.list().stream() ? workers.list().stream()
.map(Agent.class::cast) .map(Agent.class::cast)
.filter(a -> a.terminalId() != null) .filter(a -> a.terminalId() != null)
@@ -2018,6 +2159,10 @@ public final class FleetMcp {
.map(e -> collaboratorRow(e.getKey(), e.getValue())) .map(e -> collaboratorRow(e.getKey(), e.getValue()))
.toList()); .toList());
} }
// gate BEFORE assembling the row, so the key is absent rather than present-and-empty.
if (panesVisible) {
result.put("panes", paneRows(live, roster, leads, collaborators, panes, callerIsObserver));
}
// fleetd #439: coordinator/coordinatorView is lead-to-lead coordination state and must // fleetd #439: coordinator/coordinatorView is lead-to-lead coordination state and must
// never reach a worker or an architect -- gate BEFORE assembling it, not after, so the // never reach a worker or an architect -- gate BEFORE assembling it, not after, so the
// key is absent rather than present-and-empty. // key is absent rather than present-and-empty.
@@ -2343,6 +2488,116 @@ public final class FleetMcp {
return m; return m;
} }
/**
* {@code panes.tabLabels()}'s herdr scan, or an empty map on a {@code HerdrException} — a
* missing label must not cost the {@code leads}/{@code members}/{@code capacity}/
* {@code coordinator} rows that share {@code listFleet}'s own {@code catch}.
*/
private static Map<String, String> tabLabelsOrEmpty(PaneSource panes) {
try {
return panes.tabLabels().get();
} catch (HerdrException e) {
return Map.of();
}
}
/** As {@link #tabLabelsOrEmpty}, for {@code panes.workspaceLabels()}. */
private static Map<String, String> workspaceLabelsOrEmpty(PaneSource panes) {
try {
return panes.workspaceLabels().get();
} catch (HerdrException e) {
return Map.of();
}
}
/**
* One row per herdr-tracked agent pane, sorted by terminal id for a stable order. {@code live}
* is the same terminal-keyed {@link Agent} map {@code leadView}/{@code memberCapacityView}
* already read, so a pane neither configured as a lead nor spawned as a member — a hand-opened
* tab — still gets a row here.
*
* <p>fleetd #758: for an observer caller ({@code observerView}), the rows are filtered to
* {@link PaneSource#sendableToObserver} before being built, and each row is reduced — see
* {@code paneRow}.
*/
private static List<Map<String, Object>> paneRows(Map<String, Agent> live, List<MemberSession> roster,
Map<String, String> leads, Map<String, String> collaborators, PaneSource panes,
boolean observerView) {
final Map<String, String> tabLabels = tabLabelsOrEmpty(panes);
final Map<String, String> workspaceLabels = workspaceLabelsOrEmpty(panes);
Map<String, MemberSession> byTerminal = roster.stream()
.filter(s -> s.terminalId() != null)
.collect(Collectors.toMap(MemberSession::terminalId, Function.identity(), (_, b) -> b));
return live.values().stream()
.filter(a -> !observerView || panes.sendableToObserver().test(a.terminalId()))
.sorted(Comparator.comparing(Agent::terminalId))
.map(a -> paneRow(a, byTerminal.get(a.terminalId()), leads, collaborators, tabLabels,
workspaceLabels, panes, observerView))
.toList();
}
/**
* @param session the roster entry for this pane's terminal, or {@code null} for a pane the
* daemon never spawned as a member (a hand-opened tab, or a configured lead)
* @param tabLabels tab id → its herdr display label; a tab absent here, or carrying a
* {@code null} label itself, projects as a {@code null} "label"
* @param workspaceLabels workspace id → its herdr display label (the space name); a workspace
* absent here, or carrying a {@code null} label itself, projects as a
* {@code null} "workspaceLabel"
* @param observerView fleetd #758: an observer's row carries only {@code sessionId}, {@code
* label}, {@code status}, {@code role}, {@code deliverable} — never {@code
* paneId} (the {@code fleet_stop} handle), {@code workspaceId},
* {@code workspaceLabel}, {@code tabId}, {@code agentType}, or {@code cwd}
* (a member's worktree path is the lead's business)
*/
private static Map<String, Object> paneRow(Agent a, MemberSession session, Map<String, String> leads,
Map<String, String> collaborators, Map<String, String> tabLabels,
Map<String, String> workspaceLabels, PaneSource panes, boolean observerView) {
Map<String, Object> m = new LinkedHashMap<>();
m.put("sessionId", a.terminalId());
if (!observerView) {
m.put("paneId", a.paneId());
m.put("workspaceId", a.workspaceId());
m.put("workspaceLabel", a.workspaceId() == null ? null : workspaceLabels.get(a.workspaceId()));
m.put("tabId", a.tabId());
}
m.put("label", a.tabId() == null ? null : tabLabels.get(a.tabId()));
if (!observerView) {
m.put("agentType", a.agentType());
}
m.put("status", a.status() == null ? "unknown" : a.status().name().toLowerCase());
m.put("role", paneRole(a.terminalId(), session, leads, collaborators, panes));
m.put("deliverable", panes.deliverable().test(a.terminalId()));
if (!observerView && session != null && session.cwd() != null) {
m.put("cwd", session.cwd());
}
return m;
}
/**
* The role this pane resolves as: a spawned member's own {@link MemberRole}, else "lead" for a
* configured but currently-unoccupied lead pane, else "architect" for a pane bound to a
* configured architect slot with no live member session, else "collaborator" for a configured
* but currently-unoccupied collaborator tab, else "observer" for a pane this daemon neither
* spawned nor configured.
*/
private static String paneRole(String terminal, MemberSession session, Map<String, String> leads,
Map<String, String> collaborators, PaneSource panes) {
if (session != null) {
return session.role().wireName();
}
if (leads.containsKey(terminal)) {
return "lead";
}
if (panes.architectSlot().test(terminal)) {
return "architect";
}
if (collaborators.containsKey(terminal)) {
return "collaborator";
}
return "observer";
}
/** /**
* Reads the lead context gauge for one lead. {@code configDir} is this lead's configured * Reads the lead context gauge for one lead. {@code configDir} is this lead's configured
* {@code CLAUDE_CONFIG_DIR} override (see {@link LeadConfigDirSource}), derived from * {@code CLAUDE_CONFIG_DIR} override (see {@link LeadConfigDirSource}), derived from
@@ -2548,8 +2803,8 @@ public final class FleetMcp {
+ "orchestrators, each with its sessionId (the address to fleet_send to), " + "orchestrators, each with its sessionId (the address to fleet_send to), "
+ "name, live status, and 'self': true on your own row; this is how you " + "name, live status, and 'self': true on your own row; this is how you "
+ "discover a peer lead without being told its address. 'members' are the " + "discover a peer lead without being told its address. 'members' are the "
+ "sessions delegated to — each with sessionId, paneId, role (architect/dev/" + "sessions delegated to — each with sessionId, paneId, role (" + MemberRole.wireNames()
+ "reviewer), profile (the backend it runs on), state, optional " + "), profile (the backend it runs on), state, optional "
+ "worktree/branch/owner/agentSessionId, and live herdr status. agentSessionId, " + "worktree/branch/owner/agentSessionId, and live herdr status. agentSessionId, "
+ "when present, is the id to pass as fleet_spawn's resumeSessionId to relaunch " + "when present, is the id to pass as fleet_spawn's resumeSessionId to relaunch "
+ "onto that same conversation. It is ABSENT — not a guess — for a member fleetd " + "onto that same conversation. It is ABSENT — not a guess — for a member fleetd "
@@ -2630,17 +2885,22 @@ public final class FleetMcp {
private static McpSchema.Tool handoverTool() { private static McpSchema.Tool handoverTool() {
return tool(FleetTool.HANDOVER.wireName(), return tool(FleetTool.HANDOVER.wireName(),
"Replace your OWN lead session once its context is full: write a handover file, " "Replace your OWN lead session once its context is full: write a handover file, "
+ "then use this to have fleetd clear your pane and bootstrap a fresh lead " + "then use this to have fleetd end your pane's process and relaunch a fresh "
+ "session against it. Four actions: 'open' (requests a token and the " + "lead session bootstrapped against it. Four actions: 'open' (requests a "
+ "handoverPath you must write the handover file to before confirming), " + "token and the handoverPath you must write the handover file to before "
+ "'confirm' (validates every gate and — only if every one passes — schedules " + "confirming — the response also lists outstandingTickets and openAsks, "
+ "the roll; it does NOT itself clear the pane, the roll runs once this call's " + "your own async delegations and fleet_ask turns, so their ids can go into "
+ "own turn ends), 'cancel' (drops a pending request without rolling), and " + "the handover file too), 'confirm' (validates every gate and — only if every one "
+ "'status' (read-only: what happened to a token after 'confirm' — still " + "passes — schedules the roll; it does NOT itself end your pane, the roll "
+ "running (approved but not finished yet), the roll completed, the calling " + "runs once this call's own turn ends), 'cancel' (drops a pending request "
+ "turn never settled within turnSettleSeconds so no /clear was ever sent, or " + "without rolling), and 'status' (read-only: what happened to a token after "
+ "/clear itself never settled so bootstrapText was never sent; never " + "'confirm' — still running (approved but not finished yet), the roll "
+ "schedules, cancels or retries anything). Primary-only. " + "completed, the calling turn never settled within turnSettleSeconds so "
+ "nothing was touched, the old pane never confirmed dead so no relaunch was "
+ "attempted, the relaunch itself failed, the fresh pane never became ready "
+ "so bootstrapText was never sent, or the fresh pane became ready and was "
+ "bootstrapped but was never recognised as a live lead; never schedules, "
+ "cancels or retries anything). Primary-only. "
+ "There is deliberately no terminal/session/leadTerminal parameter: the pane " + "There is deliberately no terminal/session/leadTerminal parameter: the pane "
+ "to roll is always resolved from YOUR OWN connection, never a value you " + "to roll is always resolved from YOUR OWN connection, never a value you "
+ "pass, so you can only ever roll yourself — never another lead. Requires " + "pass, so you can only ever roll yourself — never another lead. Requires "
@@ -6,6 +6,7 @@ import org.slf4j.LoggerFactory;
import java.util.Optional; import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicReference; import java.util.concurrent.atomic.AtomicReference;
import java.util.function.Function;
/** /**
* Single-slot, thread-safe registry for the primary's herdr {@code terminal_id}. * Single-slot, thread-safe registry for the primary's herdr {@code terminal_id}.
@@ -18,13 +19,25 @@ import java.util.concurrent.atomic.AtomicReference;
* <p>The push loop ({@code ReplyPushLoop}) uses {@link #isKnown()} to decide * <p>The push loop ({@code ReplyPushLoop}) uses {@link #isKnown()} to decide
* whether active nudging is possible; an empty registry means the primary is * whether active nudging is possible; an empty registry means the primary is
* off-host or non-herdr and delivery falls back to pull. * off-host or non-herdr and delivery falls back to pull.
*
* <p><strong>A learned terminal can go stale; a configured lead's name cannot.</strong> A lead that
* is rolled (a fresh pane replacing the old one) keeps its name but gets a new {@code terminal_id}.
* So every terminal this class learns — the singleton and each per-target delegation — is recorded
* together with the delegating lead's name, when the caller carries one. {@link
* #currentPrimaryTerminal()} and {@link #nudgeTargetFor(String)} resolve that name back to a
* terminal through the live {@code currentTerminalForName} lookup before falling back to the
* terminal that was actually recorded. A caller with no name (an unnamed primary, an architect, a
* collaborator — none of those are leads a lookup keyed on lead names can resolve) is tracked by
* terminal alone, exactly as before this indirection existed.
*/ */
public final class PrimaryRegistry { public final class PrimaryRegistry {
private static final Logger log = LoggerFactory.getLogger(PrimaryRegistry.class); private static final Logger log = LoggerFactory.getLogger(PrimaryRegistry.class);
private final AtomicReference<String> terminal = new AtomicReference<>(); private final AtomicReference<String> terminal = new AtomicReference<>();
private final AtomicReference<String> primaryName = new AtomicReference<>();
private final boolean pinned; private final boolean pinned;
private final Function<String, String> currentTerminalForName;
/** /**
* CB-532: worker terminal → the lead that delegated to it. The single slot above answers "who is * CB-532: worker terminal → the lead that delegated to it. The single slot above answers "who is
@@ -33,12 +46,32 @@ public final class PrimaryRegistry {
* other lead's delegations. This map answers the question that actually matters — "who is * other lead's delegations. This map answers the question that actually matters — "who is
* waiting on THIS worker" — and is what lets {@code primary.terminal} be retired. * waiting on THIS worker" — and is what lets {@code primary.terminal} be retired.
*/ */
private final ConcurrentHashMap<String, String> leadByTarget = new ConcurrentHashMap<>(); private final ConcurrentHashMap<String, Delegation> leadByTarget = new ConcurrentHashMap<>();
/** A recorded delegator: the terminal learned from call traffic, and its name, if it has one. */
private record Delegation(String terminal, String name) {
}
/** /**
* @param pinnedTerminal an optional pinned terminal from config ({@code null}/blank = unpinned) * @param pinnedTerminal an optional pinned terminal from config ({@code null}/blank = unpinned)
*/ */
public PrimaryRegistry(String pinnedTerminal) { public PrimaryRegistry(String pinnedTerminal) {
this(pinnedTerminal, name -> null);
}
/**
* As above, with a live {@code lead name → current terminal} lookup — normally the inverse of
* the same {@code terminal_id → lead name} supplier {@code CallerResolver} and the lead-tab
* scan already read. A lookup that cannot place a name (it is not a currently recognised lead,
* or no lookup is wired) returns {@code null}, and every resolution here falls back to the
* terminal that was actually recorded.
*
* @param pinnedTerminal an optional pinned terminal from config ({@code null}/blank =
* unpinned)
* @param currentTerminalForName lead name → its current terminal, or {@code null} if that name
* is not a currently recognised lead
*/
public PrimaryRegistry(String pinnedTerminal, Function<String, String> currentTerminalForName) {
if (pinnedTerminal != null && !pinnedTerminal.isBlank()) { if (pinnedTerminal != null && !pinnedTerminal.isBlank()) {
this.terminal.set(pinnedTerminal); this.terminal.set(pinnedTerminal);
this.pinned = true; this.pinned = true;
@@ -46,19 +79,31 @@ public final class PrimaryRegistry {
} else { } else {
this.pinned = false; this.pinned = false;
} }
this.currentTerminalForName = currentTerminalForName != null ? currentTerminalForName : name -> null;
} }
/** /**
* Record a terminal_id. No-op when: * Record a terminal_id, with no lead name. No-op when:
* <ul> * <ul>
* <li>the registry is pinned (config override), * <li>the registry is pinned (config override),
* <li>{@code terminalId} is {@code null} or blank (non-herdr caller). * <li>{@code terminalId} is {@code null} or blank (non-herdr caller).
* </ul> * </ul>
*/ */
public void record(String terminalId) { public void record(String terminalId) {
record(terminalId, null);
}
/**
* As {@link #record(String)}, additionally recording the caller's name — present for a
* configured lead, {@code null} for an unnamed primary. The name is what lets {@link
* #currentPrimaryTerminal()} keep nudging the same lead across a roll even though its terminal
* changed.
*/
public void record(String terminalId, String name) {
if (pinned) return; if (pinned) return;
if (terminalId == null || terminalId.isBlank()) return; if (terminalId == null || terminalId.isBlank()) return;
String prev = terminal.getAndSet(terminalId); String prev = terminal.getAndSet(terminalId);
primaryName.set(blankToNull(name));
if (prev == null) { if (prev == null) {
log.debug("primary terminal learned: {}", terminalId); log.debug("primary terminal learned: {}", terminalId);
} else if (!prev.equals(terminalId)) { } else if (!prev.equals(terminalId)) {
@@ -67,7 +112,8 @@ public final class PrimaryRegistry {
} }
/** /**
* Record that {@code leadTerminal} owns the accepted delegation of worker {@code target} (CB-532). * Record that {@code leadTerminal} owns the accepted delegation of worker {@code target}
* (CB-532), with no lead name.
* *
* <p>Called from the {@code MessageService} accepted-delivery hook — only after a send has won * <p>Called from the {@code MessageService} accepted-delivery hook — only after a send has won
* the session's send lock and queued delivery — where both halves are known (CB-548). It is * the session's send lock and queued delivery — where both halves are known (CB-548). It is
@@ -77,10 +123,19 @@ public final class PrimaryRegistry {
* lead that most recently delegated to it, which is the one waiting. * lead that most recently delegated to it, which is the one waiting.
*/ */
public void recordDelegation(String target, String leadTerminal) { public void recordDelegation(String target, String leadTerminal) {
recordDelegation(target, leadTerminal, null);
}
/**
* As {@link #recordDelegation(String, String)}, additionally recording the delegating lead's
* name when the caller carries one. See {@link #record(String, String)} for why the name
* matters.
*/
public void recordDelegation(String target, String leadTerminal, String leadName) {
if (target == null || target.isBlank() || leadTerminal == null || leadTerminal.isBlank()) { if (target == null || target.isBlank() || leadTerminal == null || leadTerminal.isBlank()) {
return; return;
} }
leadByTarget.put(target, leadTerminal); leadByTarget.put(target, new Delegation(leadTerminal, blankToNull(leadName)));
} }
/** Forget a worker's delegating lead — call on release, so a torn-down session leaks nothing. */ /** Forget a worker's delegating lead — call on release, so a torn-down session leaks nothing. */
@@ -99,19 +154,59 @@ public final class PrimaryRegistry {
* recorded delegation there is no right answer, so this returns empty rather than guessing — * recorded delegation there is no right answer, so this returns empty rather than guessing —
* delivery degrades to pull, which is exactly what the durable inbox is for, instead of * delivery degrades to pull, which is exactly what the durable inbox is for, instead of
* interrupting the wrong lead with someone else's result. * interrupting the wrong lead with someone else's result.
*
* <p>A delegation recorded with a name is resolved to that lead's <em>current</em> terminal
* first — see {@link #currentTerminalForName} — so a lead that has since been rolled is still
* reachable here, not just the pane that delegated the work originally.
*/ */
public Optional<String> nudgeTargetFor(String target) { public Optional<String> nudgeTargetFor(String target) {
String lead = target == null ? null : leadByTarget.get(target); Delegation delegation = target == null ? null : leadByTarget.get(target);
return lead != null ? Optional.of(lead) : Optional.ofNullable(terminal.get()); if (delegation != null) {
return Optional.of(resolveCurrent(delegation.terminal(), delegation.name()));
}
return currentPrimaryTerminal();
} }
/** The known primary terminal, or empty if not yet learned (and not pinned). */ /**
* The known primary terminal, or empty if not yet learned (and not pinned) — the raw value as
* it was recorded, with no attempt to resolve a named lead's current pane. Callers that need a
* nudge destination which survives a lead roll want {@link #currentPrimaryTerminal()} instead.
*/
public Optional<String> primaryTerminal() { public Optional<String> primaryTerminal() {
return Optional.ofNullable(terminal.get()); return Optional.ofNullable(terminal.get());
} }
/**
* The terminal to nudge for the singleton primary right now: the recorded name resolved to its
* current terminal when one was recorded and is still a recognised lead, otherwise the terminal
* that was actually recorded — empty only when nothing has been learned or pinned at all.
*/
public Optional<String> currentPrimaryTerminal() {
String learned = terminal.get();
if (learned == null) {
return Optional.empty();
}
return Optional.of(resolveCurrent(learned, primaryName.get()));
}
/** {@code true} once a terminal has been recorded (or was pinned at construction). */ /** {@code true} once a terminal has been recorded (or was pinned at construction). */
public boolean isKnown() { public boolean isKnown() {
return terminal.get() != null; return terminal.get() != null;
} }
/**
* {@code learnedTerminal}, unless {@code name} is non-null and {@code currentTerminalForName}
* currently places that name at a different, live terminal — in which case the live one wins.
*/
private String resolveCurrent(String learnedTerminal, String name) {
if (name == null) {
return learnedTerminal;
}
String current = currentTerminalForName.apply(name);
return current != null && !current.isBlank() ? current : learnedTerminal;
}
private static String blankToNull(String s) {
return s == null || s.isBlank() ? null : s;
}
} }
@@ -445,27 +445,6 @@ public final class CompositePeerLauncher implements PeerLauncher {
+ " distinct candidate(s): " + String.join(", ", unreachable)); + " distinct candidate(s): " + String.join(", ", unreachable));
} }
/**
* Refuse an explicit-profile spawn when the profile is at its {@code maxLoad} cap.
*
* <p>maxLoad is a documented, unconditional capacity limit (see {@code FleetConfig.Profile#maxLoad}),
* and the charter makes explicit-profile spawns the normal path — so enforcing it only in placement
* ({@code PlacementPolicyUtil}, package-private, hence not linked) would leave the cap dead config
* on every call that names a profile. Same rule as placement: {@code live >= cap} is at capacity.
*
* <p>Deliberately no fallback to another profile: the caller named {@code profile} for a cost/model
* reason, and silently re-routing a paid-tier (subscription) request elsewhere is worse than
* refusing it. A caller that wants placement should omit the profile and let the policy pick.
*
* <p>Known TOCTOU limitation — documented, not fixed. {@link #liveCount} is read outside any lock and
* {@code SessionManager} registers a session only after {@code launcher.spawn} returns, so two
* genuinely concurrent spawns can both pass this check. The race already exists on the placement
* path. Closing it needs slot reservation in the registry; serializing spawn here would block on
* the readiness gate and is a far worse trade.
*
* @param profile the profile the caller explicitly named
* @throws PlacementException when the profile is at capacity
*/
/** /**
* Refuse an explicit-profile spawn whose credential is quarantined (CB-578 stage B): a prior * Refuse an explicit-profile spawn whose credential is quarantined (CB-578 stage B): a prior
* {@code BACKEND_EXHAUSTED} classification on this profile, or on another profile sharing its * {@code BACKEND_EXHAUSTED} classification on this profile, or on another profile sharing its
@@ -527,6 +506,27 @@ public final class CompositePeerLauncher implements PeerLauncher {
.collect(Collectors.toSet()); .collect(Collectors.toSet());
} }
/**
* Refuse an explicit-profile spawn when the profile is at its {@code maxLoad} cap.
*
* <p>maxLoad is a documented, unconditional capacity limit (see {@code FleetConfig.Profile#maxLoad}),
* and the charter makes explicit-profile spawns the normal path — so enforcing it only in placement
* ({@code PlacementPolicyUtil}, package-private, hence not linked) would leave the cap dead config
* on every call that names a profile. Same rule as placement: {@code live >= cap} is at capacity.
*
* <p>No fallback to another profile: the caller named {@code profile} for a cost/model
* reason, and silently re-routing a paid-tier (subscription) request elsewhere is worse than
* refusing it. A caller that wants placement should omit the profile and let the policy pick.
*
* <p>Known TOCTOU limitation — documented, not fixed. {@link #liveCount} is read outside any lock and
* {@code SessionManager} registers a session only after {@code launcher.spawn} returns, so two
* genuinely concurrent spawns can both pass this check. The race already exists on the placement
* path. Closing it needs slot reservation in the registry; serializing spawn here would block on
* the readiness gate and is a far worse trade.
*
* @param profile the profile the caller explicitly named
* @throws PlacementException when the profile is at capacity
*/
private void enforceMaxLoad(String profile) { private void enforceMaxLoad(String profile) {
// Absent config, or a config whose maxLoad normalized to null (ABSENT ⇒ unlimited at load), // Absent config, or a config whose maxLoad normalized to null (ABSENT ⇒ unlimited at load),
// means no cap — never cap what wasn't configured. Note "non-positive ⇒ unlimited" was true // means no cap — never cap what wasn't configured. Note "non-positive ⇒ unlimited" was true
@@ -474,7 +474,6 @@ public final class EnvAllowListScrub {
} }
} }
/** Best-effort recursive delete; failures are swallowed — JVM-exit cleanup is the backstop. */
/** /**
* Remove generated directories left behind by an earlier daemon process. * Remove generated directories left behind by an earlier daemon process.
* *
@@ -511,6 +510,7 @@ public final class EnvAllowListScrub {
} }
} }
/** Best-effort recursive delete; failures are swallowed — JVM-exit cleanup is the backstop. */
static void deleteRecursively(Path dir) { static void deleteRecursively(Path dir) {
if (dir == null || !Files.exists(dir)) { if (dir == null || !Files.exists(dir)) {
return; return;
@@ -2,6 +2,7 @@ package dev.ltms.fleet.msg;
import dev.ltms.fleet.herdr.AgentControl; import dev.ltms.fleet.herdr.AgentControl;
import dev.ltms.fleet.herdr.AgentStatus; import dev.ltms.fleet.herdr.AgentStatus;
import dev.ltms.fleet.herdr.PromptBox;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
@@ -23,7 +24,8 @@ import java.util.function.Supplier;
* <p><strong>Status-gated, exactly like {@link ReplyPushLoop}.</strong> A pane may only be injected * <p><strong>Status-gated, exactly like {@link ReplyPushLoop}.</strong> A pane may only be injected
* into at a turn boundary ({@link AgentStatus#injectable()} — idle, blocked or done); pasting into * into at a turn boundary ({@link AgentStatus#injectable()} — idle, blocked or done); pasting into
* a live turn corrupts it. So a tick that finds the lead busy simply does nothing and comes back * a live turn corrupts it. So a tick that finds the lead busy simply does nothing and comes back
* later. * later. The same holds for a lead whose prompt box holds unsubmitted text ({@link PromptBox}) —
* delivering there would submit the operator's half-typed line along with the message.
* *
* <p><strong>Ack only after delivery.</strong> A message is acked — removed from the broker — only * <p><strong>Ack only after delivery.</strong> A message is acked — removed from the broker — only
* once {@link AgentControl#send} has actually put it in the pane. Anything not delivered (no lead * once {@link AgentControl#send} has actually put it in the pane. Anything not delivered (no lead
@@ -52,6 +54,7 @@ public final class LeadCoordLoop {
private final LeadChannel channel; private final LeadChannel channel;
private final AgentControl agents; private final AgentControl agents;
private final PromptBox promptBox;
private final Supplier<Map<String, String>> leads; private final Supplier<Map<String, String>> leads;
private final ScheduledExecutorService scheduler; private final ScheduledExecutorService scheduler;
private final long intervalMs; private final long intervalMs;
@@ -73,6 +76,7 @@ public final class LeadCoordLoop {
ScheduledExecutorService scheduler, long intervalMs) { ScheduledExecutorService scheduler, long intervalMs) {
this.channel = channel; this.channel = channel;
this.agents = agents; this.agents = agents;
this.promptBox = new PromptBox(agents);
this.leads = leads; this.leads = leads;
this.scheduler = scheduler; this.scheduler = scheduler;
this.intervalMs = intervalMs; this.intervalMs = intervalMs;
@@ -149,6 +153,11 @@ public final class LeadCoordLoop {
lead, status, held.size()); lead, status, held.size());
return; return;
} }
if (!promptBox.clearToSubmit(lead)) {
log.debug("lead coordination: lead {} has unsubmitted text in its prompt box, holding {} message(s)",
lead, held.size());
return;
}
try { try {
agents.send(lead, DELIVERY_FORMAT.formatted(msg.from(), msg.content())); agents.send(lead, DELIVERY_FORMAT.formatted(msg.from(), msg.content()));
} catch (RuntimeException e) { } catch (RuntimeException e) {
@@ -2,6 +2,7 @@ package dev.ltms.fleet.msg;
import dev.ltms.fleet.herdr.AgentControl; import dev.ltms.fleet.herdr.AgentControl;
import dev.ltms.fleet.herdr.AgentStatus; import dev.ltms.fleet.herdr.AgentStatus;
import dev.ltms.fleet.herdr.PromptBox;
import dev.ltms.fleet.lead.LeadContextGauge; import dev.ltms.fleet.lead.LeadContextGauge;
import dev.ltms.fleet.mcp.PrimaryRegistry; import dev.ltms.fleet.mcp.PrimaryRegistry;
import dev.ltms.fleet.metrics.FleetMetrics; import dev.ltms.fleet.metrics.FleetMetrics;
@@ -33,7 +34,7 @@ import java.util.function.Supplier;
* no such block it is never constructed, so upgrading the daemon cannot silently acquire a behaviour * no such block it is never constructed, so upgrading the daemon cannot silently acquire a behaviour
* that spends the operator's model subscription on its own initiative (constraint 1). * that spends the operator's model subscription on its own initiative (constraint 1).
* *
* <p>Four invariants keep it from becoming a runaway subscription burner: * <p>Five invariants keep it from becoming a runaway subscription burner:
* <ol> * <ol>
* <li><b>Status-gated</b> — a {@code WORKING} lead is making progress and is never touched; only an * <li><b>Status-gated</b> — a {@code WORKING} lead is making progress and is never touched; only an
* injectable (idle/done/blocked) lead is even considered (constraint 2).</li> * injectable (idle/done/blocked) lead is even considered (constraint 2).</li>
@@ -45,6 +46,8 @@ import java.util.function.Supplier;
* <li><b>Never races {@link ReplyPushLoop}</b> — while that loop is actively nudging any target this * <li><b>Never races {@link ReplyPushLoop}</b> — while that loop is actively nudging any target this
* loop stands down, so two competing injections never start two turns in the same pane * loop stands down, so two competing injections never start two turns in the same pane
* (constraint 6).</li> * (constraint 6).</li>
* <li><b>Never submits the operator's draft</b> — a nudge is held while the lead's prompt box holds
* unsubmitted text ({@link PromptBox}), because the delivery pastes and submits in one call.</li>
* </ol> * </ol>
* *
* <p><b>fleetd #609 — context-high notice.</b> Optionally ({@code contextHighNudge}, opt-in like the * <p><b>fleetd #609 — context-high notice.</b> Optionally ({@code contextHighNudge}, opt-in like the
@@ -65,6 +68,7 @@ public final class LeadHeartbeatLoop {
private final PrimaryRegistry primaryRegistry; private final PrimaryRegistry primaryRegistry;
private final AgentControl agents; private final AgentControl agents;
private final PromptBox promptBox;
private final ReplyInbox inbox; private final ReplyInbox inbox;
private final Supplier<List<MemberSession>> roster; private final Supplier<List<MemberSession>> roster;
private final ReplyPushLoop pushLoop; private final ReplyPushLoop pushLoop;
@@ -135,6 +139,7 @@ public final class LeadHeartbeatLoop {
boolean requireOperatorConfirm) { boolean requireOperatorConfirm) {
this.primaryRegistry = primaryRegistry; this.primaryRegistry = primaryRegistry;
this.agents = agents; this.agents = agents;
this.promptBox = new PromptBox(agents);
this.inbox = inbox; this.inbox = inbox;
this.roster = roster; this.roster = roster;
this.pushLoop = pushLoop; this.pushLoop = pushLoop;
@@ -187,7 +192,9 @@ public final class LeadHeartbeatLoop {
/** Idle past the quiet period with nothing pending and the cap exhausted — stop until new state appears. */ /** Idle past the quiet period with nothing pending and the cap exhausted — stop until new state appears. */
QUIET_DONE, QUIET_DONE,
/** {@link ReplyPushLoop} is actively nudging — stand aside rather than start a competing turn. */ /** {@link ReplyPushLoop} is actively nudging — stand aside rather than start a competing turn. */
STAND_DOWN STAND_DOWN,
/** The lead's prompt box holds unsubmitted text — hold the nudge rather than submit that text. */
DRAFT_HELD
} }
/** /**
@@ -307,12 +314,12 @@ public final class LeadHeartbeatLoop {
* (mirroring {@link ReplyPushLoop#tick(String)}) so tests can drive it directly with a fake clock and a * (mirroring {@link ReplyPushLoop#tick(String)}) so tests can drive it directly with a fake clock and a
* fake {@link AgentControl} instead of racing the scheduler thread. */ * fake {@link AgentControl} instead of racing the scheduler thread. */
void tick() { void tick() {
boolean leadKnown = primaryRegistry.primaryTerminal().isPresent(); boolean leadKnown = primaryRegistry.currentPrimaryTerminal().isPresent();
FleetState fleet = snapshot(inbox, roster); FleetState fleet = snapshot(inbox, roster);
AgentStatus status = AgentStatus.UNKNOWN; AgentStatus status = AgentStatus.UNKNOWN;
LeadContextGauge.Reading reading = LeadContextGauge.Reading.unknown(); LeadContextGauge.Reading reading = LeadContextGauge.Reading.unknown();
if (leadKnown) { if (leadKnown) {
String leadTerminal = primaryRegistry.primaryTerminal().orElseThrow(); String leadTerminal = primaryRegistry.currentPrimaryTerminal().orElseThrow();
try { try {
status = agents.status(leadTerminal); status = agents.status(leadTerminal);
} catch (RuntimeException e) { } catch (RuntimeException e) {
@@ -330,6 +337,7 @@ public final class LeadHeartbeatLoop {
idleSinceNanos == NOT_IDLE ? null : idleSinceNanos, idleSinceNanos == NOT_IDLE ? null : idleSinceNanos,
quietCount, status, pushLoop.isActive(), leadKnown, fleet, quietCount, status, pushLoop.isActive(), leadKnown, fleet,
reading.state(), contextNotified); reading.state(), contextNotified);
d = holdIfOperatorIsTyping(d);
applyDecision(d); applyDecision(d);
switch (d.action()) { switch (d.action()) {
case INJECT -> injectNudge(d, fleet, reading); case INJECT -> injectNudge(d, fleet, reading);
@@ -337,11 +345,33 @@ public final class LeadHeartbeatLoop {
countNudge("exhausted"); countNudge("exhausted");
contextNotified = d.contextNotified(); contextNotified = d.contextNotified();
} }
case WAIT_IDLE, LEAD_BUSY, STAND_DOWN -> contextNotified = d.contextNotified(); case WAIT_IDLE, LEAD_BUSY, STAND_DOWN, DRAFT_HELD -> contextNotified = d.contextNotified();
} }
scheduleNext(); scheduleNext();
} }
/**
* Turn a decision to inject into {@link Action#DRAFT_HELD} when the lead's prompt box holds text
* the operator has not submitted. The pane read happens only for a decision that would otherwise
* send, so a busy or debouncing lead costs no extra herdr call.
*
* <p>The held decision carries this tick's idle window but the <em>pre-tick</em> quiet count and
* context latch: nothing reached the pane, so neither the quiet budget nor the one context notice
* per stretch may be spent on it.
*/
private Decision holdIfOperatorIsTyping(Decision d) {
if (d.action() != Action.INJECT) {
return d;
}
var lead = primaryRegistry.currentPrimaryTerminal();
if (lead.isEmpty() || promptBox.clearToSubmit(lead.get())) {
return d;
}
log.debug("idle-heartbeat: lead {} has unsubmitted text in its prompt box, holding the nudge",
lead.get());
return new Decision(Action.DRAFT_HELD, d.idleSinceNanos(), quietCount, contextNotified);
}
/** /**
* Persist the idle/quiet state a decision returned, so the next tick starts from it. * Persist the idle/quiet state a decision returned, so the next tick starts from it.
* *
@@ -367,7 +397,7 @@ public final class LeadHeartbeatLoop {
// itself should be built from. Otherwise a HIGH stretch that is still latched would never see the // itself should be built from. Otherwise a HIGH stretch that is still latched would never see the
// notice at all, defeating the very check this fixes. // notice at all, defeating the very check this fixes.
String notice = contextNotice(contextHighNudge, reading, contextNotified, requireOperatorConfirm); String notice = contextNotice(contextHighNudge, reading, contextNotified, requireOperatorConfirm);
var lead = primaryRegistry.primaryTerminal(); var lead = primaryRegistry.currentPrimaryTerminal();
boolean sent = lead.isPresent() && trySend(lead.get(), fleet.nudgeText() + notice, notice); boolean sent = lead.isPresent() && trySend(lead.get(), fleet.nudgeText() + notice, notice);
// The latch becomes true only when all three hold: decide() chose to notify, a notice was // The latch becomes true only when all three hold: decide() chose to notify, a notice was
// actually included in the text, and the send reached the pane without throwing. Whenever no // actually included in the text, and the send reached the pane without throwing. Whenever no
@@ -1,5 +1,6 @@
package dev.ltms.fleet.msg; package dev.ltms.fleet.msg;
import dev.ltms.fleet.auth.Principal;
import dev.ltms.fleet.herdr.AgentControl; import dev.ltms.fleet.herdr.AgentControl;
import dev.ltms.fleet.herdr.AgentStatus; import dev.ltms.fleet.herdr.AgentStatus;
import dev.ltms.fleet.herdr.HerdrRouter; import dev.ltms.fleet.herdr.HerdrRouter;
@@ -11,6 +12,7 @@ import org.slf4j.LoggerFactory;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.List; import java.util.List;
import java.util.Objects;
import java.util.UUID; import java.util.UUID;
import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionException; import java.util.concurrent.CompletionException;
@@ -282,18 +284,15 @@ public final class MessageService {
*/ */
private volatile boolean askTimedOut; private volatile boolean askTimedOut;
/** /**
* The terminal of the caller whose {@code fleet_send{wait:false}} created this ticket, or * The owner key of the caller whose {@code fleet_send{wait:false}} created this ticket, or
* {@code null} when that caller had no terminal (the unnamed primary) or the ticket was * {@code null} for the unnamed primary and overloads that do not record a caller.
* created through an overload that does not record one. {@link #poll(String, String)}
* compares a polling caller's own terminal against this field before handing back the
* ticket's state.
*/ */
private final String creatorTerminal; private final String creatorOwner;
private Task(String ticket, String target, LongSupplier nowNanos, String creatorTerminal) { private Task(String ticket, String target, LongSupplier nowNanos, String creatorOwner) {
this.ticket = ticket; this.ticket = ticket;
this.target = target; this.target = target;
this.creatorTerminal = creatorTerminal; this.creatorOwner = creatorOwner;
this.createdNanos = nowNanos.getAsLong(); this.createdNanos = nowNanos.getAsLong();
future.whenComplete((reply, ex) -> completedNanos = nowNanos.getAsLong()); future.whenComplete((reply, ex) -> completedNanos = nowNanos.getAsLong());
} }
@@ -357,7 +356,7 @@ public final class MessageService {
private final AtomicLong ticketSeq = new AtomicLong(); private final AtomicLong ticketSeq = new AtomicLong();
/** /**
* Minted once per {@code MessageService} instance and folded into every ticket id (see * Minted once per {@code MessageService} instance and folded into every ticket id (see
* {@link #sendAsync(String, String, Runnable, String)}). {@link #ticketSeq} alone restarts at * {@link #sendAsync(String, String, Runnable, Principal)}). {@link #ticketSeq} alone restarts at
* zero for every instance, so without this a ticket id can be reused across instances and * zero for every instance, so without this a ticket id can be reused across instances and
* resolve to an unrelated {@link Task} with no error; this nonce makes that impossible, because * resolve to an unrelated {@link Task} with no error; this nonce makes that impossible, because
* an id minted by one instance can never match the id space of another. * an id minted by one instance can never match the id space of another.
@@ -938,13 +937,13 @@ public final class MessageService {
/** /**
* Deliver {@code content} to {@code target} (a herdr {@code terminal_id}) and block until the * Deliver {@code content} to {@code target} (a herdr {@code terminal_id}) and block until the
* worker replies via {@link Rendezvous} or {@code timeoutMillis} elapses. {@code callerTerminal} * worker replies via {@link Rendezvous} or {@code timeoutMillis} elapses. {@code callerOwner}
* is the terminal of the caller making this call — {@code null} for the unnamed primary — and is * identifies the caller making this call and is recorded as the turn's owner. It is the only
* recorded as the turn's owner, the only caller {@link #answer(String, String, long, String)} will * caller {@link #answer(String, String, long, String)} will
* later accept an answer from if the worker pauses mid-turn to ask. * later accept an answer from if the worker pauses mid-turn to ask.
*/ */
public Reply send(String target, String content, long timeoutMillis, String callerTerminal) { public Reply send(String target, String content, long timeoutMillis, String callerOwner) {
return send(target, content, timeoutMillis, null, callerTerminal); return send(target, content, timeoutMillis, null, callerOwner);
} }
/** /**
@@ -959,13 +958,13 @@ public final class MessageService {
* acceptance means a concurrent sender that times out {@code BUSY} can never steal ownership it * acceptance means a concurrent sender that times out {@code BUSY} can never steal ownership it
* never earned. {@code null} disables the hook. * never earned. {@code null} disables the hook.
*/ */
public Reply send(String target, String content, long timeoutMillis, Runnable onAccepted, String callerTerminal) { public Reply send(String target, String content, long timeoutMillis, Runnable onAccepted, String callerOwner) {
return send(target, content, timeoutMillis, onAccepted, null, callerTerminal); return send(target, content, timeoutMillis, onAccepted, null, callerOwner);
} }
/** Run a send, optionally stopping an async task that teardown already failed before acceptance. */ /** Run a send, optionally stopping an async task that teardown already failed before acceptance. */
private Reply send(String target, String content, long timeoutMillis, Runnable onAccepted, Task task, private Reply send(String target, String content, long timeoutMillis, Runnable onAccepted, Task task,
String callerTerminal) { String callerOwner) {
long deadlineNanos = System.nanoTime() + timeoutMillis * 1_000_000L; long deadlineNanos = System.nanoTime() + timeoutMillis * 1_000_000L;
ReentrantLock lock = sessionLocks.computeIfAbsent(target, _ -> new ReentrantLock()); ReentrantLock lock = sessionLocks.computeIfAbsent(target, _ -> new ReentrantLock());
@@ -985,7 +984,7 @@ public final class MessageService {
// open race). Opening first also means a throwing onAccepted (fired before enqueue) or an // open race). Opening first also means a throwing onAccepted (fired before enqueue) or an
// enqueue failure is safely closed by the finally below: nothing is left queued, and the // enqueue failure is safely closed by the finally below: nothing is left queued, and the
// failed send leaves no stale waiter behind. // failed send leaves no stale waiter behind.
CompletableFuture<Rendezvous.Resolution> reply = rendezvous.open(target, Rendezvous.Owner.of(callerTerminal)); CompletableFuture<Rendezvous.Resolution> reply = rendezvous.open(target, Rendezvous.Owner.of(callerOwner));
// CB-640: this send now owns target's delivery, so any earlier stranded-reply or // CB-640: this send now owns target's delivery, so any earlier stranded-reply or
// still-queued fact no longer describes the live state — clear both rather than let // still-queued fact no longer describes the live state — clear both rather than let
// them outlive the send that supersedes them. // them outlive the send that supersedes them.
@@ -1165,20 +1164,20 @@ public final class MessageService {
* call, not a new status-gated delivery. The forward waiter is opened <em>before</em> the worker * call, not a new status-gated delivery. The forward waiter is opened <em>before</em> the worker
* is unblocked so a reply that lands the instant it resumes is not lost. * is unblocked so a reply that lands the instant it resumes is not lost.
* *
* <p>{@code callerTerminal} is the terminal of the caller making this call — {@code null} for * <p>{@code callerOwner} identifies the caller making this call. It is checked against the
* the unnamed primary. It is checked against the turn's recorded owner (the caller whose * turn's recorded owner (the caller whose
* accepted delegation opened it, see {@link #send(String, String, long, String)} and * accepted delegation opened it, see {@link #send(String, String, long, String)} and
* {@link #sendAsync(String, String, Runnable, String)}) before anything else runs: a mismatch, * {@link #sendAsync(String, String, Runnable, Principal)}) before anything else runs: a mismatch,
* including a turn with no owner on record at all, returns {@link Outcome#NOT_TURN_OWNER} * including a turn with no owner on record at all, returns {@link Outcome#NOT_TURN_OWNER}
* without touching the rendezvous, the session lock, or any async task bookkeeping. * without touching the rendezvous, the session lock, or any async task bookkeeping.
*/ */
public Reply answer(String turnId, String content, long timeoutMillis, String callerTerminal) { public Reply answer(String turnId, String content, long timeoutMillis, String callerOwner) {
String workerSession = rendezvous.askSession(turnId); String workerSession = rendezvous.askSession(turnId);
if (workerSession == null) { if (workerSession == null) {
return new Reply(Outcome.STALE_TURN, null); // the ask lapsed (timed out or already answered) return new Reply(Outcome.STALE_TURN, null); // the ask lapsed (timed out or already answered)
} }
Rendezvous.Owner owner = rendezvous.askOwner(turnId); Rendezvous.Owner owner = rendezvous.askOwner(turnId);
if (!Rendezvous.Owner.permits(owner, callerTerminal)) { if (!Rendezvous.Owner.permits(owner, callerOwner)) {
return new Reply(Outcome.NOT_TURN_OWNER, null); return new Reply(Outcome.NOT_TURN_OWNER, null);
} }
long deadlineNanos = System.nanoTime() + timeoutMillis * 1_000_000L; long deadlineNanos = System.nanoTime() + timeoutMillis * 1_000_000L;
@@ -1321,16 +1320,16 @@ public final class MessageService {
} }
/** /**
* As {@link #sendAsync(String, String, Runnable)}, recording {@code creatorTerminal} as this * As {@link #sendAsync(String, String, Runnable)}, recording {@code creator}'s owner key as this
* ticket's owner. {@link #poll(String, String)} refuses a later caller whose own terminal * ticket's owner. The key is derived here from the resolved principal so callers cannot pass a
* differs from this one; {@code null} records no owner (a caller with no terminal — the * terminal address where an owner identity is required.
* unnamed primary — is always allowed to poll the result regardless).
* *
* @return the ticket to poll for the eventual result * @return the ticket to poll for the eventual result
*/ */
public String sendAsync(String target, String content, Runnable onAccepted, String creatorTerminal) { public String sendAsync(String target, String content, Runnable onAccepted, Principal creator) {
String ticket = "task-" + ticketBootNonce + "-" + ticketSeq.incrementAndGet(); String ticket = "task-" + ticketBootNonce + "-" + ticketSeq.incrementAndGet();
Task task = new Task(ticket, target, nowNanos, creatorTerminal); String creatorOwner = creator == null ? null : creator.ownerKey();
Task task = new Task(ticket, target, nowNanos, creatorOwner);
tasks.put(ticket, task); tasks.put(ticket, task);
if (pushLoop != null) { if (pushLoop != null) {
// CB-588: task.future only ever completes on a terminal phase (DONE or a failure) — a // CB-588: task.future only ever completes on a terminal phase (DONE or a failure) — a
@@ -1361,7 +1360,7 @@ public final class MessageService {
} }
asyncExecutor.submit(() -> { asyncExecutor.submit(() -> {
try { try {
Reply result = send(target, content, ASYNC_TIMEOUT_MS, onAccepted, task, creatorTerminal); Reply result = send(target, content, ASYNC_TIMEOUT_MS, onAccepted, task, creatorOwner);
if (result.outcome() == Outcome.QUESTION) { if (result.outcome() == Outcome.QUESTION) {
// Keep the accepted owner until answer() finishes it. markAsyncQuestion may run // Keep the accepted owner until answer() finishes it. markAsyncQuestion may run
// just after resolveQuestion wakes this thread. // just after resolveQuestion wakes this thread.
@@ -1393,28 +1392,30 @@ public final class MessageService {
} }
/** /**
* As {@link #poll(String, String)}, with no caller terminal — the ticket's ownership is never * As {@link #poll(String, String)}, but bypasses the ownership check entirely via
* checked, so this overload must only be used where the caller's identity is otherwise * {@link #INTERNAL_NO_OWNER_CHECK}. No production code calls this overload — it exists for
* irrelevant (a test, or a surface that does not resolve a caller terminal at all). * tests that only need the ticket's state and have no caller identity to pass.
*/ */
public TaskView poll(String ticket) { public TaskView poll(String ticket) {
return poll(ticket, null); return poll(ticket, INTERNAL_NO_OWNER_CHECK);
} }
/** /**
* Snapshot the state of an async delegation. Returns {@code null} for an unknown/expired ticket. * Snapshot the state of an async delegation. Returns {@code null} for an unknown/expired ticket.
* Refuses a {@code callerTerminal} that differs from the terminal that created the ticket (see * Refuses a {@code callerOwner} that differs from the owner that created the ticket (see
* {@link #sendAsync(String, String, Runnable, String)}) with a {@link Phase#FAILED} view that * {@link #sendAsync(String, String, Runnable, Principal)}) with a {@link Phase#FAILED} view that
* carries no reply text — a caller with no terminal (the unnamed primary) is never refused. * carries no reply text. The unnamed primary's owner key is {@code null}, matched the same way
* Otherwise returns a {@link Phase#PENDING} view (with the live worker status as detail), a * as any other key — it reads a ticket another unnamed primary created, and is refused on a
* {@link Phase#DONE} view carrying the reply, or a {@link Phase#FAILED} view with the reason. * ticket a named caller created. Otherwise returns a {@link Phase#PENDING} view (with the live
* worker status as detail), a {@link Phase#DONE} view carrying the reply, or a
* {@link Phase#FAILED} view with the reason.
*/ */
public TaskView poll(String ticket, String callerTerminal) { public TaskView poll(String ticket, String callerOwner) {
Task task = tasks.get(ticket); Task task = tasks.get(ticket);
if (task == null) { if (task == null) {
return null; return null;
} }
if (!ownsTicket(task, callerTerminal)) { if (!ownsTicket(task, callerOwner)) {
return new TaskView(ticket, Phase.FAILED, null, null, return new TaskView(ticket, Phase.FAILED, null, null,
"forbidden: this ticket was created by a different session", null); "forbidden: this ticket was created by a different session", null);
} }
@@ -1454,14 +1455,27 @@ public final class MessageService {
} }
/** /**
* Whether {@code callerTerminal} may read {@code task}'s state. A caller with no terminal * Marker passed as {@code callerOwner} to bypass the ownership check entirely. No
* always may — that is the unnamed primary, resolved by token or loopback trust, which never * {@link Principal#ownerKey()} ever produces this value — every real key is either
* carries a herdr pane and must keep reading every ticket. Otherwise the caller's terminal must * {@code null} (the unnamed primary) or prefixed with its role, such as {@code "worker:"} or
* equal the terminal recorded on the task; a task with no recorded terminal matches no * {@code "leader:"}. {@link #poll(String)} passes it; {@link #pendingAsk} has no matching
* terminal-bearing caller. * no-check overload, so this stays package-private for the test that drives the bypass
* directly.
*/ */
private static boolean ownsTicket(Task task, String callerTerminal) { static final String INTERNAL_NO_OWNER_CHECK = "internal:no-owner-check";
return callerTerminal == null || callerTerminal.equals(task.creatorTerminal);
/**
* Whether {@code callerOwner} may read {@code task}'s state. {@code callerOwner} is matched
* against the task's recorded owner key by equality, including a {@code null} match — the
* unnamed primary's owner key is {@code null}, so it owns a ticket another unnamed primary
* created and nothing else, the same rule every other role follows. The only caller that
* reads any ticket is {@link #INTERNAL_NO_OWNER_CHECK}. This differs from
* {@link Rendezvous.Owner#permits}: a missing rendezvous owner is not an authenticated
* unnamed primary, so that gate refuses every caller when no owner was recorded.
*/
private static boolean ownsTicket(Task task, String callerOwner) {
return INTERNAL_NO_OWNER_CHECK.equals(callerOwner)
|| Objects.equals(callerOwner, task.creatorOwner);
} }
/** /**
@@ -1787,19 +1801,80 @@ public final class MessageService {
* {@code fleet_status} uses this to show a pending question without the caller needing the * {@code fleet_status} uses this to show a pending question without the caller needing the
* ticket. {@code null} when the session has no open async question (including a session mid a * ticket. {@code null} when the session has no open async question (including a session mid a
* <em>blocking</em> {@code fleet_ask}, which has no {@link Task} to look up — see * <em>blocking</em> {@code fleet_ask}, which has no {@link Task} to look up — see
* {@link PendingAsk}), or when {@code callerTerminal} does not own the task the question * {@link PendingAsk}), or when {@code callerOwner} does not own the task the question
* belongs to (see {@link #ownsTicket(Task, String)}). * belongs to (see {@link #ownsTicket(Task, String)}).
*/ */
public PendingAsk pendingAsk(String workerSession, String callerTerminal) { public PendingAsk pendingAsk(String workerSession, String callerOwner) {
for (Task task : tasks.values()) { for (Task task : tasks.values()) {
Reply q = task.question; Reply q = task.question;
if (q != null && workerSession.equals(task.target) && ownsTicket(task, callerTerminal)) { if (q != null && workerSession.equals(task.target) && ownsTicket(task, callerOwner)) {
return new PendingAsk(task.ticket, q.text(), q.turnId()); return new PendingAsk(task.ticket, q.text(), q.turnId());
} }
} }
return null; return null;
} }
/**
* One ticket {@code callerOwner} created, still present in {@link #tasks}, surfaced by
* {@link #outstanding} so a lead can carry its id into a handover file. {@link #phase} is the
* same value {@link #poll} would report right now, terminal phases included: a {@code DONE} or
* {@code FAILED} ticket stays in {@link #tasks} — and so stays reported here — until
* {@link #pruneTerminalTickets} evicts it.
*/
public record OutstandingTicket(String ticket, Phase phase, String target) {
}
/**
* One worker session paused in {@code fleet_ask}, with the {@code turnId} that answers it,
* surfaced by {@link #outstanding} alongside {@link OutstandingTicket}.
*/
public record OutstandingAsk(String ticket, String turnId, String workerSession) {
}
/** The outstanding tickets and open asks a single call to {@link #outstanding} reports. */
public record Outstanding(List<OutstandingTicket> tickets, List<OutstandingAsk> asks) {
}
/**
* Every ticket {@code callerOwner} created that is still in {@link #tasks} — including a
* finished one nobody has polled yet, since {@link #pruneTerminalTickets} discards its reply
* on a timer and a lead that does not carry its id forward can no longer read it after losing
* its session's context — plus the subset of those whose worker is paused in
* {@code fleet_ask}. Filtered by the same ownership rule as {@link #poll}:
* {@link #ownsTicket(Task, String)}.
*/
public Outstanding outstanding(String callerOwner) {
List<OutstandingTicket> tickets = new ArrayList<>();
List<OutstandingAsk> asks = new ArrayList<>();
for (Task task : tasks.values()) {
if (!ownsTicket(task, callerOwner)) {
continue;
}
Reply question = task.question;
Phase phase;
if (task.future.isDone()) {
phase = terminalPhase(task.future);
} else if (question != null) {
phase = Phase.ASKING;
asks.add(new OutstandingAsk(task.ticket, question.turnId(), task.target));
} else {
phase = Phase.PENDING;
}
tickets.add(new OutstandingTicket(task.ticket, phase, task.target));
}
return new Outstanding(tickets, asks);
}
/** As {@link #poll}'s own terminal-result handling, reduced to just the {@link Phase}. */
private static Phase terminalPhase(CompletableFuture<Reply> future) {
try {
Reply r = future.getNow(null);
return r != null && r.completed() ? Phase.DONE : Phase.FAILED;
} catch (CompletionException | java.util.concurrent.CancellationException e) {
return Phase.FAILED;
}
}
/** Release the async executor. */ /** Release the async executor. */
public void close() { public void close() {
asyncExecutor.shutdown(); asyncExecutor.shutdown();
@@ -1,5 +1,6 @@
package dev.ltms.fleet.msg; package dev.ltms.fleet.msg;
import java.util.UUID;
import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicLong; import java.util.concurrent.atomic.AtomicLong;
@@ -72,23 +73,22 @@ public final class Rendezvous {
/** /**
* The caller whose accepted delegation opened a turn — the only caller allowed to answer it. * The caller whose accepted delegation opened a turn — the only caller allowed to answer it.
* A {@code null} terminal means the unnamed primary, an authenticated caller with no pane. * A {@code null} owner key means the unnamed primary.
*/ */
public record Owner(String terminal) { public record Owner(String ownerKey) {
public static final Owner UNNAMED_PRIMARY = new Owner(null); public static final Owner UNNAMED_PRIMARY = new Owner(null);
public static Owner of(String terminal) { public static Owner of(String ownerKey) {
return terminal == null ? UNNAMED_PRIMARY : new Owner(terminal); return ownerKey == null ? UNNAMED_PRIMARY : new Owner(ownerKey);
} }
/** /**
* Whether {@code callerTerminal} matches {@code owner}. A {@code null} owner means no * Whether {@code callerOwner} matches {@code owner}. A {@code null} owner means no owner was
* owner was ever recorded, and that state matches no caller, not even one whose own * recorded, so it matches no caller. {@link #UNNAMED_PRIMARY} records the unnamed primary
* terminal is {@code null} — "no record" and "recorded as the unnamed primary" are * with an owner object whose key is {@code null}.
* different states.
*/ */
public static boolean permits(Owner owner, String callerTerminal) { public static boolean permits(Owner owner, String callerOwner) {
return owner != null && java.util.Objects.equals(owner.terminal(), callerTerminal); return owner != null && java.util.Objects.equals(owner.ownerKey(), callerOwner);
} }
} }
@@ -101,6 +101,14 @@ public final class Rendezvous {
/** Reverse rendezvous (CB-205): worker questions awaiting the primary's answer, keyed by {@code turnId}. */ /** Reverse rendezvous (CB-205): worker questions awaiting the primary's answer, keyed by {@code turnId}. */
private final ConcurrentHashMap<String, AskWaiter> asks = new ConcurrentHashMap<>(); private final ConcurrentHashMap<String, AskWaiter> asks = new ConcurrentHashMap<>();
private final AtomicLong askSeq = new AtomicLong(); private final AtomicLong askSeq = new AtomicLong();
/**
* Minted once per {@code Rendezvous} instance and folded into every {@code turnId} (see
* {@link #openAsk(String)}). {@link #askSeq} alone restarts at zero for every instance, so
* without this a {@code turnId} minted by one instance could be minted again by another and
* resolve to an unrelated ask with no error; this nonce makes that impossible, because an id
* minted by one instance can never match the id space of another.
*/
private final String askBootNonce = UUID.randomUUID().toString().substring(0, 6);
/** Per-session index of the currently-open ask, so duplicate fleet_ask calls coalesce onto one turn. */ /** Per-session index of the currently-open ask, so duplicate fleet_ask calls coalesce onto one turn. */
private final ConcurrentHashMap<String, String> openAsksBySession = new ConcurrentHashMap<>(); private final ConcurrentHashMap<String, String> openAsksBySession = new ConcurrentHashMap<>();
@@ -187,7 +195,7 @@ public final class Rendezvous {
while (true) { while (true) {
AskWaiter[] minted = { null }; AskWaiter[] minted = { null };
String turnId = openAsksBySession.computeIfAbsent(session, _ -> { String turnId = openAsksBySession.computeIfAbsent(session, _ -> {
String newTurnId = session + "#" + askSeq.incrementAndGet(); String newTurnId = session + "#" + askBootNonce + "-" + askSeq.incrementAndGet();
CompletableFuture<String> answer = new CompletableFuture<>(); CompletableFuture<String> answer = new CompletableFuture<>();
AskWaiter waiter = new AskWaiter(session, answer, ownerOf(session)); AskWaiter waiter = new AskWaiter(session, answer, ownerOf(session));
asks.put(newTurnId, waiter); asks.put(newTurnId, waiter);
@@ -3,6 +3,7 @@ package dev.ltms.fleet.msg;
import dev.ltms.fleet.herdr.AgentControl; import dev.ltms.fleet.herdr.AgentControl;
import dev.ltms.fleet.herdr.AgentStatus; import dev.ltms.fleet.herdr.AgentStatus;
import dev.ltms.fleet.herdr.HerdrException; import dev.ltms.fleet.herdr.HerdrException;
import dev.ltms.fleet.herdr.PromptBox;
import dev.ltms.fleet.mcp.PrimaryRegistry; import dev.ltms.fleet.mcp.PrimaryRegistry;
import dev.ltms.fleet.metrics.FleetMetrics; import dev.ltms.fleet.metrics.FleetMetrics;
import dev.ltms.fleet.metrics.Metrics; import dev.ltms.fleet.metrics.Metrics;
@@ -50,6 +51,11 @@ import java.util.stream.Collectors;
* exhausting its cap does not stop nudges about the others (post-CB-590 regression fix; see * exhausting its cap does not stop nudges about the others (post-CB-590 regression fix; see
* {@link #decide}) — whichever the durable inbox / pending set doesn't already answer via * {@link #decide}) — whichever the durable inbox / pending set doesn't already answer via
* {@code STOP}. * {@code STOP}.
*
* <p>A lead that is injectable is nudged only when its prompt box is also empty
* ({@link PromptBox}): the delivery pastes and submits in one call, so a nudge into a box holding
* the operator's half-typed line would submit that line too. A nudge held for that reason waits for
* the next tick like any other, and the pending work is re-read then.
*/ */
public final class ReplyPushLoop { public final class ReplyPushLoop {
@@ -81,6 +87,7 @@ public final class ReplyPushLoop {
private final PrimaryRegistry primaryRegistry; private final PrimaryRegistry primaryRegistry;
private final AgentControl agents; private final AgentControl agents;
private final PromptBox promptBox;
private final ReplyInbox inbox; private final ReplyInbox inbox;
private final ScheduledExecutorService scheduler; private final ScheduledExecutorService scheduler;
private final int maxReminders; private final int maxReminders;
@@ -125,6 +132,7 @@ public final class ReplyPushLoop {
int maxReminders, long backoffMs, Metrics metrics) { int maxReminders, long backoffMs, Metrics metrics) {
this.primaryRegistry = primaryRegistry; this.primaryRegistry = primaryRegistry;
this.agents = agents; this.agents = agents;
this.promptBox = new PromptBox(agents);
this.inbox = inbox; this.inbox = inbox;
this.scheduler = scheduler; this.scheduler = scheduler;
this.maxReminders = maxReminders; this.maxReminders = maxReminders;
@@ -398,11 +406,15 @@ public final class ReplyPushLoop {
log.debug("push: status check failed for lead {}, will retry", lead, e); log.debug("push: status check failed for lead {}, will retry", lead, e);
return Action.WAIT_BUSY; return Action.WAIT_BUSY;
} }
if (status.injectable()) { if (!status.injectable()) {
return Action.INJECT; log.debug("push: lead {} is {} (not injectable), waiting", lead, status);
return Action.WAIT_BUSY;
} }
log.debug("push: lead {} is {} (not injectable), waiting", lead, status); if (!promptBox.clearToSubmit(lead)) {
return Action.WAIT_BUSY; log.debug("push: lead {} has unsubmitted text in its prompt box, waiting", lead);
return Action.WAIT_BUSY;
}
return Action.INJECT;
} }
/** /**
@@ -439,6 +451,15 @@ public final class ReplyPushLoop {
* — timeout, transport error, a decode error — is treated as still live and the binding is left * — timeout, transport error, a decode error — is treated as still live and the binding is left
* alone, because guessing wrong here is unrecoverable while guessing "live" merely costs one more * alone, because guessing wrong here is unrecoverable while guessing "live" merely costs one more
* retry on the next tick, which {@link #decide} already tolerates. * retry on the next tick, which {@link #decide} already tolerates.
*
* <p><strong>The fallback is probed too.</strong> {@code PrimaryRegistry.nudgeTargetFor} already
* resolves a named delegator to its current terminal before this method ever sees it, which
* keeps a rolled lead's per-target binding live. What that resolution cannot fix is a caller
* that was never recorded with a name at all — an unnamed primary, or a lead whose tab the
* scanner cannot currently see — where the fallback it returns is still the raw terminal last
* learned from call traffic. This method returns that fallback only after the same liveness
* check, and gives up for this tick (an empty result, exactly like "no lead known at all") rather
* than hand a caller a second stale address un-probed.
*/ */
private Optional<String> resolveLiveLead(String target) { private Optional<String> resolveLiveLead(String target) {
Optional<String> lead = primaryRegistry.nudgeTargetFor(target); Optional<String> lead = primaryRegistry.nudgeTargetFor(target);
@@ -448,7 +469,12 @@ public final class ReplyPushLoop {
log.debug("push: lead {} delegated to for {} is no longer live, forgetting the stale binding " log.debug("push: lead {} delegated to for {} is no longer live, forgetting the stale binding "
+ "and falling back", lead.get(), target); + "and falling back", lead.get(), target);
primaryRegistry.forgetDelegation(target); primaryRegistry.forgetDelegation(target);
return primaryRegistry.nudgeTargetFor(target); Optional<String> fallback = primaryRegistry.nudgeTargetFor(target);
if (fallback.isEmpty() || isLive(fallback.get())) {
return fallback;
}
log.debug("push: fallback lead {} for {} is also not live, skipping this tick", fallback.get(), target);
return Optional.empty();
} }
/** /**
@@ -1,6 +1,8 @@
package dev.ltms.fleet.peer; package dev.ltms.fleet.peer;
import java.util.Locale; import java.util.Locale;
import java.util.stream.Collectors;
import java.util.stream.Stream;
/** /**
* What a member is <em>for</em> — the contract it runs under. * What a member is <em>for</em> — the contract it runs under.
@@ -100,6 +102,11 @@ public enum MemberRole {
return null; return null;
} }
/** The wire name of every role, joined with {@code ", "} in declaration order. */
public static String wireNames() {
return Stream.of(values()).map(MemberRole::wireName).collect(Collectors.joining(", "));
}
/** /**
* Parse a config/wire spelling, case-insensitively. * Parse a config/wire spelling, case-insensitively.
* *
@@ -118,14 +125,7 @@ public enum MemberRole {
} }
} }
} }
StringBuilder valid = new StringBuilder();
for (MemberRole r : values()) {
if (!valid.isEmpty()) {
valid.append(", ");
}
valid.append(r.wireName());
}
throw new IllegalArgumentException( throw new IllegalArgumentException(
"unknown member role '" + s + "'; valid roles are: " + valid); "unknown member role '" + s + "'; valid roles are: " + wireNames());
} }
} }
@@ -13,6 +13,7 @@ import dev.ltms.fleet.mcp.FleetMcp;
import dev.ltms.fleet.herdr.Agent; import dev.ltms.fleet.herdr.Agent;
import dev.ltms.fleet.herdr.HerdrClient; import dev.ltms.fleet.herdr.HerdrClient;
import dev.ltms.fleet.herdr.HerdrException; import dev.ltms.fleet.herdr.HerdrException;
import dev.ltms.fleet.herdr.PaneLocator;
import dev.ltms.fleet.inject.MemberPresence; import dev.ltms.fleet.inject.MemberPresence;
import dev.ltms.fleet.member.MemberCredentialPolicyView; import dev.ltms.fleet.member.MemberCredentialPolicyView;
import dev.ltms.fleet.peer.PeerUnreachableException; import dev.ltms.fleet.peer.PeerUnreachableException;
@@ -281,6 +282,18 @@ public final class FleetApp {
return Authz.permits(caller, action, target, knownLeadOrCollaborator); return Authz.permits(caller, action, target, knownLeadOrCollaborator);
} }
/**
* As {@link #permitsFor(Principal, Authz.Action, String, Predicate)}, also threading the
* classifier an observer's {@code SEND} is checked against; pass {@link #auth}'s own
* {@code sendableObserverTarget()} to exercise the real production gate, as {@link #allow}
* does.
*/
static boolean permitsFor(Principal caller, Authz.Action action, String target,
Predicate<String> knownLeadOrCollaborator,
Predicate<String> knownObserverTarget) {
return Authz.permits(caller, action, target, knownLeadOrCollaborator, knownObserverTarget);
}
/** /**
* Gate a handler on the CB-505 authorization table. Returns {@code true} when the request may * Gate a handler on the CB-505 authorization table. Returns {@code true} when the request may
* proceed; otherwise writes the error response and returns {@code false}. * proceed; otherwise writes the error response and returns {@code false}.
@@ -294,7 +307,7 @@ public final class FleetApp {
return true; // legacy: authorization not enforced return true; // legacy: authorization not enforced
} }
Principal caller = ctx.attribute(CALLER); Principal caller = ctx.attribute(CALLER);
if (permitsFor(caller, action, target, auth.knownLeadOrCollaborator())) { if (permitsFor(caller, action, target, auth.knownLeadOrCollaborator(), auth.sendableObserverTarget())) {
if (action != Authz.Action.READ && action != Authz.Action.METRICS if (action != Authz.Action.READ && action != Authz.Action.METRICS
&& action != Authz.Action.TASK_READ) { && action != Authz.Action.TASK_READ) {
AuditLog.allowed(caller, action, target); // reads would drown the trail AuditLog.allowed(caller, action, target); // reads would drown the trail
@@ -436,14 +449,27 @@ public final class FleetApp {
} }
} }
/**
* Tab id → its herdr display label, or an empty map on a {@code workspace.list}/{@code
* tab.list} failure — a missing label must not cost the agent roster.
*/
private Map<String, String> tabLabelsOrEmpty() {
try {
return new PaneLocator(herdr, memberHerdr).tabLabelsByTabId();
} catch (HerdrException e) {
return Map.of();
}
}
/** Discovery: every agent herdr tracks, keyed by its Claude session UUID. */ /** Discovery: every agent herdr tracks, keyed by its Claude session UUID. */
private void agents(Context ctx) { private void agents(Context ctx) {
if (!allow(ctx, routeAction("GET /agents"), null)) { if (!allow(ctx, routeAction("GET /agents"), null)) {
return; return;
} }
final Map<String, String> tabLabels = tabLabelsOrEmpty();
try { try {
ctx.status(200).json(Map.of("agents", ctx.status(200).json(Map.of("agents",
workers.list().stream().map(Agent.class::cast).map(FleetApp::view).toList())); workers.list().stream().map(Agent.class::cast).map(a -> view(a, tabLabels)).toList()));
} catch (HerdrException e) { } catch (HerdrException e) {
// fleetd #297: workers.list() reaches herdr — a transport failure must land in the same // fleetd #297: workers.list() reaches herdr — a transport failure must land in the same
// {error, detail} envelope every other failure path here uses, not escape as a bare // {error, detail} envelope every other failure path here uses, not escape as a bare
@@ -664,7 +690,7 @@ public final class FleetApp {
return; return;
} }
Principal caller = ctx.attribute(CALLER); Principal caller = ctx.attribute(CALLER);
String callerTerminal = caller == null ? null : caller.terminal(); String callerOwner = caller == null ? null : caller.ownerKey();
JsonNode body; JsonNode body;
try { try {
body = mapper.readTree(ctx.body()); body = mapper.readTree(ctx.body());
@@ -686,23 +712,27 @@ public final class FleetApp {
ctx.status(400).json(Map.of("error", "bad_request", "detail", "content is required")); ctx.status(400).json(Map.of("error", "bad_request", "detail", "content is required"));
return; return;
} }
// An observer's SEND reaches a pane that cannot otherwise distinguish this from a human
// paste (see FleetMcp#attributeIfObserver, the same rule on the MCP entry path); every
// other caller's content passes through unchanged.
content = FleetMcp.attributeIfObserver(caller, content);
timeout = Math.clamp(timeout, 1, MAX_MESSAGE_TIMEOUT_MS); timeout = Math.clamp(timeout, 1, MAX_MESSAGE_TIMEOUT_MS);
// Answering a worker's fleet_ask (CB-205): always blocks, and derives the worker from turnId. // Answering a worker's fleet_ask (CB-205): always blocks, and derives the worker from turnId.
if (turnId != null && !turnId.isBlank()) { if (turnId != null && !turnId.isBlank()) {
writeReply(ctx, id, messages.answer(turnId, content, timeout, callerTerminal), timeout); writeReply(ctx, id, messages.answer(turnId, content, timeout, callerOwner), timeout);
return; return;
} }
if (!wait) { if (!wait) {
// Fire-and-poll (CB-107): return a ticket immediately; the caller polls GET /tasks/{ticket}. // Fire-and-poll (CB-107): return a ticket immediately; the caller polls GET /tasks/{ticket}.
String ticket = messages.sendAsync(id, content, null, callerTerminal); String ticket = messages.sendAsync(id, content, null, caller);
ctx.status(202).json(Map.of("sessionId", id, "ticket", ticket, "status", "accepted")); ctx.status(202).json(Map.of("sessionId", id, "ticket", ticket, "status", "accepted"));
return; return;
} }
try { try {
writeReply(ctx, id, messages.send(id, content, timeout, callerTerminal), timeout); writeReply(ctx, id, messages.send(id, content, timeout, callerOwner), timeout);
} catch (HerdrException e) { } catch (HerdrException e) {
herdrError(ctx, e); herdrError(ctx, e);
} }
@@ -883,10 +913,9 @@ public final class FleetApp {
body.put("ready", deliverable.test(id)); body.put("ready", deliverable.test(id));
// A worker paused mid-turn in an async fleet_ask is otherwise invisible to a status // A worker paused mid-turn in an async fleet_ask is otherwise invisible to a status
// poll — surface the open question and how to answer it, same as fleet_poll's // poll — surface the open question and how to answer it, same as fleet_poll's
// Phase.ASKING view, but only to the caller whose terminal created that delegation, or // Phase.ASKING view, but only to the caller whose owner key created that delegation.
// to a caller with no terminal at all (the unnamed primary).
Principal caller = ctx.attribute(CALLER); Principal caller = ctx.attribute(CALLER);
MessageService.PendingAsk ask = messages.pendingAsk(id, caller == null ? null : caller.terminal()); MessageService.PendingAsk ask = messages.pendingAsk(id, caller == null ? null : caller.ownerKey());
if (ask != null) { if (ask != null) {
body.put("question", ask.question()); body.put("question", ask.question());
body.put("turnId", ask.turnId()); body.put("turnId", ask.turnId());
@@ -904,7 +933,7 @@ public final class FleetApp {
return; return;
} }
Principal caller = ctx.attribute(CALLER); Principal caller = ctx.attribute(CALLER);
MessageService.TaskView v = messages.poll(ctx.pathParam("ticket"), caller == null ? null : caller.terminal()); MessageService.TaskView v = messages.poll(ctx.pathParam("ticket"), caller == null ? null : caller.ownerKey());
if (v == null) { if (v == null) {
ctx.status(404).json(Map.of("error", "unknown_ticket", "detail", "no such task (or it has expired)")); ctx.status(404).json(Map.of("error", "unknown_ticket", "detail", "no such task (or it has expired)"));
return; return;
@@ -936,13 +965,19 @@ public final class FleetApp {
} }
} }
/** Stable JSON projection of an agent (null-safe for the start-time shape). */ /**
private static Map<String, Object> view(Agent a) { * Stable JSON projection of an agent (null-safe for the start-time shape).
*
* @param tabLabels tab id → its herdr display label; a tab absent from this map, or carrying
* a {@code null} label itself, projects as a {@code null} "label"
*/
private static Map<String, Object> view(Agent a, Map<String, String> tabLabels) {
Map<String, Object> m = new LinkedHashMap<>(); Map<String, Object> m = new LinkedHashMap<>();
m.put("terminalId", a.terminalId()); m.put("terminalId", a.terminalId());
m.put("paneId", a.paneId()); m.put("paneId", a.paneId());
m.put("workspaceId", a.workspaceId()); m.put("workspaceId", a.workspaceId());
m.put("tabId", a.tabId()); m.put("tabId", a.tabId());
m.put("label", tabLabels.get(a.tabId()));
m.put("sessionId", a.sessionId()); m.put("sessionId", a.sessionId());
m.put("agentType", a.agentType()); m.put("agentType", a.agentType());
m.put("status", a.status().name().toLowerCase()); m.put("status", a.status().name().toLowerCase());
@@ -442,38 +442,6 @@ public final class GitWorktrees implements Worktrees {
ENVIRONMENT_CREDENTIAL_HELPER); ENVIRONMENT_CREDENTIAL_HELPER);
} }
/**
* {@link #configureEnvironmentCredentialHelper} only ever fires for an HTTPS origin — Git never
* consults a {@code credential.helper} for an SSH transport. This repo's own origin is
* {@code ssh://git@git.ltms.dev:2224/fleet/fleetd.git}, so a member sitting on that origin never
* reaches the helper and the repo-scoped {@code WORKER_GITEA_TOKEN} is simply not used.
*
* <p>An earlier version of this javadoc justified the rewrite by claiming a member <em>cannot</em>
* push once {@code memberCredentials.policy: allow-list} blocks {@code SSH_AUTH_SOCK}, because
* "there is no private key file on this host, only an ssh-agent socket". That premise is false
* (fleetd #184): {@code ssh -G} resolves a readable, passphrase-free {@code IdentityFile} outside
* {@code ~/.ssh}, and a member — same OS user — pushes over SSH with the socket blanked. The
* rewrite is still worth having, but for the reason below rather than that one: it routes the
* member through its own scoped token instead of the operator's ssh identity, which is what makes
* a member's pushes attributable and revocable.
*
* <p>The fix is a <em>worktree-scoped</em> URL rewrite: {@code url.<https-base>.insteadOf
* <ssh-base>}, set with {@code --worktree} so it lands only in
* {@code <worktree>/.git/worktrees/<name>/config.worktree} (enabled by
* {@code extensions.worktreeConfig}, already turned on above) and never touches the shared
* repo-level config the primary checkout also reads. {@code insteadOf} — not
* {@code pushInsteadOf} — because a member may also need to fetch or rebase, and both should go
* through the member's own token for the same reason.
*
* <p>The host (and, for the rewrite's SSH-side match, the port) come from parsing the origin
* itself — never a hardcoded forge host, which is exactly what #177 removed. An origin that is
* already {@code https://} is left alone; the credential helper already covers it. An origin
* that is neither {@code ssh://} nor {@code https://} — including the scp-like shorthand
* ({@code git@host:path}, no scheme) — is left untouched deliberately: that shorthand's
* {@code host:path} split is defined by the user's ssh_config aliases, not by URI syntax, so
* guessing at it risks rewriting to the wrong place. A repo provisioned from that form keeps
* today's (broken, if the policy blocks the agent) SSH-only behaviour rather than a wrong rewrite.
*/
/** /**
* Blank the user-info of a remote URL before it reaches a log. A remote URL is not obviously a * Blank the user-info of a remote URL before it reaches a log. A remote URL is not obviously a
* credential channel, which is exactly why one has leaked here three times ({@code git remote -v} * credential channel, which is exactly why one has leaked here three times ({@code git remote -v}
@@ -485,6 +453,30 @@ public final class GitWorktrees implements Worktrees {
return url == null ? null : url.replaceAll("://[^@/]*@", "://<redacted>@"); return url == null ? null : url.replaceAll("://[^@/]*@", "://<redacted>@");
} }
/**
* {@link #configureEnvironmentCredentialHelper} only fires for an HTTPS origin — Git never
* consults a {@code credential.helper} for an SSH transport. This repo's own origin is
* {@code ssh://git@git.ltms.dev:2224/fleet/fleetd.git}, so a member on that origin never
* reaches the helper, and the repo-scoped token goes unused without a separate rewrite.
*
* <p>This method routes the member through its own scoped token instead of the operator's ssh
* identity, which is what makes a member's pushes attributable and revocable.
*
* <p>The fix is a <em>worktree-scoped</em> URL rewrite: {@code url.<https-base>.insteadOf
* <ssh-base>}, set with {@code --worktree} so it lands only in
* {@code <worktree>/.git/worktrees/<name>/config.worktree} and never touches the shared
* repo-level config the primary checkout also reads. {@code insteadOf} — not
* {@code pushInsteadOf} — because a member may also need to fetch or rebase through its own
* token.
*
* <p>The host (and, for the rewrite's SSH-side match, the port) come from parsing the origin
* itself, never a hardcoded forge host. An origin already {@code https://} is left alone; the
* credential helper already covers it. An origin that is neither {@code ssh://} nor
* {@code https://} — including the scp-like shorthand ({@code git@host:path}, no scheme) — is
* left untouched: that shorthand's {@code host:path} split is defined by the user's ssh_config
* aliases, not by URI syntax, so guessing at it risks rewriting to the wrong place, and that
* origin keeps SSH-only push behaviour instead.
*/
private void configureHttpsUrlRewriteForSshOrigin(String repoRoot, String worktreePath) { private void configureHttpsUrlRewriteForSshOrigin(String repoRoot, String worktreePath) {
if (exitCode("git", "-C", repoRoot, "config", "--get", "remote.origin.url") != 0) { if (exitCode("git", "-C", repoRoot, "config", "--get", "remote.origin.url") != 0) {
return; return;
@@ -255,6 +255,10 @@ public final class SessionManager implements TurnListener {
handle.id(), handle.terminalId(), resolvedProfile, actualRole, cwd, ownerTerminal, now, now, 0, handle.id(), handle.terminalId(), resolvedProfile, actualRole, cwd, ownerTerminal, now, now, 0,
MemberSession.State.SPAWNING, null, null, handle.charterReceipt(), handle.agentSessionId()); MemberSession.State.SPAWNING, null, null, handle.charterReceipt(), handle.agentSessionId());
registry.put(handle.id(), session); registry.put(handle.id(), session);
// A presence contact that already arrived for this terminal found no registry
// entry to transition and gave up silently. Retry it now that one exists; remove
// this call and such a session stays in SPAWNING even though it is present.
reconcilePresence(handle.terminalId());
handles.put(handle.id(), handle); handles.put(handle.id(), handle);
log.debug("acquired session id={} terminal={} profile={} owner={}", log.debug("acquired session id={} terminal={} profile={} owner={}",
handle.id(), handle.terminalId(), session.profile(), session.ownerTerminal()); handle.id(), handle.terminalId(), session.profile(), session.ownerTerminal());
@@ -466,6 +470,12 @@ public final class SessionManager implements TurnListener {
MemberSession resolved = resolveAgentSessionId(removed, removedHandle); MemberSession resolved = resolveAgentSessionId(removed, removedHandle);
notifyReleased(new ReleaseDetail(resolved.terminalId(), resolved.worktree(), notifyReleased(new ReleaseDetail(resolved.terminalId(), resolved.worktree(),
resolved.branch(), snapshotRef, resolved.agentSessionId())); resolved.branch(), snapshotRef, resolved.agentSessionId()));
String terminal = removed.terminalId();
if (terminal != null && !terminal.isBlank()) {
// Without this, a terminal stays marked present after its pane is gone, so a
// later send to the same id would read as deliverable instead of refused.
presence.forget(terminal);
}
} }
} }
// CB-581: the pane must always stop, even if the dirty check above threw. A session removed // CB-581: the pane must always stop, even if the dirty check above threw. A session removed
@@ -809,6 +819,10 @@ public final class SessionManager implements TurnListener {
handle.charterReceipt(), handle.charterReceipt(),
handle.agentSessionId()); handle.agentSessionId());
registry.put(handle.id(), session); registry.put(handle.id(), session);
// A presence contact that already arrived for this terminal found no registry entry to
// transition and gave up silently. Retry it now that one exists; remove this call and
// such a session stays in SPAWNING even though it is present.
reconcilePresence(handle.terminalId());
handles.put(handle.id(), handle); handles.put(handle.id(), handle);
log.debug("acquired worktree session id={} terminal={} profile={} branch={} path={}", log.debug("acquired worktree session id={} terminal={} profile={} branch={} path={}",
handle.id(), handle.terminalId(), session.profile(), session.branch(), session.worktree()); handle.id(), handle.terminalId(), session.profile(), session.branch(), session.worktree());
@@ -983,6 +997,20 @@ public final class SessionManager implements TurnListener {
transitionByTerminal(terminalId, MemberSession.State.SPAWNING, MemberSession.State.READY); transitionByTerminal(terminalId, MemberSession.State.SPAWNING, MemberSession.State.READY);
} }
/**
* Completes a newly registered session's {@code SPAWNING -> READY} transition when {@code
* terminalId} was already marked present before this ran. A terminal never marked present is
* left in {@code SPAWNING}; it reaches {@code READY} normally through {@link #onReady} once
* its own contact arrives. Callers must run this only once the session's registry entry is
* already visible — {@link #onReady}'s transition matches against that entry, and reconciling
* before the entry exists finds nothing to transition.
*/
private void reconcilePresence(String terminalId) {
if (terminalId != null && !terminalId.isBlank() && presence.isPresent(terminalId)) {
onReady(terminalId);
}
}
/** /**
* Lifecycle hook: a message was delivered into the worker — it is now busy on a turn. * Lifecycle hook: a message was delivered into the worker — it is now busy on a turn.
* The turn count is bumped and the activity timestamp is refreshed. A {@code DONE} session * The turn count is bumped and the activity timestamp is refreshed. A {@code DONE} session
@@ -125,6 +125,7 @@ class FleetdAssemblyTurnRegistrarBehaviouralTest {
primary: primary:
tab: "lead: primary" tab: "lead: primary"
profile: sonnet profile: sonnet
workspace: "ltms"
profiles: profiles:
sonnet: sonnet:
subscription: true subscription: true
@@ -91,6 +91,11 @@ class FleetdBackendErrorSinkTest {
return MAPPER.createObjectNode().set("agent", MAPPER.createObjectNode() return MAPPER.createObjectNode().set("agent", MAPPER.createObjectNode()
.put("terminal_id", "term_primary").put("agent_status", "idle")); .put("terminal_id", "term_primary").put("agent_status", "idle"));
} }
if ("agent.read".equals(method)) {
// The lead-nudge paths read the input box before pasting into it.
return MAPPER.createObjectNode().set("read",
MAPPER.createObjectNode().put("text", FakeHerdr.IDLE_PROMPT_CARET));
}
if ("agent.prompt".equals(method)) { if ("agent.prompt".equals(method)) {
prompts.add(params); prompts.add(params);
sendLatch.countDown(); sendLatch.countDown();
@@ -171,6 +171,7 @@ class FleetdLeadContextSourceWindowAssemblyTest {
%s: %s:
tab: "%s" tab: "%s"
profile: %s profile: %s
workspace: "ltms"
profiles: profiles:
%s: %s:
subscription: true subscription: true
@@ -6,6 +6,8 @@ import dev.ltms.fleet.guard.SubscriptionGuard;
import dev.ltms.fleet.herdr.AgentControl; import dev.ltms.fleet.herdr.AgentControl;
import dev.ltms.fleet.herdr.FakeHerdr; import dev.ltms.fleet.herdr.FakeHerdr;
import dev.ltms.fleet.herdr.HerdrClient; import dev.ltms.fleet.herdr.HerdrClient;
import dev.ltms.fleet.herdr.WorkspaceControl;
import dev.ltms.fleet.lead.LeadLauncher;
import dev.ltms.fleet.lead.LeadRollover; import dev.ltms.fleet.lead.LeadRollover;
import dev.ltms.fleet.msg.ReplyInbox; import dev.ltms.fleet.msg.ReplyInbox;
import io.javalin.Javalin; import io.javalin.Javalin;
@@ -34,7 +36,7 @@ import static org.junit.jupiter.api.Assertions.fail;
* fleetd #612 Unit A) with three methods: {@code unrelatedAnchorStillPresent} (a scaffold anchor, * fleetd #612 Unit A) with three methods: {@code unrelatedAnchorStillPresent} (a scaffold anchor,
* not an independent claim — needs no replacement of its own), {@code * not an independent claim — needs no replacement of its own), {@code
* mainStillCallsTheLeadRolloverFactory} (the call-site pin replaced by {@link * mainStillCallsTheLeadRolloverFactory} (the call-site pin replaced by {@link
* #assembledLeadRolloverRunsTheRealClearAndBootstrapSequence}), and {@code * #assembledLeadRolloverEndsTheOldPaneThroughTheRealHerdrRouter}), and {@code
* factoryGatesOnConfigPresence} (the absent-config claim replaced by {@link * factoryGatesOnConfigPresence} (the absent-config claim replaced by {@link
* #absentLeadRolloverConfigMeansNoRolloverIsBuilt} — a claim this ticket found was NOT actually * #absentLeadRolloverConfigMeansNoRolloverIsBuilt} — a claim this ticket found was NOT actually
* covered behaviourally anywhere else: {@code LeadRolloverTest}'s only related assertion is * covered behaviourally anywhere else: {@code LeadRolloverTest}'s only related assertion is
@@ -50,8 +52,8 @@ import static org.junit.jupiter.api.Assertions.fail;
* invisible to this test, even though the two are genuinely different daemons in production. This * invisible to this test, even though the two are genuinely different daemons in production. This
* version configures two distinct sockets and two distinct {@link FakeHerdr} instances (the same * version configures two distinct sockets and two distinct {@link FakeHerdr} instances (the same
* pattern {@code FleetdAssemblyConnectionIdentityTest}, fleetd #612 B2, already uses to separate * pattern {@code FleetdAssemblyConnectionIdentityTest}, fleetd #612 B2, already uses to separate
* lead from member) and asserts the roll's {@code /clear}/bootstrap sends land on the LEAD fake * lead from member) and asserts the roll's {@code pane.close} call lands on the LEAD fake and
* and never on the MEMBER one. * never on the MEMBER one.
*/ */
class FleetdLeadRolloverAssemblyTest { class FleetdLeadRolloverAssemblyTest {
@@ -170,6 +172,7 @@ class FleetdLeadRolloverAssemblyTest {
opus: opus:
tab: "lead: opus" tab: "lead: opus"
cwd: "%s" cwd: "%s"
workspace: "ltms"
leadRollover: leadRollover:
handoverPath: handover.md handoverPath: handover.md
requireOperatorConfirm: false requireOperatorConfirm: false
@@ -177,11 +180,48 @@ class FleetdLeadRolloverAssemblyTest {
return FleetConfig.load(f); return FleetConfig.load(f);
} }
@SuppressWarnings("unchecked") /**
* Unlike {@link #writeConfig}, this names a {@code profile:} for the lead and declares it
* under {@code profiles:}, so {@code LeadLauncher#relaunch} can actually start a fresh agent
* instead of refusing with "names no profile". {@code relaunchReadySeconds} is cut to 2s so
* the recognition wait (expected to time out — see the test) does not cost real test seconds.
*/
private static FleetConfig writeConfigWithRelaunchableLead(Path dir, Path leadCwd) throws Exception {
Path f = dir.resolve("fleetd.yaml");
Files.writeString(f, """
bind:
host: 127.0.0.1
port: 8765
herdrSocket: "%s"
memberHerdrSocket: "%s"
idleSleepGuard:
enabled: false
broker:
uri: "amqp://fake-test-broker/vh"
fleet:
leaders:
opus:
tab: "lead: opus"
cwd: "%s"
profile: opus
workspace: "ltms"
profiles:
opus:
subscription: true
argv: ["ccs", "opus"]
leadRollover:
handoverPath: handover.md
requireOperatorConfirm: false
relaunchReadySeconds: 2
""".formatted(LEAD_SOCKET, MEMBER_SOCKET, leadCwd.toString()));
return FleetConfig.load(f);
}
@Test @Test
@DisplayName("[BEHAVIOURAL] the real assembled LeadRollover runs the full open/confirm/continuation " @DisplayName("[BEHAVIOURAL] the real assembled LeadRollover runs the open/confirm/continuation "
+ "sequence — /clear, then bootstrapText — through the real herdr router") + "sequence through the real herdr router — ending the old pane, then giving up once it "
void assembledLeadRolloverRunsTheRealClearAndBootstrapSequence(@TempDir Path dir) throws Exception { + "never reports gone")
void assembledLeadRolloverEndsTheOldPaneThroughTheRealHerdrRouter(@TempDir Path dir) throws Exception {
Path leadCwd = dir.resolve("lead-workspace"); Path leadCwd = dir.resolve("lead-workspace");
Files.createDirectories(leadCwd); Files.createDirectories(leadCwd);
FleetConfig cfg = writeConfig(dir, leadCwd); FleetConfig cfg = writeConfig(dir, leadCwd);
@@ -204,6 +244,11 @@ class FleetdLeadRolloverAssemblyTest {
+ "Fleetd.leadRollover(...) call site — a mutation to `LeadRollover leadRollover = " + "Fleetd.leadRollover(...) call site — a mutation to `LeadRollover leadRollover = "
+ "null;` at that call site can never pass this"); + "null;` at that call site can never pass this");
// assembleAndStart's own boot work (the orphan-worker reap) makes a real call on the
// member daemon before the roll ever starts. Clear it here so the assertion below measures
// only what the roll itself does, not what daemon startup does.
member.calls.clear();
LeadRollover.PendingRollover pending = rollover.open("term_a", "fleetd #612 B3 test"); LeadRollover.PendingRollover pending = rollover.open("term_a", "fleetd #612 B3 test");
String expectedHandoverPath = leadCwd.resolve("handover.md").normalize().toString(); String expectedHandoverPath = leadCwd.resolve("handover.md").normalize().toString();
assertEquals(expectedHandoverPath, pending.handoverPath()); assertEquals(expectedHandoverPath, pending.handoverPath());
@@ -223,40 +268,109 @@ class FleetdLeadRolloverAssemblyTest {
// constructor), so this polls the real FleetMcp.leadRollover() instance's status(token) // constructor), so this polls the real FleetMcp.leadRollover() instance's status(token)
// until the real continuation finishes. // until the real continuation finishes.
LeadRollover.RollStatus status = pollUntilTerminal(rollover, pending.token()); LeadRollover.RollStatus status = pollUntilTerminal(rollover, pending.token());
assertEquals(LeadRollover.RollState.ROLLED, status.state(),
"the full happy path must complete: FakeHerdr's default agent status is 'idle', so "
+ "the turn-boundary wait settles immediately and the post-/clear wait "
+ "releases via its pickup-grace path — detail: " + status.detail());
// Prove the real herdr router actually sent BOTH messages, in order, to the real LEAD // FakeHerdr's pane.get is a fixed canned response that never reports a pane as gone, so the
// pane — this is the one thing a source-text pin on the call site could never show. // real router's death poll runs out its whole budget and the roll stops here — proving the
// real teardown call landed on the real LEAD pane without ever reaching a relaunch or a send.
assertEquals(LeadRollover.RollState.OLD_PANE_NEVER_DIED, status.state(),
"the old pane never reports gone against this fake, so the roll must stop with "
+ "OLD_PANE_NEVER_DIED rather than ever relaunching or sending anything — "
+ "detail: " + status.detail());
// Prove the real herdr router actually closed the real LEAD pane — this is the one thing a
// source-text pin on the call site could never show.
boolean closedOldPane = lead.calls.stream()
.anyMatch(c -> c.method().equals("pane.close")
&& c.params() instanceof Map<?, ?> m && "w2:p7".equals(m.get("pane_id")));
assertTrue(closedOldPane, "endOldSession must close the real old pane (w2:p7) through the "
+ "real LEAD herdr client, got calls: " + lead.calls);
// No agent.prompt is ever sent on this path: the roll stops at the pane-death wait, strictly
// before the relaunch and the final send step.
List<FakeHerdr.Call> prompts = lead.calls.stream() List<FakeHerdr.Call> prompts = lead.calls.stream()
.filter(c -> c.method().equals("agent.prompt")) .filter(c -> c.method().equals("agent.prompt"))
.toList(); .toList();
assertTrue(prompts.size() >= 2, "expected at least a /clear send and a bootstrapText send " assertTrue(prompts.isEmpty(), "a roll that stops at OLD_PANE_NEVER_DIED must never reach the "
+ "on the LEAD daemon, got " + prompts.size() + " agent.prompt calls: " + prompts); + "send step, got agent.prompt call(s) on the LEAD daemon: " + prompts);
assertEquals("/clear", ((Map<String, Object>) prompts.get(0).params()).get("text"),
"the first send must be the literal /clear housekeeping command");
Object secondText = ((Map<String, Object>) prompts.get(1).params()).get("text");
assertTrue(secondText instanceof String && ((String) secondText).contains(expectedHandoverPath),
"the second send must be the default bootstrapText naming the resolved handover "
+ "path, got: " + secondText);
// fleetd #612 B3 correction: prove the roll never touches the MEMBER daemon. A mutation // fleetd #612 B3 correction: prove the roll never touches the MEMBER daemon. A mutation
// swapping router.leadAgents() for router.memberAgents() at the real call site would move // swapping router.leadAgents() for router.memberAgents() at the real call site would move
// both sends above onto `member` instead, which this assertion catches — the thing the // the pane.close call above onto `member` instead, which this assertion catches — the thing
// single-fake version of this test could never see, because both wrapped the same client. // the single-fake version of this test could never see, because both wrapped the same client.
assertTrue(member.calls.isEmpty(), "the roll must be wired to the LEAD daemon only — got "
+ member.calls.size() + " call(s) recorded on the MEMBER daemon since the roll began: "
+ member.calls);
}
/**
* Exercises the relaunch site {@link #assembledLeadRolloverEndsTheOldPaneThroughTheRealHerdrRouter}
* never reaches: with the old pane confirmed gone, the roll relaunches a fresh lead, and
* {@code bootstrapText} must reach it even though recognition times out (FakeHerdr's
* {@code tab.list} is a fixed canned response that never reflects the relaunch's own
* {@code tab.rename}, so the fresh terminal is never recognised as a live lead). Same
* dual-socket shape as the sibling test: two distinct {@link FakeHerdr} instances, so a
* {@code bootstrapText} send wired to the wrong daemon is visible.
*/
@Test
@DisplayName("[BEHAVIOURAL] bootstrapText reaches the fresh LEAD terminal even when recognition "
+ "times out, and the MEMBER daemon never sees it")
void bootstrapTextReachesTheFreshLeadTerminalEvenWhenRecognitionTimesOut(@TempDir Path dir) throws Exception {
Path leadCwd = dir.resolve("lead-workspace");
Files.createDirectories(leadCwd);
FleetConfig cfg = writeConfigWithRelaunchableLead(dir, leadCwd);
ConfigRef config = new ConfigRef(dir.resolve("fleetd.yaml"), cfg);
SubscriptionGuard guard = new SubscriptionGuard(cfg.guard().hostSet());
RecordingResourcePorts ports = new RecordingResourcePorts();
FakeHerdr lead = new FakeHerdr();
lead.withTab("w2", "w2:t7", "lead: opus");
// Lets the old pane (w2:p7) report gone once pane.close actually reaches it, so the roll
// proceeds to relaunch instead of stopping at OLD_PANE_NEVER_DIED.
lead.paneGoneAfterClose("w2:p7");
FakeHerdr member = new FakeHerdr();
ports.herdrsBySocket.put(LEAD_SOCKET, lead);
ports.herdrsBySocket.put(MEMBER_SOCKET, member);
FleetdRuntime runtime = FleetdAssembly.assembleAndStart(new AssemblyInputs(cfg, config, guard), ports);
LeadRollover rollover = runtime.mcp().leadRollover();
assertNotNull(rollover, "leadRollover: is present in this test's config, so a real "
+ "LeadRollover must have been built");
LeadRollover.PendingRollover pending = rollover.open("term_a", "bootstrapText relaunch test");
Thread.sleep(50);
Files.writeString(Path.of(pending.handoverPath()), "handover content for bootstrapText test");
LeadRollover.RollDecision decision = rollover.confirm("term_a", pending.token(), true);
assertTrue(decision.accepted(), "confirm() must approve — got: " + decision);
LeadRollover.RollStatus status = pollUntilTerminal(rollover, pending.token());
assertEquals(LeadRollover.RollState.RELAUNCH_NOT_RECOGNISED, status.state(),
"the fresh terminal is never recognised against this fake's static tab.list, so the "
+ "roll must reach RELAUNCH_NOT_RECOGNISED — not an earlier failure state and "
+ "not ROLLED — detail: " + status.detail());
@SuppressWarnings("unchecked")
List<FakeHerdr.Call> leadPrompts = lead.calls.stream()
.filter(c -> c.method().equals("agent.prompt"))
.toList();
assertEquals(1, leadPrompts.size(), "exactly one bootstrapText send is expected, on the LEAD "
+ "daemon, once recognition gives up — got: " + leadPrompts);
Object text = ((Map<String, Object>) leadPrompts.get(0).params()).get("text");
assertTrue(text instanceof String && ((String) text).contains("handover.md"),
"the send must be bootstrapText naming the resolved handover path, got: " + text);
// Scoped to agent.prompt specifically, not every MEMBER call: the orphan-worker reap also
// talks to the MEMBER daemon once, unconditionally, at daemon boot — unrelated to this roll.
List<FakeHerdr.Call> memberPrompts = member.calls.stream() List<FakeHerdr.Call> memberPrompts = member.calls.stream()
.filter(c -> c.method().equals("agent.prompt")) .filter(c -> c.method().equals("agent.prompt"))
.toList(); .toList();
assertTrue(memberPrompts.isEmpty(), "the roll must be wired to the LEAD daemon only — got " assertTrue(memberPrompts.isEmpty(), "bootstrapText must never be sent to the MEMBER daemon, "
+ memberPrompts.size() + " agent.prompt call(s) on the MEMBER daemon instead: " + "got: " + memberPrompts);
+ memberPrompts);
} }
private static LeadRollover.RollStatus pollUntilTerminal(LeadRollover rollover, String token) private static LeadRollover.RollStatus pollUntilTerminal(LeadRollover rollover, String token)
throws InterruptedException { throws InterruptedException {
long deadline = System.nanoTime() + java.util.concurrent.TimeUnit.SECONDS.toNanos(10); long deadline = System.nanoTime() + java.util.concurrent.TimeUnit.SECONDS.toNanos(15);
while (System.nanoTime() < deadline) { while (System.nanoTime() < deadline) {
LeadRollover.RollStatus status = rollover.status(token); LeadRollover.RollStatus status = rollover.status(token);
if (status.state() != LeadRollover.RollState.PENDING if (status.state() != LeadRollover.RollState.PENDING
@@ -283,8 +397,10 @@ class FleetdLeadRolloverAssemblyTest {
"""); """);
ConfigRef config = new ConfigRef(yaml, FleetConfig.load(yaml)); ConfigRef config = new ConfigRef(yaml, FleetConfig.load(yaml));
AgentControl agents = new AgentControl(new FakeHerdr()); AgentControl agents = new AgentControl(new FakeHerdr());
WorkspaceControl spaces = new WorkspaceControl(new FakeHerdr());
LeadLauncher launcher = new LeadLauncher(agents, spaces, config.get());
LeadRollover rollover = Fleetd.leadRollover(config.get(), agents, config, Map::of); LeadRollover rollover = Fleetd.leadRollover(config.get(), agents, spaces, launcher, config, Map::of);
assertNull(rollover, "leadRollover: is absent from this config, so the factory's opt-in " assertNull(rollover, "leadRollover: is absent from this config, so the factory's opt-in "
+ "gate (`if (cfg.leadRollover() == null) return null;`) must fire and no " + "gate (`if (cfg.leadRollover() == null) return null;`) must fire and no "
@@ -4,6 +4,8 @@ import dev.ltms.fleet.config.ConfigRef;
import dev.ltms.fleet.config.FleetConfig; import dev.ltms.fleet.config.FleetConfig;
import dev.ltms.fleet.herdr.AgentControl; import dev.ltms.fleet.herdr.AgentControl;
import dev.ltms.fleet.herdr.FakeHerdr; import dev.ltms.fleet.herdr.FakeHerdr;
import dev.ltms.fleet.herdr.WorkspaceControl;
import dev.ltms.fleet.lead.LeadLauncher;
import dev.ltms.fleet.lead.LeadRollover; import dev.ltms.fleet.lead.LeadRollover;
import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
@@ -54,6 +56,11 @@ class FleetdLeadRolloverWorkspaceLookupTest {
return new AgentControl(new FakeHerdr()); return new AgentControl(new FakeHerdr());
} }
/** None of this class's tests reach the deferred continuation, so a plain fake is enough. */
private static LeadLauncher fakeLauncher(FleetConfig cfg) {
return new LeadLauncher(fakeAgents(), new WorkspaceControl(new FakeHerdr()), cfg);
}
@Test @Test
@DisplayName("[BEHAVIOURAL] Fleetd.leadRollover(...) resolves a relative handoverPath against " @DisplayName("[BEHAVIOURAL] Fleetd.leadRollover(...) resolves a relative handoverPath against "
+ "the CALLING lead's configured cwd, not the daemon's own working directory") + "the CALLING lead's configured cwd, not the daemon's own working directory")
@@ -74,7 +81,8 @@ class FleetdLeadRolloverWorkspaceLookupTest {
""".formatted(leadCwd.toString())); """.formatted(leadCwd.toString()));
ConfigRef config = new ConfigRef(yaml, FleetConfig.load(yaml)); ConfigRef config = new ConfigRef(yaml, FleetConfig.load(yaml));
LeadRollover rollover = Fleetd.leadRollover(config.get(), fakeAgents(), config, LeadRollover rollover = Fleetd.leadRollover(config.get(), fakeAgents(),
new WorkspaceControl(new FakeHerdr()), fakeLauncher(config.get()), config,
() -> Map.of("term_opus", "opus")); () -> Map.of("term_opus", "opus"));
assertNotNull(rollover, "leadRollover: is present in the loaded config, so the factory " assertNotNull(rollover, "leadRollover: is present in the loaded config, so the factory "
+ "must construct an object"); + "must construct an object");
@@ -105,7 +113,8 @@ class FleetdLeadRolloverWorkspaceLookupTest {
// No lead has been discovered yet — exactly the real shape of a lead the live tab scan // No lead has been discovered yet — exactly the real shape of a lead the live tab scan
// has not yet scanned, or one with no fleet.leaders entry at all. // has not yet scanned, or one with no fleet.leaders entry at all.
LeadRollover rollover = Fleetd.leadRollover(config.get(), fakeAgents(), config, Map::of); LeadRollover rollover = Fleetd.leadRollover(config.get(), fakeAgents(),
new WorkspaceControl(new FakeHerdr()), fakeLauncher(config.get()), config, Map::of);
assertNotNull(rollover); assertNotNull(rollover);
LeadRollover.PendingRollover pending = rollover.open("term_unknown", "test"); LeadRollover.PendingRollover pending = rollover.open("term_unknown", "test");
@@ -144,7 +153,8 @@ class FleetdLeadRolloverWorkspaceLookupTest {
// below — exactly the natural mistake to make, since leads are discovered by a live tab // below — exactly the natural mistake to make, since leads are discovered by a live tab
// scan that runs AFTER this factory is constructed at startup. // scan that runs AFTER this factory is constructed at startup.
Map<String, String> liveLeadTerminals = new HashMap<>(); Map<String, String> liveLeadTerminals = new HashMap<>();
LeadRollover rollover = Fleetd.leadRollover(config.get(), fakeAgents(), config, LeadRollover rollover = Fleetd.leadRollover(config.get(), fakeAgents(),
new WorkspaceControl(new FakeHerdr()), fakeLauncher(config.get()), config,
() -> liveLeadTerminals); () -> liveLeadTerminals);
assertNotNull(rollover); assertNotNull(rollover);
@@ -143,6 +143,7 @@ class FleetdLeadSeatAssemblyTest {
opus: opus:
tab: "lead: opus" tab: "lead: opus"
profile: sonnet profile: sonnet
workspace: "ltms"
profiles: profiles:
sonnet: sonnet:
subscription: true subscription: true
@@ -1,96 +1,174 @@
package dev.ltms.fleet; package dev.ltms.fleet;
import com.tngtech.archunit.base.DescribedPredicate; import com.tngtech.archunit.core.domain.Dependency;
import com.tngtech.archunit.core.domain.JavaClass; import com.tngtech.archunit.core.domain.JavaClass;
import com.tngtech.archunit.core.domain.JavaClass.Predicates; import com.tngtech.archunit.core.domain.JavaClasses;
import com.tngtech.archunit.core.importer.ClassFileImporter; import com.tngtech.archunit.core.importer.ClassFileImporter;
import com.tngtech.archunit.core.importer.ImportOption; import com.tngtech.archunit.core.importer.ImportOption;
import com.tngtech.archunit.library.dependencies.SliceRule; import com.tngtech.archunit.library.dependencies.SliceRule;
import com.tngtech.archunit.library.dependencies.SlicesRuleDefinition; import com.tngtech.archunit.library.dependencies.SlicesRuleDefinition;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;
import java.util.TreeSet;
import static org.junit.jupiter.api.Assertions.fail;
/** /**
* fleetd #131 (CB-627): enforce package boundaries with an ArchUnit test instead of a * Enforces package boundaries between the top-level {@code dev.ltms.fleet.*} packages.
* Maven module split.
* *
* <p>This test fails the build the moment a NEW cycle appears between the top-level * <p>{@link #BASELINE_EDGES} names the exact {@code origin class -> target class}
* {@code dev.ltms.fleet.*} packages. Today's cycles are recorded below as explicit, * dependencies allowed to cross a top-level package boundary. Any dependency between two
* narrow exceptions: each one ignores dependencies between exactly the two named * top-level packages that is not in that set fails this test, including a brand new
* packages, in both directions, and nothing else. A cycle through any other pair of * dependency between a pair of packages that already has other baselined edges. A baseline
* packages -- or a brand new pair -- still fails this test. * entry whose dependency no longer exists in the code also fails this test, so the baseline
* always names exactly today's exceptions and nothing more.
* *
* <p><b>Main code only.</b> The import excludes test classes * <p><b>Main code only.</b> The import excludes test classes
* ({@link ImportOption.Predefined#DO_NOT_INCLUDE_TESTS}). Test code legitimately wires * ({@link ImportOption.Predefined#DO_NOT_INCLUDE_TESTS}). Scanning off the classpath via
* across many packages for setup and mocking; that is not part of the shipped * {@code importPackages(...)} keeps this test correct regardless of the working directory
* architecture this rule protects. Verified: importing test classes too pulls in a much * the build is invoked from.
* larger, noisier cycle set -- {@code herdr}, {@code member}, {@code peer}, {@code
* config}, {@code guard} and {@code placement} all show up in cycles that disappear the
* moment test classes are excluded. Scanning off the classpath via {@code
* importPackages(...)} (not a hardcoded {@code target/classes} path) also keeps this
* test correct regardless of the working directory the build is invoked from.
*
* <p><b>No package moves here</b> -- ticket #131 is explicit that removing a cycle is
* its own, later PR. See the comment on each exception below for which ticket step
* removes it.
*/ */
class PackageCyclesTest { class PackageCyclesTest {
/**
* Exact {@code "origin -> target"} class dependencies allowed to cross a top-level
* package boundary. Each entry is one directed edge between two specific classes; a
* two-way relationship between a pair of packages is listed as two separate entries,
* one per direction.
*/
private static final Set<String> BASELINE_EDGES = Set.of(
"dev.ltms.fleet.auth.CallerResolver -> dev.ltms.fleet.mcp.ConnectionIdentity",
"dev.ltms.fleet.auth.CallerResolver -> dev.ltms.fleet.mcp.ConnectionIdentity$Caller",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.auth.AuditLog",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.auth.Authz",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.auth.Authz$Action",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.auth.CallerResolver",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.auth.Principal",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.auth.Role",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.LeadChannel",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.LeadChannel$MailboxState",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.LeadMessage",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.MessageService",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.MessageService$AskOutcome",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.MessageService$AskResult",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.MessageService$Outcome",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.MessageService$Outstanding",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.MessageService$PendingAsk",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.MessageService$Phase",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.MessageService$Reply",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.MessageService$ReplyOutcome",
"dev.ltms.fleet.mcp.FleetMcp -> dev.ltms.fleet.msg.MessageService$TaskView",
"dev.ltms.fleet.mcp.FleetMcp$1 -> dev.ltms.fleet.msg.MessageService$AskOutcome",
"dev.ltms.fleet.mcp.FleetMcp$1 -> dev.ltms.fleet.msg.MessageService$Outcome",
"dev.ltms.fleet.mcp.FleetMcp$1 -> dev.ltms.fleet.msg.MessageService$Phase",
"dev.ltms.fleet.mcp.FleetMcp$CoordinationSource -> dev.ltms.fleet.msg.LeadChannel",
"dev.ltms.fleet.msg.LeadHeartbeatLoop -> dev.ltms.fleet.mcp.PrimaryRegistry",
"dev.ltms.fleet.msg.ReplyPushLoop -> dev.ltms.fleet.mcp.PrimaryRegistry",
"dev.ltms.fleet.inject.CompletionResolver -> dev.ltms.fleet.msg.Rendezvous",
"dev.ltms.fleet.inject.CompletionResolver -> dev.ltms.fleet.msg.Rendezvous$Resolution",
"dev.ltms.fleet.inject.CompletionResolver -> dev.ltms.fleet.msg.TurnToken",
"dev.ltms.fleet.inject.CompletionResolver$InFlight -> dev.ltms.fleet.msg.Rendezvous$Resolution",
"dev.ltms.fleet.inject.Injector -> dev.ltms.fleet.msg.TurnToken",
"dev.ltms.fleet.inject.Injector$Pending -> dev.ltms.fleet.msg.TurnToken",
"dev.ltms.fleet.inject.TurnListener -> dev.ltms.fleet.msg.TurnToken",
"dev.ltms.fleet.inject.TurnRegistrar -> dev.ltms.fleet.msg.TurnToken",
"dev.ltms.fleet.msg.MessageService -> dev.ltms.fleet.inject.Injector",
"dev.ltms.fleet.msg.MessageService -> dev.ltms.fleet.inject.Injector$Cancellation",
"dev.ltms.fleet.msg.MessageService -> dev.ltms.fleet.inject.Injector$Delivery",
"dev.ltms.fleet.metrics.FleetMetrics -> dev.ltms.fleet.msg.ReplyInbox",
"dev.ltms.fleet.msg.LeadHeartbeatLoop -> dev.ltms.fleet.metrics.Metrics",
"dev.ltms.fleet.msg.MessageService -> dev.ltms.fleet.metrics.Metrics",
"dev.ltms.fleet.msg.ReplyPushLoop -> dev.ltms.fleet.metrics.Metrics",
"dev.ltms.fleet.msg.LeadHeartbeatLoop -> dev.ltms.fleet.session.MemberSession",
"dev.ltms.fleet.msg.LeadHeartbeatLoop -> dev.ltms.fleet.session.MemberSession$State",
"dev.ltms.fleet.session.SessionManager -> dev.ltms.fleet.msg.TurnToken"
);
private static final String ROOT_PACKAGE = "dev.ltms.fleet.";
@Test @Test
void packagesAreFreeOfCycles() { void packagesAreFreeOfCycles() {
var classes = new ClassFileImporter() JavaClasses classes = new ClassFileImporter()
.withImportOption(ImportOption.Predefined.DO_NOT_INCLUDE_TESTS) .withImportOption(ImportOption.Predefined.DO_NOT_INCLUDE_TESTS)
.importPackages("dev.ltms.fleet"); .importPackages("dev.ltms.fleet");
checkBaselineMatchesTodaysEdges(classes);
SliceRule rule = SlicesRuleDefinition.slices() SliceRule rule = SlicesRuleDefinition.slices()
.matching("dev.ltms.fleet.(*)..") .matching("dev.ltms.fleet.(*)..")
.should().beFreeOfCycles(); .should().beFreeOfCycles();
for (String edge : BASELINE_EDGES) {
// fleetd #131 step 1: move ConnectionIdentity so authz stops depending on the String[] originAndTarget = edge.split(" -> ");
// MCP layer. Evidence: auth/CallerResolver.java:3 imports mcp.ConnectionIdentity; rule = rule.ignoreDependency(originAndTarget[0], originAndTarget[1]);
// mcp/FleetMcp.java:3-7 imports auth.AuditLog, Authz, CallerResolver, Principal, }
// Role.
rule = ignoreCycle(rule, "auth", "mcp");
// fleetd #131 step 2: PrimaryRegistry is used by loops in msg; move it, or put
// an interface between msg and mcp. Evidence: msg/ReplyPushLoop.java:5 and
// msg/LeadHeartbeatLoop.java:5 import mcp.PrimaryRegistry; mcp/FleetMcp.java:15-18
// imports msg.LeadChannel, LeadMessage, MessageService, Rendezvous.
rule = ignoreCycle(rule, "mcp", "msg");
// fleetd #131 -- found while implementing this test, NOT one of the ticket's
// original three; it names its own follow-up step before removal. Evidence:
// inject/CompletionResolver.java:4-5, inject/Injector.java:6 and
// inject/TurnListener.java:3 import msg.Rendezvous / msg.TurnToken;
// msg/MessageService.java:6 imports inject.Injector.
rule = ignoreCycle(rule, "inject", "msg");
// fleetd #131 -- same as above, its own follow-up. Evidence:
// metrics/FleetMetrics.java:3 imports msg.ReplyInbox; msg/MessageService.java:7-8,
// msg/LeadHeartbeatLoop.java:6-7 and msg/ReplyPushLoop.java:6-7 import
// metrics.FleetMetrics / metrics.Metrics.
rule = ignoreCycle(rule, "metrics", "msg");
// fleetd #131 -- same as above, its own follow-up. Evidence:
// session/SessionManager.java:7 imports msg.TurnToken;
// msg/LeadHeartbeatLoop.java:8 imports session.MemberSession.
rule = ignoreCycle(rule, "msg", "session");
rule.check(classes); rule.check(classes);
} }
/** /**
* Accepts today's known cycle between two top-level packages, and nothing else. * Fails with the exact offending edge when the live code and {@link #BASELINE_EDGES}
* Ignoring both directions removes exactly this pair from cycle detection; every * disagree: a dependency crossing a baselined package pair that is not in the baseline,
* other dependency -- including any new one added later, between these same two * or a baseline entry whose dependency no longer exists.
* packages or any other pair -- is still checked.
*/ */
private static SliceRule ignoreCycle(SliceRule rule, String packageA, String packageB) { private static void checkBaselineMatchesTodaysEdges(JavaClasses classes) {
return rule Set<String> baselinedPackagePairs = new TreeSet<>();
.ignoreDependency(residesIn(packageA), residesIn(packageB)) for (String edge : BASELINE_EDGES) {
.ignoreDependency(residesIn(packageB), residesIn(packageA)); String[] originAndTarget = edge.split(" -> ");
baselinedPackagePairs.add(unorderedPair(
topLevelPackageOf(originAndTarget[0]), topLevelPackageOf(originAndTarget[1])));
}
Set<String> liveEdgesInBaselinedPairs = new TreeSet<>();
for (JavaClass javaClass : classes) {
for (Dependency dependency : javaClass.getDirectDependenciesFromSelf()) {
JavaClass origin = dependency.getOriginClass();
JavaClass target = dependency.getTargetClass();
String originPackage = topLevelPackageOf(origin.getFullName());
String targetPackage = topLevelPackageOf(target.getFullName());
if (originPackage.isEmpty() || targetPackage.isEmpty() || originPackage.equals(targetPackage)) {
continue;
}
if (baselinedPackagePairs.contains(unorderedPair(originPackage, targetPackage))) {
liveEdgesInBaselinedPairs.add(origin.getFullName() + " -> " + target.getFullName());
}
}
}
List<String> problems = new ArrayList<>();
for (String liveEdge : liveEdgesInBaselinedPairs) {
if (!BASELINE_EDGES.contains(liveEdge)) {
String[] originAndTarget = liveEdge.split(" -> ");
problems.add("new dependency not in the baseline: " + liveEdge
+ " (packages " + topLevelPackageOf(originAndTarget[0])
+ " -> " + topLevelPackageOf(originAndTarget[1]) + ")");
}
}
for (String baselineEdge : BASELINE_EDGES) {
if (!liveEdgesInBaselinedPairs.contains(baselineEdge)) {
String[] originAndTarget = baselineEdge.split(" -> ");
problems.add("stale baseline entry, no such dependency exists: " + baselineEdge
+ " (packages " + topLevelPackageOf(originAndTarget[0])
+ " -> " + topLevelPackageOf(originAndTarget[1]) + ")");
}
}
if (!problems.isEmpty()) {
fail("PackageCyclesTest baseline is out of date:\n " + String.join("\n ", problems));
}
} }
private static DescribedPredicate<JavaClass> residesIn(String topLevelPackage) { private static String unorderedPair(String packageA, String packageB) {
return Predicates.resideInAPackage("dev.ltms.fleet." + topLevelPackage + ".."); return packageA.compareTo(packageB) <= 0 ? packageA + "|" + packageB : packageB + "|" + packageA;
}
private static String topLevelPackageOf(String fullyQualifiedClassName) {
if (!fullyQualifiedClassName.startsWith(ROOT_PACKAGE)) {
return "";
}
String rest = fullyQualifiedClassName.substring(ROOT_PACKAGE.length());
int dot = rest.indexOf('.');
return dot < 0 ? "" : rest.substring(0, dot);
} }
} }
@@ -15,6 +15,7 @@ class AuthzTest {
private static final Principal ARCH_DESIGN = Principal.architect("lead-designer", "term_design", 400); private static final Principal ARCH_DESIGN = Principal.architect("lead-designer", "term_design", 400);
private static final Principal ARCH_OTHER = Principal.architect("reviewer", "term_review", 500); private static final Principal ARCH_OTHER = Principal.architect("reviewer", "term_review", 500);
private static final Principal COLLABORATOR = Principal.collaborator("ops", "term_collab", 600); private static final Principal COLLABORATOR = Principal.collaborator("ops", "term_collab", 600);
private static final Principal OBSERVER = Principal.observer("term_observer", 700);
@Test @Test
void anonymousIsAuthorizedForNothing() { void anonymousIsAuthorizedForNothing() {
@@ -266,4 +267,100 @@ class AuthzTest {
assertTrue(WORKER_A.isSpawnedMember()); assertTrue(WORKER_A.isSpawnedMember());
assertTrue(ARCH_DESIGN.isSpawnedMember()); assertTrue(ARCH_DESIGN.isSpawnedMember());
} }
// ── the observer matrix ─────────────────────────────────────────────────────────────────────
@Test
void anObserverMayReadAndScrapeMetrics() {
assertTrue(Authz.permits(OBSERVER, READ, null));
assertTrue(Authz.permits(OBSERVER, METRICS, null));
}
@Test
void anObserverMayReplyAndAskOnlyAsItsOwnPane() {
assertTrue(Authz.permits(OBSERVER, REPLY, "term_observer"), "its own pane is its own");
assertTrue(Authz.permits(OBSERVER, ASK, "term_observer"));
assertFalse(Authz.permits(OBSERVER, REPLY, "term_design"),
"an observer must not reply on another pane");
assertFalse(Authz.permits(OBSERVER, REPLY, null),
"an absent target must not pass the own-session rule");
}
/**
* Every action beyond READ/METRICS/REPLY/ASK/SEND, asserted denied for an observer —
* including {@code TASK_READ}, which is the entire point of this role: an unconfigured pane
* must not be able to poll a ticket or read another session's status. {@code SEND} is excluded
* here and given its own matrix below, since — unlike every action in this loop — its grant is
* conditional on the target, not fixed.
*/
@Test
void anObserverIsDeniedEverythingBeyondReadMetricsReplyAskAndSend() {
for (Authz.Action a : Authz.Action.values()) {
if (a == READ || a == METRICS || a == REPLY || a == ASK || a == SEND) {
continue;
}
assertFalse(Authz.permits(OBSERVER, a, "term_observer", target -> true),
"an observer must not " + a + " even when the classifier accepts every target");
}
}
@Test
void anObserverIsNotCountedAsAnyOtherRole() {
assertFalse(OBSERVER.isPrimary());
assertFalse(OBSERVER.isWorker());
assertFalse(OBSERVER.isArchitect());
assertFalse(OBSERVER.isCollaborator());
assertFalse(OBSERVER.isSpawnedMember());
assertTrue(OBSERVER.isObserver());
}
// ── the observer SEND matrix ────────────────────────────────────────────────────────────────
/**
* {@code SEND} for an observer is the one grant that is conditional rather than fixed, exactly
* like a collaborator's: flipping only the observer-target classifier's answer flips only this
* outcome.
*/
@Test
void anObserverMaySendOnlyWhenTheClassifierAcceptsTheTargetAsAnObserver() {
assertTrue(Authz.permits(OBSERVER, SEND, "term_other_observer",
Authz.NO_KNOWN_LEAD_OR_COLLABORATOR, target -> true),
"the classifier accepting the target as an observer must grant SEND");
assertFalse(Authz.permits(OBSERVER, SEND, "term_other_observer",
Authz.NO_KNOWN_LEAD_OR_COLLABORATOR, target -> false),
"the classifier refusing the target must deny SEND");
assertFalse(Authz.permits(OBSERVER, SEND, "term_other_observer"),
"the real production classifier recognises no terminal as an observer target yet, "
+ "so SEND is refused by the three- and four-argument convenience forms");
}
/**
* Control for the test above: every other action's result for an observer does not move when
* the observer-target classifier does. Only {@code SEND} is wired to it.
*/
@Test
void theObserverTargetClassifierMovesOnlySendForAnObserver() {
for (Authz.Action a : Authz.Action.values()) {
if (a == SEND) {
continue;
}
assertEquals(
Authz.permits(OBSERVER, a, "term_observer"),
Authz.permits(OBSERVER, a, "term_observer",
Authz.NO_KNOWN_LEAD_OR_COLLABORATOR, target -> true),
a + " must not depend on the observer-target classifier at all");
}
}
/**
* An observer's {@code SEND} is gated on a different classifier than a collaborator's: the
* collaborator classifier accepting every target must not itself grant an observer's SEND.
*/
@Test
void anObserversSendDoesNotMoveOnTheCollaboratorClassifier() {
assertFalse(Authz.permits(OBSERVER, SEND, "term_lead", target -> true),
"an observer's SEND must consult the observer-target classifier, never the "
+ "lead-or-collaborator one");
}
} }
@@ -57,17 +57,23 @@ class CallerResolverTest {
return members; return members;
} }
/**
* With no roster wired up at all (the simple constructor), a loopback pane that owns a herdr
* pane but is not recognised as a live spawned member lands on the {@link Role#OBSERVER} floor
* — unforgeable and never token-gated, exactly like a worker's own identity, because it comes
* from the same connection-derived pane mapping.
*/
@Test @Test
void aLoopbackWorkerPaneResolvesToWorkerRegardlessOfAuthMode() { void aLoopbackPaneWithNoLiveRosterResolvesToObserverRegardlessOfAuthMode() {
Principal underTrust = new CallerResolver(workerIdentity()).resolve("127.0.0.1", 42, null); Principal underTrust = new CallerResolver(workerIdentity()).resolve("127.0.0.1", 42, null);
Principal underToken = new CallerResolver(workerIdentity(), true, "s3cret") Principal underToken = new CallerResolver(workerIdentity(), true, "s3cret")
.resolve("127.0.0.1", 42, null); .resolve("127.0.0.1", 42, null);
assertEquals(Role.WORKER, underTrust.role()); assertEquals(Role.OBSERVER, underTrust.role());
assertEquals("term_a", underTrust.terminal()); assertEquals("term_a", underTrust.terminal());
assertEquals(Role.WORKER, underToken.role(), assertEquals(Role.OBSERVER, underToken.role(),
"worker identity is unforgeable and must never be token-gated — otherwise enabling " "the floor is unforgeable and must never be token-gated — otherwise enabling auth "
+ "auth would lock the whole fleet out of fleet_reply"); + "would lock every unconfigured pane out of even READ");
assertEquals("term_a", underToken.terminal()); assertEquals("term_a", underToken.terminal());
} }
@@ -91,20 +97,20 @@ class CallerResolverTest {
} }
@Test @Test
void otherPanesRemainWorkersWhenAPinIsSet() { void otherPanesRemainAtTheFloorWhenAPinIsSet() {
Principal p = CallerResolver.pinnedTo(workerIdentity(), false, null, "term_someone_else") Principal p = CallerResolver.pinnedTo(workerIdentity(), false, null, "term_someone_else")
.resolve("127.0.0.1", 42, null); .resolve("127.0.0.1", 42, null);
assertEquals(Role.WORKER, p.role()); assertEquals(Role.OBSERVER, p.role());
assertEquals("term_a", p.terminal()); assertEquals("term_a", p.terminal());
} }
/** The pin is optional config, so an absent or whitespace one must change nothing at all. */ /** The pin is optional config, so an absent or whitespace one must change nothing at all. */
@Test @Test
void aBlankPinLeavesWorkerResolutionUntouched() { void aBlankPinLeavesFloorResolutionUntouched() {
assertEquals(Role.WORKER, assertEquals(Role.OBSERVER,
CallerResolver.pinnedTo(workerIdentity(), false, null, " ").resolve("127.0.0.1", 42, null).role()); CallerResolver.pinnedTo(workerIdentity(), false, null, " ").resolve("127.0.0.1", 42, null).role());
assertEquals(Role.WORKER, assertEquals(Role.OBSERVER,
CallerResolver.pinnedTo(workerIdentity(), false, null, null).resolve("127.0.0.1", 42, null).role()); CallerResolver.pinnedTo(workerIdentity(), false, null, null).resolve("127.0.0.1", 42, null).role());
} }
@@ -225,13 +231,13 @@ class CallerResolverTest {
* mid-scan teardown into a refusal — the real match is still found and resolves as a worker. * mid-scan teardown into a refusal — the real match is still found and resolves as a worker.
*/ */
@Test @Test
void aHerdrErrorOnANonOwningPaneStillResolvesTheRealWorker() { void aHerdrErrorOnANonOwningPaneStillResolvesTheRealPane() {
FakeHerdr vanishedElsewhere = new FakeHerdr().processInfoFailsForPane("w2:p9", "pane_not_found"); FakeHerdr vanishedElsewhere = new FakeHerdr().processInfoFailsForPane("w2:p9", "pane_not_found");
ConnectionIdentity id = new ConnectionIdentity(new PaneLocator(vanishedElsewhere), _ -> FakeHerdr.WORKER_PID); ConnectionIdentity id = new ConnectionIdentity(new PaneLocator(vanishedElsewhere), _ -> FakeHerdr.WORKER_PID);
Principal p = new CallerResolver(id).resolve("127.0.0.1", 55555, null); Principal p = new CallerResolver(id).resolve("127.0.0.1", 55555, null);
assertEquals(Role.WORKER, p.role()); assertEquals(Role.OBSERVER, p.role());
assertEquals("term_a", p.terminal()); assertEquals("term_a", p.terminal());
} }
@@ -275,12 +281,12 @@ class CallerResolverTest {
} }
@Test @Test
void aPaneAbsentFromTheRegistryIsStillAWorker() { void aPaneAbsentFromTheRegistryFallsToTheObserverFloor() {
Principal p = new CallerResolver(workerIdentity(), false, null, Principal p = new CallerResolver(workerIdentity(), false, null,
Map.of("term_elsewhere", "gpt-sol-5.6")) Map.of("term_elsewhere", "gpt-sol-5.6"))
.resolve("127.0.0.1", 42, null); .resolve("127.0.0.1", 42, null);
assertEquals(Role.WORKER, p.role()); assertEquals(Role.OBSERVER, p.role());
assertEquals("term_a", p.terminal()); assertEquals("term_a", p.terminal());
assertNull(p.name()); assertNull(p.name());
} }
@@ -306,16 +312,16 @@ class CallerResolverTest {
} }
@Test @Test
void anEmptyRegistryLeavesEveryPaneAWorker() { void anEmptyRegistryLeavesEveryPaneAtTheObserverFloor() {
Map<String, String> noLeads = null; Map<String, String> noLeads = null;
assertEquals(Role.WORKER, assertEquals(Role.OBSERVER,
new CallerResolver(workerIdentity(), false, null, Map.of()) new CallerResolver(workerIdentity(), false, null, Map.of())
.resolve("127.0.0.1", 42, null).role()); .resolve("127.0.0.1", 42, null).role());
assertEquals(Role.WORKER, assertEquals(Role.OBSERVER,
new CallerResolver(workerIdentity(), false, null, noLeads) new CallerResolver(workerIdentity(), false, null, noLeads)
.resolve("127.0.0.1", 42, null).role()); .resolve("127.0.0.1", 42, null).role());
// CB-531: and the same for the live-registry form, whose supplier may also be absent. // And the same for the live-registry form, whose supplier may also be absent.
assertEquals(Role.WORKER, assertEquals(Role.OBSERVER,
CallerResolver.withLeads(workerIdentity(), false, null, null) CallerResolver.withLeads(workerIdentity(), false, null, null)
.resolve("127.0.0.1", 42, null).role()); .resolve("127.0.0.1", 42, null).role());
} }
@@ -388,7 +394,7 @@ class CallerResolverTest {
Map<String, String> live = new java.util.HashMap<>(); Map<String, String> live = new java.util.HashMap<>();
CallerResolver r = CallerResolver.withLeads(workerIdentity(), false, null, () -> live); CallerResolver r = CallerResolver.withLeads(workerIdentity(), false, null, () -> live);
assertEquals(Role.WORKER, r.resolve("127.0.0.1", 42, null).role()); assertEquals(Role.OBSERVER, r.resolve("127.0.0.1", 42, null).role());
live.put("term_a", "gpt-sol-5.6"); // the scanner sees a newly-labelled tab live.put("term_a", "gpt-sol-5.6"); // the scanner sees a newly-labelled tab
@@ -405,7 +411,7 @@ class CallerResolverTest {
mutable.put("term_a", "sneaky"); mutable.put("term_a", "sneaky");
assertEquals(Role.WORKER, r.resolve("127.0.0.1", 42, null).role()); assertEquals(Role.OBSERVER, r.resolve("127.0.0.1", 42, null).role());
} }
// ── CB-548: architect slots ───────────────────────────────────────────────────────────────── // ── CB-548: architect slots ─────────────────────────────────────────────────────────────────
@@ -435,13 +441,13 @@ class CallerResolverTest {
} }
@Test @Test
void anUnboundPaneStillResolvesAsAWorker() { void anUnboundPaneResolvesToTheObserverFloor() {
MemberRegistry members = new MemberRegistry(new FleetConfig.Fleet(Map.of(), MemberRegistry members = new MemberRegistry(new FleetConfig.Fleet(Map.of(),
Map.of("lead-designer", new FleetConfig.Slot("sonnet")), Map.of(), Map.of(), null)); Map.of("lead-designer", new FleetConfig.Slot("sonnet")), Map.of(), Map.of(), null));
Principal p = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null, Principal p = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null,
Map::of, members).resolve("127.0.0.1", 42, null); Map::of, members).resolve("127.0.0.1", 42, null);
assertEquals(Role.WORKER, p.role()); assertEquals(Role.OBSERVER, p.role());
assertNull(p.name()); assertNull(p.name());
} }
@@ -467,7 +473,7 @@ class CallerResolverTest {
CallerResolver r = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null, CallerResolver r = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null,
Map::of, members); Map::of, members);
assertEquals(Role.WORKER, r.resolve("127.0.0.1", 42, null).role()); assertEquals(Role.OBSERVER, r.resolve("127.0.0.1", 42, null).role());
assertTrue(members.bind("architect:lead-designer", "term_a")); // the later lifecycle binds the slot assertTrue(members.bind("architect:lead-designer", "term_a")); // the later lifecycle binds the slot
@@ -494,12 +500,13 @@ class CallerResolverTest {
} }
@Test @Test
void aBoundNonArchitectSlotStillResolvesAsAWorker() { void aBoundNonArchitectSlotResolvesToTheObserverFloorNotArchitect() {
Principal p = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null, Principal p = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null,
Map::of, boundMembers("dev:builder", MemberRole.DEV)) Map::of, boundMembers("dev:builder", MemberRole.DEV))
.resolve("127.0.0.1", 42, null); .resolve("127.0.0.1", 42, null);
assertEquals(Role.WORKER, p.role(), "a dev binding must never grant architect rights"); assertEquals(Role.OBSERVER, p.role(), "a dev binding must never grant architect rights, "
+ "and this construction path wires no roster to recognise it as the live dev it is");
} }
@Test @Test
@@ -510,17 +517,17 @@ class CallerResolverTest {
assertThrows(IllegalArgumentException.class, () -> new CallerResolver(id, true, " ")); assertThrows(IllegalArgumentException.class, () -> new CallerResolver(id, true, " "));
} }
@Test @Test
void aWorkerOnAnyLoopbackSourceAddressIsStillAWorkerNotThePrimary() { void aPaneOnAnyLoopbackSourceAddressIsStillAtTheFloorNotThePrimary() {
// fleetd #305: the escalation. ConnectionIdentity used to accept only 127.0.0.1, so a // fleetd #305: the escalation this guards against. ConnectionIdentity used to accept only
// worker connecting from 127.0.0.2 resolved to no terminal, and this resolver's own // 127.0.0.1, so a pane connecting from 127.0.0.2 resolved to no terminal, and this
// (wider) loopback check then made it the PRIMARY — granting spawn, stop, send and drain. // resolver's own (wider) loopback check then made it the PRIMARY — granting spawn, stop,
// Measured on the Linux fleet host: binding a source of 127.0.0.2 succeeds there, so the // send and drain. Measured on the Linux fleet host: binding a source of 127.0.0.2 succeeds
// path is real and not theoretical. // there, so the path is real and not theoretical.
CallerResolver r = new CallerResolver(workerIdentity(), false, null); CallerResolver r = new CallerResolver(workerIdentity(), false, null);
for (String src : new String[]{"127.0.0.1", "127.0.0.2", "127.1.2.3", "::ffff:127.0.0.2"}) { for (String src : new String[]{"127.0.0.1", "127.0.0.2", "127.1.2.3", "::ffff:127.0.0.2"}) {
Principal p = r.resolve(src, 55555, null); Principal p = r.resolve(src, 55555, null);
assertEquals(Role.WORKER, p.role(), "a worker must stay a worker from source " + src); assertEquals(Role.OBSERVER, p.role(), "the pane must stay off PRIMARY from source " + src);
assertEquals("term_a", p.terminal(), "worker terminal from source " + src); assertEquals("term_a", p.terminal(), "pane terminal from source " + src);
} }
} }
@@ -826,14 +833,14 @@ class CallerResolverTest {
assertEquals("term_a", p.terminal()); assertEquals("term_a", p.terminal());
} }
/** Regression: an empty collaborator registry leaves every pane exactly as before. */ /** Regression: an empty collaborator registry leaves every pane at the unconfigured-pane floor. */
@Test @Test
void anEmptyCollaboratorRegistryLeavesEveryPaneAsBefore() { void anEmptyCollaboratorRegistryLeavesEveryPaneAtTheObserverFloor() {
Principal p = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null, Map::of, Principal p = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null, Map::of,
new MemberRegistry(null), t -> null, Map::of) new MemberRegistry(null), t -> null, Map::of)
.resolve("127.0.0.1", 42, null); .resolve("127.0.0.1", 42, null);
assertEquals(Role.WORKER, p.role()); assertEquals(Role.OBSERVER, p.role());
assertNull(p.name()); assertNull(p.name());
} }
@@ -862,6 +869,39 @@ class CallerResolverTest {
assertFalse(r.knownLeadOrCollaborator().test("term_other")); assertFalse(r.knownLeadOrCollaborator().test("term_other"));
} }
// ── fleetd #705: narrowing the unconfigured-pane floor to OBSERVER ──────────────────────────
/**
* The case this ticket exists for: a pane the resolver cannot place as a live spawned member,
* a lead, a bound architect slot, or a configured collaborator must land on the narrow
* {@link Role#OBSERVER} floor, never the {@link Role#WORKER} the old fallback granted.
*
* <p>The second assertion is the control the ticket requires: a terminal the roster DOES
* recognise as a live spawned member must still resolve its own role. Without it, this test
* would also pass if the fix accidentally turned every caller into an observer.
*/
@Test
void anUnconfiguredPaneResolvesObserverButARegisteredMemberStillResolvesItsOwnRole() {
Principal unconfigured = new CallerResolver(workerIdentity()).resolve("127.0.0.1", 42, null);
assertEquals(Role.OBSERVER, unconfigured.role(),
"a pane matching none of the configured or live-roster roles must fall to the "
+ "floor, not WORKER");
assertEquals("term_a", unconfigured.terminal());
Principal registered = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null,
Map::of, new MemberRegistry(null),
t -> "term_a".equals(t) ? MemberRole.DEV : null, Map::of)
.resolve("127.0.0.1", 42, null);
assertEquals(Role.WORKER, registered.role(),
"control: a live spawned member must keep resolving its own role, never the "
+ "unconfigured-pane floor");
}
@Test
void describeNamesTheObserverByItsPane() {
assertEquals("observer:term_a", Principal.observer("term_a", 1).describe());
}
@Test @Test
void knownLeadOrCollaboratorIsFalseForASpawnedMembersTerminal() { void knownLeadOrCollaboratorIsFalseForASpawnedMembersTerminal() {
// The exact scenario a collaborator's SEND must never reach: a live spawned member's own // The exact scenario a collaborator's SEND must never reach: a live spawned member's own
@@ -872,4 +912,72 @@ class CallerResolverTest {
assertFalse(r.knownLeadOrCollaborator().test("term_a")); assertFalse(r.knownLeadOrCollaborator().test("term_a"));
} }
// ── fleetd #743: sendableObserverTarget() reads the same maps and functions resolve() does ────
/**
* A terminal this resolver recognises as none of the privileged roles is exactly the one
* {@code resolve} would itself hand back {@link Role#OBSERVER} for.
*/
@Test
void sendableObserverTargetIsTrueForATerminalKnownAsNoOtherRole() {
CallerResolver r = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null,
() -> Map.of("term_lead", "opus-5.0"), new MemberRegistry(null),
t -> "term_worker".equals(t) ? MemberRole.DEV : null,
() -> Map.of("term_collab", "ops"));
assertTrue(r.sendableObserverTarget().test("term_other"));
}
@Test
void sendableObserverTargetIsFalseForALeadTerminal() {
CallerResolver r = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null,
() -> Map.of("term_lead", "opus-5.0"), new MemberRegistry(null), t -> null, Map::of);
assertFalse(r.sendableObserverTarget().test("term_lead"),
"a lead's own terminal must never be a sendable observer target");
}
@Test
void sendableObserverTargetIsFalseForACollaboratorTerminal() {
CallerResolver r = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null, Map::of,
new MemberRegistry(null), t -> null, () -> Map.of("term_collab", "ops"));
assertFalse(r.sendableObserverTarget().test("term_collab"),
"a collaborator's own terminal must never be a sendable observer target");
}
@Test
void sendableObserverTargetIsFalseForALiveSpawnedMembersTerminal() {
// Covers both a worker and an architect: spawnedMemberRole.apply(target) is non-null for
// either, and resolve() never falls through to OBSERVER once it is.
CallerResolver r = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null, Map::of,
new MemberRegistry(null),
t -> switch (t) {
case "term_worker" -> MemberRole.DEV;
case "term_architect" -> MemberRole.ARCHITECT;
default -> null;
}, Map::of);
assertFalse(r.sendableObserverTarget().test("term_worker"));
assertFalse(r.sendableObserverTarget().test("term_architect"));
}
@Test
void sendableObserverTargetIsFalseForABoundArchitectSlotWithNoLiveMember() {
// The edge case resolve() itself carries: a terminal bound to a configured architect slot
// but with no live spawned-member session yet.
CallerResolver r = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null, Map::of,
boundMembers("architect:lead-designer", MemberRole.ARCHITECT), t -> null, Map::of);
assertFalse(r.sendableObserverTarget().test("term_a"));
}
@Test
void sendableObserverTargetIsFalseForANullTarget() {
CallerResolver r = CallerResolver.withLeadsAndMembers(workerIdentity(), false, null, Map::of,
new MemberRegistry(null), t -> null, Map::of);
assertFalse(r.sendableObserverTarget().test(null));
}
} }
@@ -295,9 +295,10 @@ class MemberRegistryLiveTest {
assertTrue(out.applied(), "the reload must actually take effect: " + out.summary()); assertTrue(out.applied(), "the reload must actually take effect: " + out.summary());
Principal after = resolver.resolve("127.0.0.1", 42, null); Principal after = resolver.resolve("127.0.0.1", 42, null);
assertEquals(Role.WORKER, after.role(), assertEquals(Role.OBSERVER, after.role(),
"removing the slot from config must demote the bound session to worker on its " "removing the slot from config must demote the bound session on its NEXT request — "
+ "NEXT request — this is the ticket's whole point"); + "this harness wires no live roster for term_a, so the demotion lands on "
+ "the unconfigured-pane floor");
assertEquals("term_a", after.terminal(), "same pane, same terminal — only the role changed"); assertEquals("term_a", after.terminal(), "same pane, same terminal — only the role changed");
} }
@@ -0,0 +1,31 @@
package dev.ltms.fleet.auth;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
class PrincipalTest {
@Test
void ownerKeyCoversEveryRole() {
assertEquals("leader:opus", Principal.leader("opus", "term_lead", 1).ownerKey());
assertNull(Principal.primary(2).ownerKey());
assertEquals("worker:term_worker", Principal.worker("term_worker", 3).ownerKey());
assertEquals("architect:term_arch", Principal.architect("opus", "term_arch", 4).ownerKey());
assertEquals("collaborator:ops", Principal.collaborator("ops", "term_collab", 5).ownerKey());
assertEquals("observer:term_observer", Principal.observer("term_observer", 6).ownerKey());
assertEquals("anonymous", Principal.anonymous().ownerKey());
}
@Test
void rolePrefixesKeepLeadAndArchitectKeysDistinct() {
String lead = Principal.leader("opus", "term_lead", 1).ownerKey();
String architect = Principal.architect("design", "opus", 2).ownerKey();
assertEquals("leader:opus", lead);
assertEquals("architect:opus", architect);
assertNotEquals(lead, architect);
}
}
@@ -537,12 +537,13 @@ class FleetConfigTest {
} }
/** /**
* CB-579: {@code tab} is the only field a lead's identity depends on now, so it is required * A lead's tab label is a fixed constant, not a per-entry field, so an entry with no {@code
* whether the entry is creatable or recognise-only — without it the entry can never be found. * tab:} is the normal case — it is still found by that constant label in its own {@code
* workspace}, not refused as useless.
*/ */
@Test @Test
void aLeadWithNoTabRefusesToStart(@TempDir Path dir) throws Exception { void aLeadWithNoTabIsAcceptedAndFoundByTheFixedLabel(@TempDir Path dir) throws Exception {
Path f = dir.resolve("useless-lead.yaml"); Path f = dir.resolve("no-tab-lead.yaml");
Files.writeString(f, """ Files.writeString(f, """
bind: bind:
port: 8080 port: 8080
@@ -553,9 +554,9 @@ class FleetConfigTest {
"""); """);
FleetConfig cfg = FleetConfig.load(f); FleetConfig cfg = FleetConfig.load(f);
IllegalStateException e = assertThrows(IllegalStateException.class, cfg::validateMembers); assertDoesNotThrow(cfg::validateMembers);
assertTrue(e.getMessage().contains("ghost"), "the message must name the useless entry"); assertEquals(List.of(FleetConfig.Leader.LEAD_TAB_LABEL),
assertTrue(e.getMessage().contains("tab:"), "the message must say what is missing"); cfg.fleet().leaders().get("ghost").acceptedLabels());
} }
/** /**
@@ -775,22 +776,45 @@ class FleetConfigTest {
} }
/** /**
* fleetd #677: identity is matched on a lead's exact {@code tab} alone, so two leads sharing * A lead's tab label is fixed, so two leads sharing one {@code workspace} would both resolve
* one tab means only one of them is ever found — the guard must catch this independently of * to the one tab named {@code lead} there — the guard must catch this independently of the
* the member-template checks above. * member-template checks above.
*/ */
@Test @Test
void twoLeadsSharingTheSameExactTabRefusesToStart(@TempDir Path dir) throws Exception { void twoLeadersSharingTheSameWorkspaceRefuseToStart(@TempDir Path dir) throws Exception {
Path f = dir.resolve("shared-tab.yaml"); Path f = dir.resolve("shared-workspace.yaml");
Files.writeString(f, """ Files.writeString(f, """
bind: bind:
port: 8080 port: 8080
fleet: fleet:
leaders: leaders:
opus: opus:
tab: "shared tab" workspace: "shared"
sonnet: sonnet:
tab: "shared tab" workspace: "shared"
""");
FleetConfig cfg = FleetConfig.load(f);
IllegalStateException e =
assertThrows(IllegalStateException.class, cfg::validateLeadTabPrefixes);
assertTrue(e.getMessage().contains("opus"), "the message must name one offending lead");
assertTrue(e.getMessage().contains("sonnet"), "the message must name the other offending lead");
assertTrue(e.getMessage().contains("shared"), "the message must name the shared workspace");
}
/** The workspace collision check is case-insensitive, matching how spaces are looked up. */
@Test
void twoLeadersSharingTheSameWorkspaceInDifferentCaseRefuseToStart(@TempDir Path dir) throws Exception {
Path f = dir.resolve("shared-workspace-case.yaml");
Files.writeString(f, """
bind:
port: 8080
fleet:
leaders:
opus:
workspace: "Shared"
sonnet:
workspace: "shared"
"""); """);
FleetConfig cfg = FleetConfig.load(f); FleetConfig cfg = FleetConfig.load(f);
@@ -800,52 +824,77 @@ class FleetConfigTest {
assertTrue(e.getMessage().contains("sonnet"), "the message must name the other offending lead"); assertTrue(e.getMessage().contains("sonnet"), "the message must name the other offending lead");
} }
/** /** Control for the two tests above: distinct workspaces load cleanly, with no {@code tab:} at all. */
* fleetd #693: the guard matches tabs case-insensitively, because
* {@code LeadTabScanner} keys its tab map on a lowercased label — two tabs differing only in
* case collide there too, and the guard must catch that independently of the exact-match case
* above.
*/
@Test @Test
void twoLeadsSharingTheSameTabInDifferentCaseRefusesToStart(@TempDir Path dir) throws Exception { void twoLeadersWithDistinctWorkspacesAndNoTabAreAllowed(@TempDir Path dir) throws Exception {
Path f = dir.resolve("shared-tab-case.yaml"); Path f = dir.resolve("distinct-workspaces.yaml");
Files.writeString(f, """ Files.writeString(f, """
bind: bind:
port: 8080 port: 8080
fleet: fleet:
leaders: leaders:
opus: opus:
tab: "Shared Tab" workspace: "space-opus"
sonnet: sonnet:
tab: "shared tab" workspace: "space-sonnet"
""");
FleetConfig cfg = FleetConfig.load(f);
IllegalStateException e =
assertThrows(IllegalStateException.class, cfg::validateLeadTabPrefixes);
assertTrue(e.getMessage().contains("opus"), "the message must name one offending lead");
assertTrue(e.getMessage().contains("sonnet"), "the message must name the other offending lead");
}
/** Control for {@link #twoLeadsSharingTheSameExactTabRefusesToStart}: distinct tabs load cleanly. */
@Test
void twoLeadsWithDistinctExactTabsAreAllowed(@TempDir Path dir) throws Exception {
Path f = dir.resolve("distinct-tabs.yaml");
Files.writeString(f, """
bind:
port: 8080
fleet:
leaders:
opus:
tab: "opus tab"
sonnet:
tab: "sonnet tab"
"""); """);
FleetConfig cfg = FleetConfig.load(f); FleetConfig cfg = FleetConfig.load(f);
assertDoesNotThrow(cfg::validateLeadTabPrefixes); assertDoesNotThrow(cfg::validateLeadTabPrefixes);
} }
/**
* A member tab-label template that can render exactly as the fixed lead tab label would let a
* member's own tab be read back as a lead — refused outright, with no lead needing to be
* configured at all.
*/
@Test
void aFleetTabLabelTemplateThatCanRenderAsTheFixedLeadTabLabelRefusesToStart(@TempDir Path dir)
throws Exception {
Path f = dir.resolve("template-renders-as-lead.yaml");
Files.writeString(f, """
bind:
port: 8080
fleet:
tabLabel: "lead"
leaders:
opus:
workspace: "fleet"
""");
FleetConfig cfg = FleetConfig.load(f);
IllegalStateException e =
assertThrows(IllegalStateException.class, cfg::validateLeadTabPrefixes);
assertTrue(e.getMessage().contains("fleet.tabLabel"),
"the message must name the offending template");
assertTrue(e.getMessage().contains(FleetConfig.Leader.LEAD_TAB_LABEL),
"the message must name the fixed lead tab label it collides with");
}
/**
* A collaborator's {@code tab} equal to the fixed lead tab label would shadow a lead sharing
* that space — refused outright.
*/
@Test
void aCollaboratorTabEqualToTheFixedLeadTabLabelRefusesToStart(@TempDir Path dir) throws Exception {
Path f = dir.resolve("collaborator-is-lead.yaml");
Files.writeString(f, """
bind:
port: 8080
fleet:
collaborators:
impostor:
tab: "lead"
""");
FleetConfig cfg = FleetConfig.load(f);
IllegalStateException e =
assertThrows(IllegalStateException.class, cfg::validateLeadTabPrefixes);
assertTrue(e.getMessage().contains("impostor"), "the message must name the offending collaborator");
assertTrue(e.getMessage().contains(FleetConfig.Leader.LEAD_TAB_LABEL),
"the message must name the fixed lead tab label it collides with");
}
// ── validatePanePlacementAgainstLeadTabs ──────────────────────────────────────────────────── // ── validatePanePlacementAgainstLeadTabs ────────────────────────────────────────────────────
/** /**
@@ -874,8 +923,12 @@ class FleetConfigTest {
assertTrue(e.getMessage().contains("gx10"), "the message must name the offending profile"); assertTrue(e.getMessage().contains("gx10"), "the message must name the offending profile");
} }
/**
* A lead is found by its fixed tab label regardless of its own {@code tab} field, so a
* {@code fleet.leaders} entry with no {@code tab} must still arm the guard.
*/
@Test @Test
void aPanePlacedProfileWithNoLeadTabIsAllowed(@TempDir Path dir) throws Exception { void aPanePlacedProfileWithNoLeadTabRefusesToStart(@TempDir Path dir) throws Exception {
Path f = dir.resolve("pane-no-tab.yaml"); Path f = dir.resolve("pane-no-tab.yaml");
Files.writeString(f, """ Files.writeString(f, """
bind: bind:
@@ -888,9 +941,59 @@ class FleetConfigTest {
opus: opus:
profile: gx10 profile: gx10
"""); """);
FleetConfig cfg = FleetConfig.load(f);
IllegalStateException e = assertThrows(IllegalStateException.class,
cfg::validatePanePlacementAgainstLeadTabs);
assertTrue(e.getMessage().contains("gx10"), "the message must name the offending profile");
assertTrue(e.getMessage().contains("the only fix when a lead triggered this"),
"the message must say placement: tab is the only fix for a lead");
assertFalse(e.getMessage().contains("remove the tab from every fleet.leaders"),
"the message must not send the operator in a circle by advising a tab: removal");
}
/**
* {@code placement:} is optional, and {@link FleetConfig.Profile}'s own compact constructor
* defaults an absent or blank value to {@code "tab"}, so a profile naming no placement at all
* is tab-placed and the guard must not fire for it.
*/
@Test
void aProfileWithNoPlacementKeyDefaultsToTabPlacementAndIsAllowed(@TempDir Path dir)
throws Exception {
Path f = dir.resolve("no-placement-key.yaml");
Files.writeString(f, """
bind:
port: 8080
profiles:
gx10: {}
fleet:
leaders:
opus:
profile: gx10
""");
FleetConfig cfg = FleetConfig.load(f);
assertTrue(cfg.profiles().get("gx10").tabPlacement(),
"Profile's compact constructor defaults an absent placement to \"tab\"");
assertDoesNotThrow(cfg::validatePanePlacementAgainstLeadTabs,
"a profile with no placement: key is tab-placed, not pane-placed");
}
/** Control: no {@code fleet.leaders} entry and no collaborator tab still starts fine. */
@Test
void aPanePlacedProfileWithAnEmptyFleetBlockIsAllowed(@TempDir Path dir) throws Exception {
Path f = dir.resolve("pane-empty-fleet.yaml");
Files.writeString(f, """
bind:
port: 8080
profiles:
gx10:
placement: pane
fleet: {}
""");
assertDoesNotThrow(() -> FleetConfig.load(f).validatePanePlacementAgainstLeadTabs(), assertDoesNotThrow(() -> FleetConfig.load(f).validatePanePlacementAgainstLeadTabs(),
"a leader with no tab feeds nothing into the scanner, so pane placement is safe"); "no fleet.leaders entry and no collaborator tab means pane placement is safe");
} }
@Test @Test
@@ -103,7 +103,7 @@ class FleetConfigWithDefaultsPreservesEveryComponentTest {
// comment there), same as broker/primary/leadHeartbeat/... above — a real, non-null value // comment there), same as broker/primary/leadHeartbeat/... above — a real, non-null value
// here proves it, rather than leaving it null and proving nothing. // here proves it, rather than leaving it null and proving nothing.
v.put("leadRollover", new FleetConfig.LeadRollover( v.put("leadRollover", new FleetConfig.LeadRollover(
"/handover/guard.md", true, 3600, 20, 20, "read the handover file")); "/handover/guard.md", true, 3600, 20, 45, "read the handover file"));
assertNamesMatchComponents(v); assertNamesMatchComponents(v);
return v; return v;
} }
@@ -7,6 +7,7 @@ import java.util.ArrayList;
import java.util.LinkedHashMap; import java.util.LinkedHashMap;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList; import java.util.concurrent.CopyOnWriteArrayList;
@@ -23,6 +24,41 @@ public final class FakeHerdr implements HerdrClient {
/** The foreground PID of the one agent pane (term_a) in the canned {@code pane.process_info}. */ /** The foreground PID of the one agent pane (term_a) in the canned {@code pane.process_info}. */
public static final long WORKER_PID = 4242; public static final long WORKER_PID = 4242;
/**
* A {@code detection} region of the current Claude Code TUI, whose input box is empty: a caret
* line between two rules, above the footer.
*/
public static final String IDLE_PROMPT_CARET = """
──────────────────────── lead: opus ─
❯
─────────────────────────────────────
lead: opus · Opus 5 (1M context) · ~/LTMS/claude-bridge
⏵⏵ auto mode on (shift+tab to cycle) · ← 1 agent""";
/** The same region with the operator's unsubmitted line still at the caret. */
public static final String DRAFTED_PROMPT_CARET = """
──────────────────────── lead: opus ─
❯ yes, send it to lead: opus
─────────────────────────────────────
lead: opus · Opus 5 (1M context) · ~/LTMS/claude-bridge
⏵⏵ auto mode on (shift+tab to cycle) · ← 1 agent""";
/** A {@code detection} region of an older TUI, which drew a bordered box, with that box empty. */
public static final String IDLE_PROMPT_BOX = """
⏺ done
╭────────────────────────╮
│ > │
╰────────────────────────╯
⏵⏵ auto mode on""";
/** The older TUI's bordered box, still holding the operator's unsubmitted line. */
public static final String DRAFTED_PROMPT_BOX = """
⏺ done
╭────────────────────────╮
│ > fix the issue when I │
╰────────────────────────╯
⏵⏵ auto mode on""";
private final ObjectMapper mapper = new ObjectMapper(); private final ObjectMapper mapper = new ObjectMapper();
/** /**
* Thread-safe on purpose. Background loops — {@link dev.ltms.fleet.msg.ReplyPushLoop} and the * Thread-safe on purpose. Background loops — {@link dev.ltms.fleet.msg.ReplyPushLoop} and the
@@ -47,18 +83,25 @@ public final class FakeHerdr implements HerdrClient {
private final Map<String, String> processInfoErrorCodeFor = new ConcurrentHashMap<>(); private final Map<String, String> processInfoErrorCodeFor = new ConcurrentHashMap<>();
private String tabCloseErrorCode = null; private String tabCloseErrorCode = null;
private final Map<String, String> tabCloseErrorCodeFor = new ConcurrentHashMap<>(); private final Map<String, String> tabCloseErrorCodeFor = new ConcurrentHashMap<>();
private String workspaceListErrorCode = null;
private String agentSendErrorCode = null; private String agentSendErrorCode = null;
private boolean noPanes = false; private boolean noPanes = false;
private volatile String agentStatus = "idle"; // steady-state agent.get status private volatile String agentStatus = "idle"; // steady-state agent.get status
private volatile String agentType = "claude"; // detected agent kind on agent.get; null = undetected private volatile String agentType = "claude"; // detected agent kind on agent.get; null = undetected
private volatile String agentSessionId = null; // agent_session.value on agent.get; null = omitted private volatile String agentSessionId = null; // agent_session.value on agent.get; null = omitted
private volatile String readText = "worker transcript tail"; // canned agent.read output private volatile String readText = "worker transcript tail"; // canned agent.read output
/** Canned {@code detection}-source output, or {@code null} to serve {@link #readText} there too. */
private volatile String detectionText = null;
private int pinnedStarts = 0; // how many upcoming agent.start calls report a fixed pane private int pinnedStarts = 0; // how many upcoming agent.start calls report a fixed pane
private String pinnedStartTerminal; private String pinnedStartTerminal;
private String pinnedStartPane; private String pinnedStartPane;
private Runnable onAgentStart; // fires the instant agent.start is called — see onAgentStart(Runnable) private Runnable onAgentStart; // fires the instant agent.start is called — see onAgentStart(Runnable)
private volatile int agentGetOkCalls = Integer.MAX_VALUE; // how many agent.get calls succeed first private volatile int agentGetOkCalls = Integer.MAX_VALUE; // how many agent.get calls succeed first
private volatile String agentGetFailCode = null; // error code every agent.get call after that reports private volatile String agentGetFailCode = null; // error code every agent.get call after that reports
/** pane ids that {@link #paneGoneAfterClose} has opted into reporting gone — see that method. */
private final Set<String> paneGoneAfterCloseIds = ConcurrentHashMap.newKeySet();
/** pane ids a {@code pane.close} call has actually reached, for {@link #paneGoneAfterCloseIds}. */
private final Set<String> closedPaneIds = ConcurrentHashMap.newKeySet();
public FakeHerdr healthy(boolean h) { public FakeHerdr healthy(boolean h) {
this.healthy = h; this.healthy = h;
@@ -137,6 +180,12 @@ public final class FakeHerdr implements HerdrClient {
return this; return this;
} }
/** Make {@code workspace.list} fail with this herdr error code; every other method still succeeds. */
public FakeHerdr workspaceListFailsWith(String code) {
this.workspaceListErrorCode = code;
return this;
}
/** /**
* Make {@code pane.list} report no panes at all — models a second herdr daemon (CB-185) that * Make {@code pane.list} report no panes at all — models a second herdr daemon (CB-185) that
* simply does not host the pane a {@link PaneLocator} is searching for. * simply does not host the pane a {@link PaneLocator} is searching for.
@@ -195,12 +244,26 @@ public final class FakeHerdr implements HerdrClient {
return this; return this;
} }
/** The text {@code agent.read} returns (the CB-106 completion scrape). */ /**
* The text {@code agent.read} returns (the CB-106 completion scrape). It serves the
* {@code detection} source as well unless {@link #detectionText} overrides that one.
*/
public FakeHerdr readText(String text) { public FakeHerdr readText(String text) {
this.readText = text; this.readText = text;
return this; return this;
} }
/**
* Override the text {@code agent.read} returns for the {@code detection} source only — the
* prompt/footer tail herdr uses for status detection, a different region from the transcript the
* other sources carry. Needed by a test whose subject reads the input box, since one
* {@link #readText} cannot be both a worker's transcript and a lead's empty prompt.
*/
public FakeHerdr detectionText(String text) {
this.detectionText = text;
return this;
}
/** Make delivery ({@code agent.prompt} / {@code agent.send_keys}) fail with this error code. */ /** Make delivery ({@code agent.prompt} / {@code agent.send_keys}) fail with this error code. */
public FakeHerdr agentSendFailsWith(String code) { public FakeHerdr agentSendFailsWith(String code) {
this.agentSendErrorCode = code; this.agentSendErrorCode = code;
@@ -220,6 +283,19 @@ public final class FakeHerdr implements HerdrClient {
return this; return this;
} }
/**
* Make {@code pane.get(paneId)} report the pane gone (a {@code pane_not_found} {@link
* HerdrException}, exactly as {@link WorkspaceControl#locatePane} expects to see once a pane
* has really disappeared) once a {@code pane.close} call for that same {@code paneId} has
* actually reached this fake. Every other pane, and this pane before its own close, keeps
* reporting the default canned {@code pane.get} response — opt-in, by pane id, so no existing
* test's {@code pane.get} behaviour changes.
*/
public FakeHerdr paneGoneAfterClose(String paneId) {
paneGoneAfterCloseIds.add(paneId);
return this;
}
/** /**
* Run {@code hook} synchronously the instant an {@code agent.start} call reaches this fake — * Run {@code hook} synchronously the instant an {@code agent.start} call reaches this fake —
* i.e. the instant the peer PROCESS would start against a real herdr daemon. A test uses this * i.e. the instant the peer PROCESS would start against a real herdr daemon. A test uses this
@@ -284,11 +360,17 @@ public final class FakeHerdr implements HerdrClient {
case "ping" -> mapper.readTree( case "ping" -> mapper.readTree(
("{\"type\":\"pong\",\"version\":\"%s\",\"protocol\":%d}") ("{\"type\":\"pong\",\"version\":\"%s\",\"protocol\":%d}")
.formatted(pingVersion, pingProtocol)); .formatted(pingVersion, pingProtocol));
case "workspace.list" -> mapper.readTree((""" case "workspace.list" -> {
if (workspaceListErrorCode != null) {
throw new HerdrException("herdr error [" + workspaceListErrorCode + "]: workspace.list failed",
workspaceListErrorCode, null);
}
yield mapper.readTree(("""
{"type":"workspace_list","workspaces":[ {"type":"workspace_list","workspaces":[
{"workspace_id":"w1","label":"dev-mgnl","focused":true,"pane_count":7,"agent_status":"unknown"}, {"workspace_id":"w1","label":"dev-mgnl","focused":true,"pane_count":7,"agent_status":"unknown"},
{"workspace_id":"w2","label":"ltms","focused":false,"pane_count":5,"agent_status":"done"}%s]}""") {"workspace_id":"w2","label":"ltms","focused":false,"pane_count":5,"agent_status":"done"}%s]}""")
.formatted(extraWorkspaces.isEmpty() ? "" : "," + String.join(",", extraWorkspaces))); .formatted(extraWorkspaces.isEmpty() ? "" : "," + String.join(",", extraWorkspaces)));
}
case "agent.list" -> mapper.readTree((""" case "agent.list" -> mapper.readTree(("""
{"type":"agent_list","agents":[ {"type":"agent_list","agents":[
{"terminal_id":"term_a","agent":"claude","agent_status":"idle", {"terminal_id":"term_a","agent":"claude","agent_status":"idle",
@@ -329,8 +411,13 @@ public final class FakeHerdr implements HerdrClient {
"agent_status":"%s","workspace_id":"w2","tab_id":"w2:t7","pane_id":"w2:p7"%s}}""") "agent_status":"%s","workspace_id":"w2","tab_id":"w2:t7","pane_id":"w2:p7"%s}}""")
.formatted(agentField, agentStatus, sessionField)); .formatted(agentField, agentStatus, sessionField));
} }
case "agent.read" -> mapper.readTree(mapper.writeValueAsString( case "agent.read" -> {
java.util.Map.of("type", "agent_read", "read", java.util.Map.of("text", readText)))); Object source = params instanceof Map<?, ?> m ? m.get("source") : null;
String text = "detection".equals(source) && detectionText != null
? detectionText : readText;
yield mapper.readTree(mapper.writeValueAsString(
java.util.Map.of("type", "agent_read", "read", java.util.Map.of("text", text))));
}
case "agent.start" -> { case "agent.start" -> {
if (onAgentStart != null) { if (onAgentStart != null) {
onAgentStart.run(); onAgentStart.run();
@@ -422,9 +509,18 @@ public final class FakeHerdr implements HerdrClient {
} }
yield mapper.readTree("{\"type\":\"ok\"}"); yield mapper.readTree("{\"type\":\"ok\"}");
} }
case "pane.get" -> mapper.readTree(""" case "pane.get" -> {
Object paneIdParam = params instanceof Map<?, ?> m ? m.get("pane_id") : null;
String paneIdKey = paneIdParam == null ? null : String.valueOf(paneIdParam);
if (paneIdKey != null && paneGoneAfterCloseIds.contains(paneIdKey)
&& closedPaneIds.contains(paneIdKey)) {
throw new HerdrException("herdr error [pane_not_found]: pane.get failed",
"pane_not_found", null);
}
yield mapper.readTree("""
{"type":"pane_info","pane":{"pane_id":"w9:pW","workspace_id":"w9", {"type":"pane_info","pane":{"pane_id":"w9:pW","workspace_id":"w9",
"tab_id":"w9:t2","agent_status":"idle"}}"""); "tab_id":"w9:t2","agent_status":"idle"}}""");
}
case "pane.list" -> noPanes case "pane.list" -> noPanes
? mapper.readTree("{\"type\":\"pane_list\",\"panes\":[]}") ? mapper.readTree("{\"type\":\"pane_list\",\"panes\":[]}")
: mapper.readTree(""" : mapper.readTree("""
@@ -458,6 +554,9 @@ public final class FakeHerdr implements HerdrClient {
throw new HerdrException("herdr error [" + code + "]: pane.close failed", throw new HerdrException("herdr error [" + code + "]: pane.close failed",
code, null); code, null);
} }
if (paneIdParam != null) {
closedPaneIds.add(String.valueOf(paneIdParam));
}
yield mapper.readTree("{\"type\":\"ok\"}"); yield mapper.readTree("{\"type\":\"ok\"}");
} }
default -> throw new HerdrException("fake has no canned response for " + method); default -> throw new HerdrException("fake has no canned response for " + method);
@@ -158,15 +158,24 @@ class LeadTabScannerTest {
return Map.of("lead: opus-5.0", "opus-5.0", "lead: gpt-sol-5.6", "gpt-sol-5.6"); return Map.of("lead: opus-5.0", "opus-5.0", "lead: gpt-sol-5.6", "gpt-sol-5.6");
} }
/** {@code twoLeads()}'s own space — every lead-label fixture below lives here unless noted. */
private static final String MAIN_SPACE = "main";
/** Wraps a flat label → name map under one space, the shape {@link LeadTabScanner} now takes. */
private static Map<String, Map<String, String>> inSpace(String space, Map<String, String> labelToName) {
return Map.of(space, labelToName);
}
private LeadTabScanner scanner(TopologyHerdr herdr, Map<String, String> tabToName, private LeadTabScanner scanner(TopologyHerdr herdr, Map<String, String> tabToName,
AtomicLong clock) { AtomicLong clock) {
return new LeadTabScanner(herdr, tabToName, Set.of("fleetd-workers"), TTL, clock::get); return new LeadTabScanner(herdr, inSpace(MAIN_SPACE, tabToName), Set.of("fleetd-workers"),
TTL, clock::get);
} }
private LeadTabScanner scannerWithCollaborators(TopologyHerdr herdr, Map<String, String> tabToName, private LeadTabScanner scannerWithCollaborators(TopologyHerdr herdr, Map<String, String> tabToName,
Map<String, String> collaboratorTabToName, Map<String, String> collaboratorTabToName,
AtomicLong clock) { AtomicLong clock) {
return new LeadTabScanner(herdr, tabToName, collaboratorTabToName, return new LeadTabScanner(herdr, inSpace(MAIN_SPACE, tabToName), collaboratorTabToName,
Set.of("fleetd-workers"), TTL, clock::get); Set.of("fleetd-workers"), TTL, clock::get);
} }
@@ -237,14 +246,61 @@ class LeadTabScannerTest {
.tab("w1:t2", "w1", "worker: gx10 #1") .tab("w1:t2", "w1", "worker: gx10 #1")
.pane("w1:p1", "w1:t1", "term_opus") .pane("w1:p1", "w1:t1", "term_opus")
.pane("w1:p2", "w1:t2", "term_worker"); .pane("w1:p2", "w1:t2", "term_worker");
LeadTabScanner s = new LeadTabScanner(herdr, Map.of("lead: opus-5.0", "opus-5.0"), LeadTabScanner s = new LeadTabScanner(herdr,
Set.of(), TTL, new AtomicLong()::get); inSpace("fleet", Map.of("lead: opus-5.0", "opus-5.0")), Set.of(), TTL,
new AtomicLong()::get);
assertEquals("opus-5.0", s.get().get("term_opus"), assertEquals("opus-5.0", s.get().get("term_opus"),
"a lead sharing the members' workspace is still discovered — the label, not the " "a lead sharing the members' workspace is still discovered — the label, not the "
+ "workspace, is what matches it"); + "workspace, is what matches it");
} }
// ── fleetd #770: space is the uniqueness boundary, not the label alone ──────────────────────
/**
* Two leads can share the exact same label (the fixed {@code lead} tab label) as long as they
* sit in different spaces — each tab resolves to its own space's lead, never the other one's.
*/
@Test
void aTabLabelledLeadResolvesToItsOwnSpacesLeadNotTheOtherSpaces() {
TopologyHerdr herdr = new TopologyHerdr()
.workspace("wa", "space-a")
.workspace("wb", "space-b")
.tab("wa:t1", "wa", "lead")
.tab("wb:t1", "wb", "lead")
.pane("wa:p1", "wa:t1", "term_a")
.pane("wb:p1", "wb:t1", "term_b");
Map<String, Map<String, String>> leadLabelsBySpace = Map.of(
"space-a", Map.of("lead", "alpha"),
"space-b", Map.of("lead", "beta"));
LeadTabScanner s = new LeadTabScanner(herdr, leadLabelsBySpace, Set.of(), TTL,
new AtomicLong()::get);
Map<String, String> leads = s.get();
assertEquals("alpha", leads.get("term_a"), "space-a's tab must resolve to space-a's lead");
assertEquals("beta", leads.get("term_b"), "space-b's tab must resolve to space-b's lead");
}
/**
* A lead's deprecated legacy {@code tab:} label is still matched, but only within that lead's
* own configured space — exactly the shape {@code FleetdAssembly} builds via {@code
* Leader.acceptedLabels()}.
*/
@Test
void aLegacyTabLabelStillResolvesWithinItsOwnSpace() {
TopologyHerdr herdr = new TopologyHerdr()
.workspace("w1", "fleet")
.tab("w1:t1", "w1", "lead: opus")
.pane("w1:p1", "w1:t1", "term_opus");
Map<String, Map<String, String>> leadLabelsBySpace =
Map.of("fleet", Map.of("lead", "opus", "lead: opus", "opus"));
LeadTabScanner s = new LeadTabScanner(herdr, leadLabelsBySpace, Set.of(), TTL,
new AtomicLong()::get);
assertEquals("opus", s.get().get("term_opus"),
"the deprecated tab label must still resolve this lead within its own space");
}
@Test @Test
void aLabelWithNoConfiguredEntryIsIgnored() { void aLabelWithNoConfiguredEntryIsIgnored() {
TopologyHerdr herdr = new TopologyHerdr().workspace("w1", "main") TopologyHerdr herdr = new TopologyHerdr().workspace("w1", "main")
@@ -0,0 +1,132 @@
package dev.ltms.fleet.herdr;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
/** Reading a Claude Code input box, so a paste-and-submit delivery never submits the operator's draft. */
class PromptBoxTest {
private static final String EMPTY = FakeHerdr.IDLE_PROMPT_CARET;
private static final String DRAFTED = FakeHerdr.DRAFTED_PROMPT_CARET;
// --- pure classification -------------------------------------------------
@Test
void anEmptyCaretLineIsAnEmptyBox() {
assertEquals(new PromptBox.Reading(PromptBox.State.EMPTY, 0), PromptBox.classify(EMPTY));
}
@Test
void aCaretLineHoldingTextIsADraftAndCountsItsCharacters() {
PromptBox.Reading reading = PromptBox.classify(DRAFTED);
assertEquals(PromptBox.State.DRAFT, reading.state());
assertEquals("yes,sendittolead:opus".length(), reading.characters(),
"padding does not count — only what the operator typed");
}
@Test
void aBorderedBoxIsReadToo() {
assertEquals(PromptBox.State.EMPTY, PromptBox.classify(FakeHerdr.IDLE_PROMPT_BOX).state(),
"an older TUI draws a bordered box, and its panes must still be readable");
assertEquals(PromptBox.State.DRAFT, PromptBox.classify(FakeHerdr.DRAFTED_PROMPT_BOX).state());
}
@Test
void aSingleTypedCharacterIsADraft() {
assertEquals(PromptBox.State.DRAFT, PromptBox.classify("❯ f").state());
assertEquals(PromptBox.State.DRAFT, PromptBox.classify("│ > f │").state());
}
@Test
void aCursorBlockInAnOtherwiseEmptyBoxIsEmpty() {
assertEquals(PromptBox.State.EMPTY, PromptBox.classify("❯ █").state(),
"a terminal capture may leave the cursor cell in an empty box");
assertEquals(PromptBox.State.EMPTY, PromptBox.classify("│ > █ │").state());
}
@Test
void theBoxLineIsReadToItsEndWhateverFollowsIt() {
assertEquals(PromptBox.State.DRAFT, PromptBox.classify("❯ half a line\n ⏵⏵ auto mode on").state());
assertEquals(PromptBox.State.EMPTY, PromptBox.classify("❯\n ⏵⏵ auto mode on").state());
}
@Test
void theLastBoxLineOnThePaneIsTheLiveOne() {
assertEquals(PromptBox.State.DRAFT,
PromptBox.classify("❯ an earlier prompt\n⏺ its answer\n❯ typing now").state(),
"the detection region carries scrollback, so earlier prompts sit above the live box");
assertEquals(PromptBox.State.EMPTY,
PromptBox.classify("❯ an earlier prompt\n⏺ its answer\n❯").state());
}
@Test
void aMarkerPartWayAlongALineIsNotABox() {
assertEquals(PromptBox.State.UNREADABLE, PromptBox.classify("⏺ type ❯ to get a prompt").state(),
"a caret the operator quoted is transcript text, not an input box");
assertEquals(PromptBox.State.EMPTY, PromptBox.classify("⏺ type ❯ to get a prompt\n❯").state(),
"and it must not shadow the real box further down");
}
@Test
void aPaneWithNoBoxIsUnreadable() {
assertEquals(PromptBox.State.UNREADABLE, PromptBox.classify("garbled ansi noise").state());
assertEquals(PromptBox.State.UNREADABLE, PromptBox.classify("").state());
assertEquals(PromptBox.State.UNREADABLE, PromptBox.classify(null).state());
}
@Test
void aGeneratingTurnIsUnreadableEvenWithAnEmptyBox() {
assertEquals(PromptBox.State.UNREADABLE,
PromptBox.classify(EMPTY + "\n ✳ Thinking… (12s · esc to interrupt)").state());
}
@Test
void aGeneratingMarkerInScrollbackAboveTheBoxDoesNotMakeThePaneUnreadable() {
assertEquals(PromptBox.State.EMPTY,
PromptBox.classify(" ✳ Thinking… (12s · esc to interrupt)\n⏺ done\n" + EMPTY).state(),
"that marker survives in scrollback, and holding on it would hold every delivery forever");
}
// --- the gate ------------------------------------------------------------
@Test
void anEmptyBoxClearsTheGateAndReadsTheDetectionRegion() {
FakeHerdr herdr = new FakeHerdr().detectionText(EMPTY);
assertTrue(new PromptBox(new AgentControl(herdr)).clearToSubmit("term_a"));
@SuppressWarnings("unchecked")
var params = (java.util.Map<String, Object>) herdr.lastCall("agent.read").params();
assertEquals("detection", params.get("source"),
"the input box is drawn in the detection region, not in transcript scrollback");
}
@Test
void aDraftedBoxHoldsTheGate() {
assertFalse(new PromptBox(new AgentControl(new FakeHerdr().detectionText(DRAFTED))).clearToSubmit("term_a"));
}
@Test
void anUnreadablePaneHoldsTheGate() {
assertFalse(new PromptBox(new AgentControl(new FakeHerdr().detectionText("garbled"))).clearToSubmit("term_a"));
}
@Test
void aFailedReadHoldsTheGate() {
assertFalse(new PromptBox(new AgentControl(new FakeHerdr().healthy(false))).clearToSubmit("term_a"),
"a pane this cannot read must never be pasted into");
}
@Test
void theGateClearsAgainOnceTheBoxEmpties() {
FakeHerdr herdr = new FakeHerdr().detectionText(DRAFTED);
PromptBox box = new PromptBox(new AgentControl(herdr));
assertFalse(box.clearToSubmit("term_a"));
herdr.detectionText(EMPTY);
assertTrue(box.clearToSubmit("term_a"));
}
}
@@ -89,6 +89,11 @@ class BackendOutageFlowTest {
return MAPPER.createObjectNode().set("agent", MAPPER.createObjectNode() return MAPPER.createObjectNode().set("agent", MAPPER.createObjectNode()
.put("terminal_id", "term_primary").put("agent_status", "idle")); .put("terminal_id", "term_primary").put("agent_status", "idle"));
} }
if ("agent.read".equals(method)) {
// The lead-nudge paths read the input box before pasting into it.
return MAPPER.createObjectNode().set("read",
MAPPER.createObjectNode().put("text", FakeHerdr.IDLE_PROMPT_CARET));
}
if ("agent.prompt".equals(method)) { if ("agent.prompt".equals(method)) {
prompts.add(params); prompts.add(params);
sendLatch.countDown(); sendLatch.countDown();
@@ -59,6 +59,17 @@ class InjectorTest {
assertEquals(List.of("hello"), sent()); assertEquals(List.of("hello"), sent());
} }
@Test
void deliveringToAMemberReadsNoPane() {
// No human types into a spawned member's pane, so its delivery path must not pay for a
// prompt-box read the way a lead's nudge paths do.
injector.enqueue(T, "task", TestTurnTokens.inert(T));
injector.onStatus(T, AgentStatus.IDLE);
assertEquals(List.of("task"), sent());
assertFalse(herdr.called("agent.read"), "a member delivery must not read its pane");
}
@Test @Test
void holdsDeliveryUntilTheWorkerIsAvailable() { void holdsDeliveryUntilTheWorkerIsAvailable() {
// CB-113: idle alone is not enough — hold until the worker's MCP is connected (ready). // CB-113: idle alone is not enough — hold until the worker's MCP is connected (ready).
@@ -114,13 +114,13 @@ class LeadLauncherTest {
"name is lead-<name>-<nonce>-<seq>: " + startedName(herdr)); "name is lead-<name>-<nonce>-<seq>: " + startedName(herdr));
} }
/** The tab is labelled with the configured `tab:` so the scanner finds the lead on the next resolve. */ /** The tab is labelled with the fixed lead tab label so the scanner finds the lead on the next resolve. */
@Test @Test
void labelsTheTabWithTheConfiguredTabValue() { void labelsTheTabWithTheFixedLeadTabLabel() {
FakeHerdr herdr = new FakeHerdr(); FakeHerdr herdr = new FakeHerdr();
launcher(herdr, configWith(lead("opus", "lead: opus", 1))).ensureLeads(); launcher(herdr, configWith(lead("opus", "lead: opus", 1))).ensureLeads();
assertEquals("lead: opus", assertEquals("lead",
((Map<?, ?>) herdr.lastCall("tab.rename").params()).get("label")); ((Map<?, ?>) herdr.lastCall("tab.rename").params()).get("label"));
} }
@@ -148,6 +148,25 @@ class LeadLauncherTest {
assertFalse(herdr.called("tab.close"), "a labelled tab WITH a live agent must never be closed"); assertFalse(herdr.called("tab.close"), "a labelled tab WITH a live agent must never be closed");
} }
/**
* The tab a live lead actually sits in still carries its deprecated legacy {@code tab:} label,
* not the fixed {@code lead} tab label a freshly auto-launched instance would get. Counting must
* still recognise it as the live lead via {@link FleetConfig.Leader#acceptedLabels()}, or a
* daemon restart would read it as missing and launch a second orchestrator next to the first.
*/
@Test
void aLiveLeadInALegacyLabelledTabIsCountedSoNothingIsLaunched() {
FakeHerdr herdr = new FakeHerdr()
.withWorkspace("wL", "fleet")
.withTab("wL", "wL:t1", "lead: opus")
.withAgent("lead-opus", "term_lead", "wL:p1", "wL:t1");
assertEquals(0, launcher(herdr, configWith(lead("opus", "lead: opus", 1))).ensureLeads(),
"the legacy-labelled live lead must be counted — nothing may be launched");
assertFalse(herdr.called("agent.start"),
"a tab label fixed to a constant must not blind the count to a legacy-labelled lead");
}
/** /**
* The reason liveness is not "does the label exist". A tab left labelled by a session that has * The reason liveness is not "does the label exist". A tab left labelled by a session that has
* since died must not block the relaunch, or one crash disables auto-launch permanently. * since died must not block the relaunch, or one crash disables auto-launch permanently.
@@ -263,7 +282,7 @@ class LeadLauncherTest {
assertFalse(herdr.called("tab.close"), "a tab running an agent again must never be closed"); assertFalse(herdr.called("tab.close"), "a tab running an agent again must never be closed");
assertFalse(herdr.called("agent.start"), "the lead is live again — nothing to relaunch"); assertFalse(herdr.called("agent.start"), "the lead is live again — nothing to relaunch");
assertEquals("wL:t1", ((Map<?, ?>) herdr.lastCall("tab.rename").params()).get("tab_id")); assertEquals("wL:t1", ((Map<?, ?>) herdr.lastCall("tab.rename").params()).get("tab_id"));
assertEquals("lead: opus", ((Map<?, ?>) herdr.lastCall("tab.rename").params()).get("label"), assertEquals("lead", ((Map<?, ?>) herdr.lastCall("tab.rename").params()).get("label"),
"the pending-close flag must be cleared once the tab is confirmed live again"); "the pending-close flag must be cleared once the tab is confirmed live again");
} }
@@ -285,7 +304,7 @@ class LeadLauncherTest {
@Test @Test
void aHandOpenedLeadWithTheConfiguredTabLabelCountsAsLive() { void aHandOpenedLeadWithTheConfiguredTabLabelCountsAsLive() {
FakeHerdr herdr = new FakeHerdr() FakeHerdr herdr = new FakeHerdr()
.withWorkspace("wX", "main") .withWorkspace("wX", "fleet")
.withTab("wX", "wX:t1", "lead: opus") .withTab("wX", "wX:t1", "lead: opus")
.withAgent("hand-opened", "term_hand", "wX:p1", "wX:t1"); .withAgent("hand-opened", "term_hand", "wX:p1", "wX:t1");
@@ -571,4 +590,137 @@ class LeadLauncherTest {
assertTrue(warn.contains("opus"), "names the profile: " + warn); assertTrue(warn.contains("opus"), "names the profile: " + warn);
assertTrue(warn.contains("x".repeat(60)), "names the culprit argument: " + warn); assertTrue(warn.contains("x".repeat(60)), "names the culprit argument: " + warn);
} }
// ── fleetd #726 unit 1: the single-lead relaunch seam ─────────────────────────────────────
/**
* The returned agent's {@code terminalId()}/{@code paneId()} are the ones the fake
* {@code AgentControl} actually started — not a coincidental field left over from the caller.
* {@code paneId()} echoes the exact {@code pane_id} the launch's own {@code agent.start} call
* carried (protocol 19: the agent starts into the pane it is asked to), and {@code
* terminalId()} is herdr's own generated id, which the fake always shapes as {@code
* term_new_<n>}.
*/
@Test
void relaunchReturnsTheStartedAgent() {
FakeHerdr herdr = new FakeHerdr();
dev.ltms.fleet.herdr.Agent started =
launcher(herdr, configWith(lead("opus", "lead: opus", 1))).relaunch("opus");
assertNotNull(started, "a launchable, configured lead must start");
Object startedPaneIdParam = ((Map<?, ?>) herdr.lastCall("agent.start").params()).get("pane_id");
assertEquals(startedPaneIdParam, started.paneId(),
"paneId() must be the pane the agent.start call actually targeted");
assertTrue(started.terminalId() != null && started.terminalId().startsWith("term_new_"),
"terminalId() must be herdr's own generated id: " + started.terminalId());
}
/** The new tab is labelled with the fixed lead tab label, and AFTER the start. */
@Test
void relaunchLabelsTheNewTabAfterStarting() {
FakeHerdr herdr = new FakeHerdr();
dev.ltms.fleet.herdr.Agent started =
launcher(herdr, configWith(lead("opus", "lead: opus", 1))).relaunch("opus");
assertNotNull(started);
assertEquals("lead", ((Map<?, ?>) herdr.lastCall("tab.rename").params()).get("label"));
int startIndex = indexOfLastCall(herdr, "agent.start");
int renameIndex = indexOfLastCall(herdr, "tab.rename");
assertTrue(renameIndex > startIndex,
"the tab must be renamed AFTER the start succeeds, not before: start=" + startIndex
+ " rename=" + renameIndex);
}
private static int indexOfLastCall(FakeHerdr herdr, String method) {
int idx = -1;
List<FakeHerdr.Call> calls = herdr.calls;
for (int i = 0; i < calls.size(); i++) {
if (calls.get(i).method().equals(method)) {
idx = i;
}
}
return idx;
}
@Test
void relaunchOfAnUnknownLeadNameReturnsNullAndStartsNothing() {
FakeHerdr herdr = new FakeHerdr();
dev.ltms.fleet.herdr.Agent started =
launcher(herdr, configWith(lead("opus", "lead: opus", 1))).relaunch("not-declared");
assertNull(started);
assertFalse(herdr.called("agent.start"));
assertFalse(herdr.called("workspace.create"));
assertFalse(herdr.called("tab.create"));
}
@Test
void relaunchOfARecogniseOnlyLeadReturnsNullAndStartsNothing() {
FakeHerdr herdr = new FakeHerdr();
dev.ltms.fleet.herdr.Agent started =
launcher(herdr, configWith(lead(null, "lead: dead", 1))).relaunch("opus");
assertNull(started);
assertFalse(herdr.called("agent.start"));
}
@Test
void relaunchWithAnUnconfiguredProfileReturnsNullAndStartsNothing() {
FakeHerdr herdr = new FakeHerdr();
dev.ltms.fleet.herdr.Agent started =
launcher(herdr, configWith(lead("nope", "lead: opus", 1))).relaunch("opus");
assertNull(started);
assertFalse(herdr.called("agent.start"));
}
/**
* The outer retry {@link LeadLauncher#relaunch(String)} owns, separate from {@code
* ResilientAgentLaunch}'s internal {@code agent_name_taken} retry: a failed attempt must not
* be the end of the whole relaunch. Each of the first two attempts exhausts {@code
* ResilientAgentLaunch.NAME_RETRIES} name attempts (every one of them rejected), so each
* attempt's own tab is created and then closed; the third attempt's first name is free.
*/
@Test
void relaunchRetriesTheWholeAttemptAndSucceedsOnTheThird() {
FakeHerdr herdr = new FakeHerdr()
.agentNameTakenTimes(2 * ResilientAgentLaunch.NAME_RETRIES);
dev.ltms.fleet.herdr.Agent started =
fastLauncher(herdr, configWith(lead("opus", "lead: opus", 1))).relaunch("opus");
assertNotNull(started, "the third attempt's first name is free — it must succeed");
assertEquals(3, herdr.calls.stream().filter(c -> c.method().equals("tab.create")).count(),
"one tab per attempt: three attempts");
assertEquals(2, herdr.calls.stream().filter(c -> c.method().equals("tab.close")).count(),
"the two failed attempts' tabs must be closed");
}
/**
* Every attempt fails outright (a herdr error {@code ResilientAgentLaunch} does not retry at
* all) — {@link LeadLauncher#relaunch(String)} must give up after exactly {@code
* RELAUNCH_ATTEMPTS} and must not leak any of the tabs it created along the way.
*/
@Test
void relaunchGivesUpAfterExactlyRelaunchAttemptsAndLeaksNoTab() {
FakeHerdr herdr = new FakeHerdr().agentStartFailsWith("some_other_error");
dev.ltms.fleet.herdr.Agent started =
fastLauncher(herdr, configWith(lead("opus", "lead: opus", 1))).relaunch("opus");
assertNull(started, "every attempt failed — relaunch must give up, not hang or guess");
assertEquals(LeadLauncher.RELAUNCH_ATTEMPTS,
herdr.calls.stream().filter(c -> c.method().equals("agent.start")).count(),
"exactly RELAUNCH_ATTEMPTS attempts, no more, no fewer");
long tabsCreated = herdr.calls.stream().filter(c -> c.method().equals("tab.create")).count();
long tabsClosed = herdr.calls.stream().filter(c -> c.method().equals("tab.close")).count();
assertEquals(LeadLauncher.RELAUNCH_ATTEMPTS, tabsCreated);
assertEquals(tabsCreated, tabsClosed, "every tab this method created must be closed — no leaks");
}
} }
File diff suppressed because it is too large Load Diff
@@ -17,6 +17,7 @@ import dev.ltms.fleet.metrics.Metrics;
import dev.ltms.fleet.msg.InMemoryReplyInbox; import dev.ltms.fleet.msg.InMemoryReplyInbox;
import dev.ltms.fleet.msg.MessageService; import dev.ltms.fleet.msg.MessageService;
import dev.ltms.fleet.msg.Rendezvous; import dev.ltms.fleet.msg.Rendezvous;
import dev.ltms.fleet.peer.MemberRole;
import dev.ltms.fleet.session.FakeWorktrees; import dev.ltms.fleet.session.FakeWorktrees;
import dev.ltms.fleet.session.SessionManager; import dev.ltms.fleet.session.SessionManager;
import dev.ltms.fleet.member.ClaudeCodeLauncher; import dev.ltms.fleet.member.ClaudeCodeLauncher;
@@ -269,6 +270,60 @@ class FleetMcpAuthzTest {
"a spawned member's own terminal must stay unreachable, even once the classifier is real"); "a spawned member's own terminal must stay unreachable, even once the classifier is real");
} }
// --- fleetd #743: the observer SEND matrix, over MCP's denyFor -------------------------------
private static final Principal OBSERVER = Principal.observer("term_observer", 700);
/**
* Wires one real {@link CallerResolver} that recognises a lead, a collaborator, and a live
* spawned worker, leaving "term_other_observer" classified as none of them — so the same
* wiring both denies an observer's {@code SEND} to every privileged role and grants it to
* another unclassified pane, proving the refusals are the rule and not a missing fixture.
*/
@Test
void anObserverMaySendOnlyToAnotherObserverNeverToALeadWorkerOrArchitect() {
ConnectionIdentity identity = new ConnectionIdentity(new PaneLocator(herdr), _ -> 999_999);
CallerResolver callers = CallerResolver.withLeadsAndMembers(identity, false, null,
() -> Map.of("term_lead_known", "lead-x"), new MemberRegistry(null),
t -> "term_a".equals(t) ? MemberRole.DEV : null,
() -> Map.of("term_collab_known", "ops2"));
FleetMcp m = mcp(true, callers);
assertNotNull(m.denyFor(OBSERVER, Authz.Action.SEND, "term_lead_known"),
"an observer must never reach a lead's terminal");
assertNotNull(m.denyFor(OBSERVER, Authz.Action.SEND, "term_collab_known"),
"an observer must never reach a collaborator's terminal");
assertNotNull(m.denyFor(OBSERVER, Authz.Action.SEND, "term_a"),
"an observer must never reach a live spawned member's terminal");
// CONTROL: the same wiring, the same denyFor call, a target recognised as none of the
// three privileged roles above -- this is what proves the three refusals above are the
// rule working, not a classifier that refuses every target regardless of what it is.
assertNull(m.denyFor(OBSERVER, Authz.Action.SEND, "term_other_observer"),
"an observer must reach another pane that resolves as an observer itself");
}
/**
* As {@link #anObserverMaySendOnlyToAnotherObserverNeverToALeadWorkerOrArchitect}, for a
* terminal bound to a configured architect slot but hosting no live spawned-member session --
* the case {@link CallerResolver#resolve} itself treats separately from a live worker/architect.
*/
@Test
void anObserverMayNotSendToABoundArchitectSlotEither() {
ConnectionIdentity identity = new ConnectionIdentity(new PaneLocator(herdr), _ -> 999_999);
MemberRegistry members = new MemberRegistry(new FleetConfig.Fleet(Map.of(),
Map.of("lead-designer", new FleetConfig.Slot("sonnet")), Map.of(), Map.of(), null));
assertTrue(members.bind("architect:lead-designer", "term_bound_architect"));
CallerResolver callers = CallerResolver.withLeadsAndMembers(identity, false, null, Map::of,
members, t -> null, Map::of);
FleetMcp m = mcp(true, callers);
assertNotNull(m.denyFor(OBSERVER, Authz.Action.SEND, "term_bound_architect"),
"a terminal bound to a configured architect slot must stay unreachable to an observer");
// CONTROL: the same wiring, a target the bind above never touched.
assertNull(m.denyFor(OBSERVER, Authz.Action.SEND, "term_other_observer"));
}
@Test @Test
void theLegacyConstructorLeavesTheGateOpen() { void theLegacyConstructorLeavesTheGateOpen() {
// The 22 pre-existing FleetMcpTest cases rely on no authorization being enforced. // The 22 pre-existing FleetMcpTest cases rely on no authorization being enforced.
@@ -453,6 +508,28 @@ class FleetMcpAuthzTest {
assertFalse(FleetMcp.membersVisibleTo(ANON), "authenticated as nothing must not see it either"); assertFalse(FleetMcp.membersVisibleTo(ANON), "authenticated as nothing must not see it either");
} }
// --- who may see fleet_list's panes array ----------------------------------------------------
/**
* {@link FleetMcp#panesVisibleTo} is the whole policy decision for {@code fleet_list}'s
* {@code panes} array: visible to every role that may {@code SEND} to some other pane -- the
* primary, an architect, a collaborator, and an observer (to another observer pane only, with
* its row filtered and reduced -- see {@code listFleet}) -- never a worker, never an
* anonymous caller.
*/
@Test
void onlyPrimaryArchitectCollaboratorAndObserverMaySeeThePanesArray() {
assertTrue(FleetMcp.panesVisibleTo(PRIMARY), "the primary must see the panes array");
assertTrue(FleetMcp.panesVisibleTo(ARCH_DESIGN), "an architect must see the panes array");
assertTrue(FleetMcp.panesVisibleTo(COLLABORATOR), "a collaborator must see its own peer roster");
assertFalse(FleetMcp.panesVisibleTo(WORKER_A),
"a worker holds READ but can never SEND, so it must not see the panes array");
assertTrue(FleetMcp.panesVisibleTo(Principal.observer("term_obs", 700)),
"an observer holds SEND to another observer pane, so it must see the (filtered, "
+ "reduced) panes array");
assertFalse(FleetMcp.panesVisibleTo(ANON), "authenticated as nothing must not see it either");
}
/** /**
* Same reasoning as {@link #theFleetListHandlerActuallyConsultsCoordinatorVisibleTo}: the * Same reasoning as {@link #theFleetListHandlerActuallyConsultsCoordinatorVisibleTo}: the
* predicate above can be perfectly correct while the one production call site never asks it. * predicate above can be perfectly correct while the one production call site never asks it.
@@ -504,12 +581,12 @@ class FleetMcpAuthzTest {
} }
/** /**
* {@code fleet_poll{ticket}} must thread the calling connection's own terminal into * {@code fleet_poll{ticket}} must thread the calling connection's owner key into
* {@link MessageService#poll(String, String)}, so a worker cannot read a ticket a different * {@link MessageService#poll(String, String)}, so a worker cannot read a ticket a different
* session created. * session created.
*/ */
@Test @Test
void theFleetPollHandlerActuallyThreadsCallerTerminalIntoPoll() throws Exception { void theFleetPollHandlerActuallyThreadsCallerOwnerIntoPoll() throws Exception {
String source = Files.readString(MCP_SOURCE); String source = Files.readString(MCP_SOURCE);
int start = source.indexOf("pollHandler ="); int start = source.indexOf("pollHandler =");
@@ -525,18 +602,18 @@ class FleetMcpAuthzTest {
"control failed: the scraped pollHandler block contains no poll(messages, ...) call " "control failed: the scraped pollHandler block contains no poll(messages, ...) call "
+ "at all -- the anchors have drifted, this test is not testing what it claims to"); + "at all -- the anchors have drifted, this test is not testing what it claims to");
assertTrue(handlerBlock.contains("callerTerminal(exchange)"), assertTrue(handlerBlock.contains("principal(exchange).ownerKey()"),
"the fleet_poll handler must thread callerTerminal(exchange) into poll(...), not omit " "the fleet_poll handler must thread principal(exchange).ownerKey() into poll(...), not omit "
+ "it or pass a literal null -- block: " + handlerBlock); + "it or pass a literal null -- block: " + handlerBlock);
} }
/** /**
* {@code fleet_status} must thread the calling connection's own terminal into * {@code fleet_status} must thread the calling connection's owner key into
* {@link FleetMcp#status(MessageService, String, String)}, so a caller that did not create a * {@link FleetMcp#status(MessageService, String, String)}, so a caller that did not create a
* worker's open delegation cannot read its pending question through the status handler either. * worker's open delegation cannot read its pending question through the status handler either.
*/ */
@Test @Test
void theFleetStatusHandlerActuallyThreadsCallerTerminalIntoStatus() throws Exception { void theFleetStatusHandlerActuallyThreadsCallerOwnerIntoStatus() throws Exception {
String source = Files.readString(MCP_SOURCE); String source = Files.readString(MCP_SOURCE);
int start = source.indexOf("statusHandler ="); int start = source.indexOf("statusHandler =");
@@ -552,8 +629,8 @@ class FleetMcpAuthzTest {
"control failed: the scraped statusHandler block contains no status(messages, ...) " "control failed: the scraped statusHandler block contains no status(messages, ...) "
+ "call at all -- the anchors have drifted, this test is not testing what it claims to"); + "call at all -- the anchors have drifted, this test is not testing what it claims to");
assertTrue(handlerBlock.contains("callerTerminal(exchange)"), assertTrue(handlerBlock.contains("principal(exchange).ownerKey()"),
"the fleet_status handler must thread callerTerminal(exchange) into status(...), not " "the fleet_status handler must thread principal(exchange).ownerKey() into status(...), not "
+ "omit it or pass a literal null -- block: " + handlerBlock); + "omit it or pass a literal null -- block: " + handlerBlock);
} }
@@ -11,6 +11,7 @@ import dev.ltms.fleet.herdr.FakeHerdr;
import dev.ltms.fleet.herdr.PaneLocator; import dev.ltms.fleet.herdr.PaneLocator;
import dev.ltms.fleet.herdr.WorkspaceControl; import dev.ltms.fleet.herdr.WorkspaceControl;
import dev.ltms.fleet.inject.Injector; import dev.ltms.fleet.inject.Injector;
import dev.ltms.fleet.lead.LeadLauncher;
import dev.ltms.fleet.lead.LeadRollover; import dev.ltms.fleet.lead.LeadRollover;
import dev.ltms.fleet.member.ClaudeCodeLauncher; import dev.ltms.fleet.member.ClaudeCodeLauncher;
import dev.ltms.fleet.msg.InMemoryReplyInbox; import dev.ltms.fleet.msg.InMemoryReplyInbox;
@@ -24,6 +25,8 @@ import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir; import org.junit.jupiter.api.io.TempDir;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.nio.file.Files; import java.nio.file.Files;
import java.nio.file.Path; import java.nio.file.Path;
import java.util.List; import java.util.List;
@@ -36,14 +39,14 @@ import static org.junit.jupiter.api.Assertions.*;
* fleetd #480 Unit C — the {@code fleet_handover} MCP tool, the surface that finally calls * fleetd #480 Unit C — the {@code fleet_handover} MCP tool, the surface that finally calls
* {@link LeadRollover#open}/{@link LeadRollover#confirm}/{@link LeadRollover#cancel}. * {@link LeadRollover#open}/{@link LeadRollover#confirm}/{@link LeadRollover#cancel}.
* *
* <p>Uses {@link LeadRollover}'s PUBLIC constructor (real wall clock, real 250ms settle poll, a * <p>Uses {@link LeadRollover}'s PUBLIC constructor (real wall clock, real 250ms poll, a real
* real virtual-thread continuation runner) rather than its package-private test constructor — * virtual-thread continuation runner) rather than its package-private test constructor — this
* this test lives in {@code dev.ltms.fleet.mcp}, not {@code dev.ltms.fleet.lead}, and does not * test lives in {@code dev.ltms.fleet.mcp}, not {@code dev.ltms.fleet.lead}, and does not need to
* need to control the post-{@code confirm()} continuation's timing: it only asserts the * control the post-{@code confirm()} continuation's timing: it only asserts the SYNCHRONOUS return
* SYNCHRONOUS return value of {@code open}/{@code confirm}/{@code cancel}, which is exactly what * value of {@code open}/{@code confirm}/{@code cancel}, which is exactly what {@code
* {@code FleetMcp.handover} forwards to the client. {@code turnSettleSeconds}/{@code * FleetMcp.handover} forwards to the client. {@code turnSettleSeconds}/{@code
* clearSettleSeconds} are kept at 1s so a confirmed request's background continuation (which this * relaunchReadySeconds} are kept at 1s so a confirmed request's background continuation (which
* class does not wait on or assert against) gives up quickly rather than polling for 20s on a * this class does not wait on or assert against) gives up quickly rather than polling for 20s on a
* daemon virtual thread. * daemon virtual thread.
*/ */
class FleetMcpHandoverTest { class FleetMcpHandoverTest {
@@ -53,9 +56,14 @@ class FleetMcpHandoverTest {
private static final String LEAD = "term_lead"; private static final String LEAD = "term_lead";
private static final String OTHER_LEAD = "term_other_lead"; private static final String OTHER_LEAD = "term_other_lead";
private static final String LEAD_OWNER = "leader:lead";
private final FakeHerdr herdr = new FakeHerdr(); private final FakeHerdr herdr = new FakeHerdr();
private final AgentControl agents = new AgentControl(herdr); private final AgentControl agents = new AgentControl(herdr);
/** Fed to every direct {@code FleetMcp.handover} call below — none of this class's own tests
* exercise ticket/ask ownership, so a single instance with no delegations is enough. */
private final MessageService messages = new MessageService(agents, new Injector(agents),
new Rendezvous(), new InMemoryReplyInbox());
private FleetMcp mcp; private FleetMcp mcp;
@AfterEach @AfterEach
@@ -67,10 +75,26 @@ class FleetMcpHandoverTest {
return new FleetConfig.LeadRollover(handoverPath, false, 3600, 1, 1, "read the handover file"); return new FleetConfig.LeadRollover(handoverPath, false, 3600, 1, 1, "read the handover file");
} }
private static FleetConfig minimalFleetConfig() {
try {
Path yaml = Files.createTempFile("fleet-mcp-handover-test", ".yaml");
Files.writeString(yaml, "bind:\n port: 8080\n");
return FleetConfig.load(yaml);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
private LeadRollover newRollover(String handoverPath) { private LeadRollover newRollover(String handoverPath) {
// Every handoverPath this test class uses comes from tmp.resolve(...), which is already // Every handoverPath this test class uses comes from tmp.resolve(...), which is already
// absolute, so the workspace lookup is never actually consulted — a no-op lookup is enough. // absolute, so the workspace lookup is never actually consulted — a no-op lookup is enough.
return new LeadRollover(agents, () -> cfg(handoverPath), _ -> null); // None of this class's tests reach the recognition-wait or the relaunch call, so the
// launcher's own functional correctness is irrelevant here — any constructed instance,
// backed by the same fake herdr, is enough.
WorkspaceControl spaces = new WorkspaceControl(herdr);
LeadLauncher launcher = new LeadLauncher(agents, spaces, minimalFleetConfig());
return new LeadRollover(agents, spaces, launcher, () -> cfg(handoverPath),
_ -> null, _ -> null, Map::of);
} }
/** A fully wired FleetMcp on fakes (mirrors FleetMcpAuthzTest's helper), plus a leadRollover. */ /** A fully wired FleetMcp on fakes (mirrors FleetMcpAuthzTest's helper), plus a leadRollover. */
@@ -132,16 +156,16 @@ class FleetMcpHandoverTest {
@DisplayName("with leadRollover: absent, every action returns a clean NOT_CONFIGURED refusal and never throws") @DisplayName("with leadRollover: absent, every action returns a clean NOT_CONFIGURED refusal and never throws")
void nullLeadRolloverRefusesCleanlyForEveryAction() { void nullLeadRolloverRefusesCleanlyForEveryAction() {
McpSchema.CallToolResult open = assertDoesNotThrow( McpSchema.CallToolResult open = assertDoesNotThrow(
() -> FleetMcp.handover(null, LEAD, Map.of("action", "open"))); () -> FleetMcp.handover(null, messages, LEAD, LEAD_OWNER, Map.of("action", "open")));
assertFalse(open.isError(), "a refusal is not a protocol error: " + textOf(open)); assertFalse(open.isError(), "a refusal is not a protocol error: " + textOf(open));
assertTrue(textOf(open).contains("NOT_CONFIGURED"), textOf(open)); assertTrue(textOf(open).contains("NOT_CONFIGURED"), textOf(open));
McpSchema.CallToolResult confirm = assertDoesNotThrow(() -> FleetMcp.handover(null, LEAD, McpSchema.CallToolResult confirm = assertDoesNotThrow(() -> FleetMcp.handover(null, messages, LEAD, LEAD_OWNER,
Map.of("action", "confirm", "token", "whatever"))); Map.of("action", "confirm", "token", "whatever")));
assertFalse(confirm.isError()); assertFalse(confirm.isError());
assertTrue(textOf(confirm).contains("NOT_CONFIGURED"), textOf(confirm)); assertTrue(textOf(confirm).contains("NOT_CONFIGURED"), textOf(confirm));
McpSchema.CallToolResult cancel = assertDoesNotThrow(() -> FleetMcp.handover(null, LEAD, McpSchema.CallToolResult cancel = assertDoesNotThrow(() -> FleetMcp.handover(null, messages, LEAD, LEAD_OWNER,
Map.of("action", "cancel", "token", "whatever"))); Map.of("action", "cancel", "token", "whatever")));
assertFalse(cancel.isError()); assertFalse(cancel.isError());
assertTrue(textOf(cancel).contains("NOT_CONFIGURED"), textOf(cancel)); assertTrue(textOf(cancel).contains("NOT_CONFIGURED"), textOf(cancel));
@@ -150,11 +174,11 @@ class FleetMcpHandoverTest {
@Test @Test
@DisplayName("a blank/unknown action is a clean tool error, never an exception") @DisplayName("a blank/unknown action is a clean tool error, never an exception")
void unknownActionIsACleanError() { void unknownActionIsACleanError() {
McpSchema.CallToolResult missing = assertDoesNotThrow(() -> FleetMcp.handover(null, LEAD, Map.of())); McpSchema.CallToolResult missing = assertDoesNotThrow(() -> FleetMcp.handover(null, messages, LEAD, LEAD_OWNER, Map.of()));
assertTrue(missing.isError()); assertTrue(missing.isError());
McpSchema.CallToolResult bogus = assertDoesNotThrow( McpSchema.CallToolResult bogus = assertDoesNotThrow(
() -> FleetMcp.handover(null, LEAD, Map.of("action", "bogus"))); () -> FleetMcp.handover(null, messages, LEAD, LEAD_OWNER, Map.of("action", "bogus")));
assertTrue(bogus.isError()); assertTrue(bogus.isError());
} }
@@ -210,7 +234,7 @@ class FleetMcpHandoverTest {
Files.writeString(handover, "not written yet"); Files.writeString(handover, "not written yet");
LeadRollover rollover = newRollover(handover.toString()); LeadRollover rollover = newRollover(handover.toString());
McpSchema.CallToolResult openResult = FleetMcp.handover(rollover, LEAD, Map.of("action", "open")); McpSchema.CallToolResult openResult = FleetMcp.handover(rollover, messages, LEAD, LEAD_OWNER, Map.of("action", "open"));
assertFalse(openResult.isError(), textOf(openResult)); assertFalse(openResult.isError(), textOf(openResult));
String token = extractToken(textOf(openResult)); String token = extractToken(textOf(openResult));
@@ -219,13 +243,13 @@ class FleetMcpHandoverTest {
Thread.sleep(50); Thread.sleep(50);
Files.writeString(handover, "the real handover content"); Files.writeString(handover, "the real handover content");
McpSchema.CallToolResult wrongCaller = FleetMcp.handover(rollover, OTHER_LEAD, McpSchema.CallToolResult wrongCaller = FleetMcp.handover(rollover, messages, OTHER_LEAD, "leader:other-lead",
Map.of("action", "confirm", "token", token)); Map.of("action", "confirm", "token", token));
assertFalse(wrongCaller.isError(), "a refusal is a legitimate outcome, not a protocol error"); assertFalse(wrongCaller.isError(), "a refusal is a legitimate outcome, not a protocol error");
assertTrue(textOf(wrongCaller).contains("NOT_YOUR_ROLLOVER"), assertTrue(textOf(wrongCaller).contains("NOT_YOUR_ROLLOVER"),
"a different lead terminal confirming must surface NOT_YOUR_ROLLOVER: " + textOf(wrongCaller)); "a different lead terminal confirming must surface NOT_YOUR_ROLLOVER: " + textOf(wrongCaller));
McpSchema.CallToolResult confirmed = FleetMcp.handover(rollover, LEAD, McpSchema.CallToolResult confirmed = FleetMcp.handover(rollover, messages, LEAD, LEAD_OWNER,
Map.of("action", "confirm", "token", token)); Map.of("action", "confirm", "token", token));
assertFalse(confirmed.isError(), textOf(confirmed)); assertFalse(confirmed.isError(), textOf(confirmed));
assertTrue(textOf(confirmed).contains("\"accepted\":true"), assertTrue(textOf(confirmed).contains("\"accepted\":true"),
@@ -239,7 +263,7 @@ class FleetMcpHandoverTest {
void cancelUnknownTokenIsCleanNotAFailure() { void cancelUnknownTokenIsCleanNotAFailure() {
LeadRollover rollover = newRollover(tmp.resolve("h.md").toString()); LeadRollover rollover = newRollover(tmp.resolve("h.md").toString());
McpSchema.CallToolResult r = FleetMcp.handover(rollover, LEAD, McpSchema.CallToolResult r = FleetMcp.handover(rollover, messages, LEAD, LEAD_OWNER,
Map.of("action", "cancel", "token", "does-not-exist")); Map.of("action", "cancel", "token", "does-not-exist"));
assertFalse(r.isError()); assertFalse(r.isError());
assertTrue(textOf(r).contains("\"cancelled\":false"), textOf(r)); assertTrue(textOf(r).contains("\"cancelled\":false"), textOf(r));
@@ -250,9 +274,9 @@ class FleetMcpHandoverTest {
@DisplayName("cancel on a token actually opened reports cancelled:true") @DisplayName("cancel on a token actually opened reports cancelled:true")
void cancelKnownTokenSucceeds() { void cancelKnownTokenSucceeds() {
LeadRollover rollover = newRollover(tmp.resolve("h.md").toString()); LeadRollover rollover = newRollover(tmp.resolve("h.md").toString());
String token = extractToken(textOf(FleetMcp.handover(rollover, LEAD, Map.of("action", "open")))); String token = extractToken(textOf(FleetMcp.handover(rollover, messages, LEAD, LEAD_OWNER, Map.of("action", "open"))));
McpSchema.CallToolResult r = FleetMcp.handover(rollover, LEAD, McpSchema.CallToolResult r = FleetMcp.handover(rollover, messages, LEAD, LEAD_OWNER,
Map.of("action", "cancel", "token", token)); Map.of("action", "cancel", "token", token));
assertFalse(r.isError()); assertFalse(r.isError());
assertTrue(textOf(r).contains("\"cancelled\":true"), textOf(r)); assertTrue(textOf(r).contains("\"cancelled\":true"), textOf(r));
@@ -263,7 +287,7 @@ class FleetMcpHandoverTest {
@Test @Test
@DisplayName("status on a null LeadRollover is a clean NOT_CONFIGURED refusal, never a throw") @DisplayName("status on a null LeadRollover is a clean NOT_CONFIGURED refusal, never a throw")
void statusWithNullLeadRolloverRefusesCleanly() { void statusWithNullLeadRolloverRefusesCleanly() {
McpSchema.CallToolResult r = assertDoesNotThrow(() -> FleetMcp.handover(null, LEAD, McpSchema.CallToolResult r = assertDoesNotThrow(() -> FleetMcp.handover(null, messages, LEAD, LEAD_OWNER,
Map.of("action", "status", "token", "whatever"))); Map.of("action", "status", "token", "whatever")));
assertFalse(r.isError()); assertFalse(r.isError());
assertTrue(textOf(r).contains("NOT_CONFIGURED"), textOf(r)); assertTrue(textOf(r).contains("NOT_CONFIGURED"), textOf(r));
@@ -274,7 +298,7 @@ class FleetMcpHandoverTest {
void statusOnUnknownTokenReportsUnknown() { void statusOnUnknownTokenReportsUnknown() {
LeadRollover rollover = newRollover(tmp.resolve("h.md").toString()); LeadRollover rollover = newRollover(tmp.resolve("h.md").toString());
McpSchema.CallToolResult r = FleetMcp.handover(rollover, LEAD, McpSchema.CallToolResult r = FleetMcp.handover(rollover, messages, LEAD, LEAD_OWNER,
Map.of("action", "status", "token", "does-not-exist")); Map.of("action", "status", "token", "does-not-exist"));
assertFalse(r.isError()); assertFalse(r.isError());
assertTrue(textOf(r).contains("\"state\":\"UNKNOWN\""), textOf(r)); assertTrue(textOf(r).contains("\"state\":\"UNKNOWN\""), textOf(r));
@@ -284,9 +308,9 @@ class FleetMcpHandoverTest {
@DisplayName("status on a token that is still pending (opened, not confirmed) reports PENDING") @DisplayName("status on a token that is still pending (opened, not confirmed) reports PENDING")
void statusOnPendingTokenReportsPending() { void statusOnPendingTokenReportsPending() {
LeadRollover rollover = newRollover(tmp.resolve("h.md").toString()); LeadRollover rollover = newRollover(tmp.resolve("h.md").toString());
String token = extractToken(textOf(FleetMcp.handover(rollover, LEAD, Map.of("action", "open")))); String token = extractToken(textOf(FleetMcp.handover(rollover, messages, LEAD, LEAD_OWNER, Map.of("action", "open"))));
McpSchema.CallToolResult r = FleetMcp.handover(rollover, LEAD, McpSchema.CallToolResult r = FleetMcp.handover(rollover, messages, LEAD, LEAD_OWNER,
Map.of("action", "status", "token", token)); Map.of("action", "status", "token", token));
assertFalse(r.isError()); assertFalse(r.isError());
assertTrue(textOf(r).contains("\"state\":\"PENDING\""), textOf(r)); assertTrue(textOf(r).contains("\"state\":\"PENDING\""), textOf(r));
@@ -304,5 +328,40 @@ class FleetMcpHandoverTest {
"the tool's own description must advertise the 'status' action: " + tool.description()); "the tool's own description must advertise the 'status' action: " + tool.description());
} }
// --- unit 5: open() reports outstanding tickets and open asks ------------------------------
@Test
@DisplayName("open() keeps token/handoverPath/requestedAtMillis and reports empty outstanding collections when the caller has nothing")
void openReportsEmptyOutstandingCollectionsWhenCallerHasNothing() {
LeadRollover rollover = newRollover(tmp.resolve("h.md").toString());
McpSchema.CallToolResult r = FleetMcp.handover(rollover, messages, LEAD, LEAD_OWNER,
Map.of("action", "open"));
assertFalse(r.isError(), textOf(r));
String json = textOf(r);
assertTrue(json.contains("\"token\":"), json);
assertTrue(json.contains("\"handoverPath\":"), json);
assertTrue(json.contains("\"requestedAtMillis\":"), json);
assertTrue(json.contains("\"outstandingTickets\":[]"),
"a caller with nothing gets an empty array, not an absent key: " + json);
assertTrue(json.contains("\"openAsks\":[]"),
"a caller with nothing gets an empty array, not an absent key: " + json);
}
@Test
@DisplayName("open() reports an owned pending ticket with its phase and target")
void openReportsAnOwnedPendingTicketWithItsPhase() {
LeadRollover rollover = newRollover(tmp.resolve("h.md").toString());
String ticket = messages.sendAsync("term_worker", "a task", null, Principal.leader("lead", LEAD, 1));
McpSchema.CallToolResult r = FleetMcp.handover(rollover, messages, LEAD, LEAD_OWNER,
Map.of("action", "open"));
assertFalse(r.isError(), textOf(r));
String json = textOf(r);
assertTrue(json.contains("\"ticket\":\"" + ticket + "\""), json);
assertTrue(json.contains("\"phase\":\"PENDING\""), json);
assertTrue(json.contains("\"target\":\"term_worker\""), json);
}
// --- acceptance 7 (wiring) is covered by FleetdLeadRolloverWiringTest, unchanged ----------- // --- acceptance 7 (wiring) is covered by FleetdLeadRolloverWiringTest, unchanged -----------
} }
@@ -0,0 +1,142 @@
package dev.ltms.fleet.mcp;
import dev.ltms.fleet.auth.CallerResolver;
import dev.ltms.fleet.auth.MemberRegistry;
import dev.ltms.fleet.config.FleetConfig;
import dev.ltms.fleet.guard.SubscriptionGuard;
import dev.ltms.fleet.herdr.AgentControl;
import dev.ltms.fleet.herdr.AgentStatus;
import dev.ltms.fleet.herdr.FakeHerdr;
import dev.ltms.fleet.herdr.PaneLocator;
import dev.ltms.fleet.herdr.WorkspaceControl;
import dev.ltms.fleet.inject.Injector;
import dev.ltms.fleet.member.ClaudeCodeLauncher;
import dev.ltms.fleet.msg.InMemoryReplyInbox;
import dev.ltms.fleet.msg.MessageService;
import dev.ltms.fleet.msg.Rendezvous;
import dev.ltms.fleet.session.FakeWorktrees;
import dev.ltms.fleet.session.SessionManager;
import io.modelcontextprotocol.client.McpClient;
import io.modelcontextprotocol.client.McpSyncClient;
import io.modelcontextprotocol.client.transport.HttpClientStreamableHttpTransport;
import io.modelcontextprotocol.spec.McpClientTransport;
import io.modelcontextprotocol.spec.McpSchema;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.ServerConnector;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.eclipse.jetty.servlet.ServletHolder;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import java.util.List;
import java.util.Map;
import java.util.Set;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* fleetd #743, driven end to end: a real MCP client over a real HTTP transport, resolved by the
* real {@link CallerResolver} to {@link dev.ltms.fleet.auth.Role#OBSERVER}, sending to another
* unclassified pane. {@link FleetMcpAuthzTest} already proves {@code denyFor} grants this case and
* that the handler calls {@code attributeIfObserver}; this test is the one path that also proves
* the grant is not dead at a second gate (CB-548's two-gate trap) by driving the real
* {@link Injector} to the point of its real herdr call, and reads the exact text the receiving
* pane would see.
*/
class FleetMcpObserverSendDeliveryTest {
private static final String TARGET = "term_other_observer";
private final FakeHerdr herdr = new FakeHerdr();
private final AgentControl agents = new AgentControl(herdr);
private final Injector injector = new Injector(agents);
private final Rendezvous rendezvous = new Rendezvous();
private final MessageService messages = new MessageService(agents, injector, rendezvous,
new InMemoryReplyInbox());
private FleetMcp mcp;
private Server server;
@AfterEach
void tearDown() throws Exception {
if (server != null) {
server.stop();
}
if (mcp != null) {
mcp.close();
}
}
@Test
void anObserversSendIsAttributedAndReachesTheRealInjector() throws Exception {
FleetConfig.Profile cfg = new FleetConfig.Profile(
"ltms-local", "http://gx00.gw:8000", "coder", null, "FLEETD_WORKER_TOKEN", null,
"tab", "fleetd-workers", "worker: {profile} #{n}", null, null, null);
ClaudeCodeLauncher workers = new ClaudeCodeLauncher(agents, new WorkspaceControl(herdr),
new SubscriptionGuard(Set.of("gx00.gw")), Map.of(cfg.profile(), cfg), cfg.profile(),
_ -> "tok");
SessionManager sessions = new SessionManager(workers, new FakeWorktrees());
// The fake's pane list carries a second pane, "term_shell", whose shell pid is 9001 and
// which hosts no agent -- a herdr-owned pane recognised as no lead, architect, collaborator
// or live spawned member, so the real resolver lands it on the observer floor.
ConnectionIdentity identity = new ConnectionIdentity(new PaneLocator(herdr), _ -> 9001L);
CallerResolver callers = CallerResolver.withLeadsAndMembers(identity, false, null,
Map::of, new MemberRegistry(null));
mcp = new FleetMcp(messages, workers, sessions, identity, sessions.asPresence(),
new PrimaryRegistry(null), callers, FleetMcp.AuthorizationMode.ENFORCED,
null, FleetMcp.CapacitySource.none(), new FleetMcp.HealthCoverageSource(() -> "off"),
FleetMcp.QuarantineSource.none(), null, FleetMcp.OutageSource.none(),
FleetMcp.LeadSeatSource.none(), List.of(), null);
ServletContextHandler handler = new ServletContextHandler();
handler.setContextPath("/");
handler.addServlet(new ServletHolder(mcp.servlet()), "/mcp");
server = new Server(0);
server.setHandler(handler);
server.start();
String baseUrl = "http://127.0.0.1:"
+ ((ServerConnector) server.getConnectors()[0]).getLocalPort();
McpSchema.CallToolResult result = sendFleetSend(baseUrl, TARGET, "hi there");
assertFalse(result.isError(), "an observer sending to another observer must be accepted: "
+ textOf(result));
long waiterDeadline = System.currentTimeMillis() + 3000;
while (!rendezvous.isWaiting(TARGET) && System.currentTimeMillis() < waiterDeadline) {
Thread.sleep(5);
}
assertTrue(rendezvous.isWaiting(TARGET), "the async send must have opened its rendezvous waiter");
injector.onStatus(TARGET, AgentStatus.IDLE); // drives the real delivery attempt to herdr
long deliveryDeadline = System.currentTimeMillis() + 3000;
while (!herdr.called("agent.prompt") && System.currentTimeMillis() < deliveryDeadline) {
Thread.sleep(5);
}
assertTrue(herdr.called("agent.prompt"), "the delivery attempt must have reached herdr");
@SuppressWarnings("unchecked")
Map<String, Object> params = (Map<String, Object>) herdr.lastCall("agent.prompt").params();
assertEquals("[fleet_send from observer term_shell]\nhi there", params.get("text"),
"the receiving pane must see the sender's own daemon-resolved terminal, never a raw "
+ "echo of the content and never a client-supplied name");
}
private static McpSchema.CallToolResult sendFleetSend(String baseUrl, String target, String content) {
McpClientTransport transport = HttpClientStreamableHttpTransport.builder(baseUrl)
.endpoint("/mcp")
.build();
try (McpSyncClient client = McpClient.sync(transport).build()) {
client.initialize();
return client.callTool(McpSchema.CallToolRequest.builder("fleet_send")
.arguments(Map.of("sessionId", target, "content", content, "wait", false))
.build());
}
}
private static String textOf(McpSchema.CallToolResult r) {
return ((McpSchema.TextContent) r.content().getFirst()).text();
}
}
@@ -1,5 +1,6 @@
package dev.ltms.fleet.mcp; package dev.ltms.fleet.mcp;
import dev.ltms.fleet.Fleetd;
import dev.ltms.fleet.auth.CallerResolver; import dev.ltms.fleet.auth.CallerResolver;
import dev.ltms.fleet.auth.MemberRegistry; import dev.ltms.fleet.auth.MemberRegistry;
import dev.ltms.fleet.auth.Principal; import dev.ltms.fleet.auth.Principal;
@@ -329,12 +330,13 @@ class FleetMcpTest {
/** /**
* Same hijack and control through the fire-and-poll ({@code sendAsync}) path: the owner comes * Same hijack and control through the fire-and-poll ({@code sendAsync}) path: the owner comes
* from the ticket's recorded creator terminal, not from a caller threaded through a live call. * from the resolved principal, not from a caller argument threaded through a live call.
*/ */
@Test @Test
void aDifferentCallersMcpAnswerIsRefusedForAnAsyncSendButTheRealOwnerSucceeds() throws Exception { void aDifferentCallersMcpAnswerIsRefusedForAnAsyncSendButTheRealOwnerSucceeds() throws Exception {
Principal owner = Principal.worker("term_owner", 1);
McpSchema.CallToolResult accepted = McpSchema.CallToolResult accepted =
FleetMcp.sendAsync(messages, T, "do it", null, Set.of(), "term_owner"); FleetMcp.sendAsync(messages, T, "do it", null, Set.of(), owner);
String ticket = textOf(accepted).substring(textOf(accepted).indexOf("ticket=") + "ticket=".length()).trim(); String ticket = textOf(accepted).substring(textOf(accepted).indexOf("ticket=") + "ticket=".length()).trim();
long deadline = System.currentTimeMillis() + 3000; long deadline = System.currentTimeMillis() + 3000;
@@ -354,14 +356,15 @@ class FleetMcpTest {
assertEquals(MessageService.Phase.ASKING, asking.phase()); assertEquals(MessageService.Phase.ASKING, asking.phase());
String turnId = asking.turnId(); String turnId = asking.turnId();
McpSchema.CallToolResult hijacked = FleetMcp.answer(messages, turnId, "evil.yaml", 500L, "term_attacker"); McpSchema.CallToolResult hijacked = FleetMcp.answer(messages, turnId, "evil.yaml", 500L,
"worker:term_attacker");
assertTrue(hijacked.isError(), "a caller that did not create this delegation must get an error"); assertTrue(hijacked.isError(), "a caller that did not create this delegation must get an error");
assertFalse(ask.isDone(), "a refused answer must not resolve the worker's blocked fleet_ask"); assertFalse(ask.isDone(), "a refused answer must not resolve the worker's blocked fleet_ask");
assertEquals(MessageService.Phase.ASKING, messages.poll(ticket).phase(), assertEquals(MessageService.Phase.ASKING, messages.poll(ticket).phase(),
"a refused answer must not advance the async ticket's phase"); "a refused answer must not advance the async ticket's phase");
CompletableFuture<McpSchema.CallToolResult> answer = CompletableFuture.supplyAsync( CompletableFuture<McpSchema.CallToolResult> answer = CompletableFuture.supplyAsync(
() -> FleetMcp.answer(messages, turnId, "config.yaml", 5000L, "term_owner")); () -> FleetMcp.answer(messages, turnId, "config.yaml", 5000L, owner.ownerKey()));
assertEquals("config.yaml", textOf(ask.get(5, TimeUnit.SECONDS))); assertEquals("config.yaml", textOf(ask.get(5, TimeUnit.SECONDS)));
deadline = System.currentTimeMillis() + 3000; deadline = System.currentTimeMillis() + 3000;
while (!rendezvous.isWaiting(T) && System.currentTimeMillis() < deadline) { while (!rendezvous.isWaiting(T) && System.currentTimeMillis() < deadline) {
@@ -1108,6 +1111,253 @@ class FleetMcpTest {
assertTrue(asArchitect.contains("\"sessionId\":\"term_collab\""), asArchitect); assertTrue(asArchitect.contains("\"sessionId\":\"term_collab\""), asArchitect);
} }
// --- fleet_list's panes array -----------------------------------------------------------------
/** Calls the canonical {@code listFleet} overload directly, so a test can set the pane-discovery
* payload and its visibility independently of a real {@code Principal} / MCP exchange. */
private static McpSchema.CallToolResult listFleetWithPanes(FakeHerdr h, FleetMcp.PaneSource panes,
boolean panesVisible) {
SessionManager sessions = new SessionManager(workerService(h, "http://gx00.gw:8000", Set.of("gx00.gw")));
return FleetMcp.listFleet(
workerService(h, "http://gx00.gw:8000", Set.of("gx00.gw")), sessions, null,
FleetMcp.CapacitySource.none(), new FleetMcp.HealthCoverageSource(() -> "off"),
FleetMcp.LoopHealthSource.none(), FleetMcp.QuarantineSource.none(), FleetMcp.OutageSource.none(),
FleetMcp.LeadSeatSource.none(), new LeadContextGauge(), FleetMcp.LeadConfigDirSource.none(),
Map.of(), "", Map.of(), false,
FleetMcp.CoordinationSource.none(), false, true, true, panes, panesVisible);
}
/**
* A pane whose tab herdr reports with a label gets that label and the exact terminal id
* {@code fleet_send} takes as a target, carried as {@code sessionId}. fleetd #771: the pane's
* workspace carries its herdr space name too, next to {@code workspaceId}.
*/
@Test
void listReportsAPaneRowWithItsTabLabelAndSendableSessionId() {
FakeHerdr h = new FakeHerdr().withTab("w2", "w2:t7", "trinotes");
MemberPresence presence = new MemberPresence();
presence.markPresent("term_a");
FleetMcp.PaneSource panes = new FleetMcp.PaneSource(
() -> new PaneLocator(h).tabLabelsByTabId(),
() -> new PaneLocator(h).workspaceLabelsByWorkspaceId(),
Fleetd.deliverableTo(presence, Map::of, Map::of), _ -> false, _ -> false);
String out = textOf(listFleetWithPanes(h, panes, true));
assertTrue(out.contains("\"panes\":["), out);
assertTrue(out.contains("\"sessionId\":\"term_a\""), out);
assertTrue(out.contains("\"label\":\"trinotes\""), out);
assertTrue(out.contains("\"workspaceLabel\":\"ltms\""),
"term_a's agent lives on workspace w2, whose herdr label is \"ltms\": " + out);
assertTrue(out.contains("\"deliverable\":true"), out);
}
/**
* A pane whose tab carries no label known to herdr still gets a row -- a missing label must
* never throw, and must never drop the pane from the array, only report a {@code null} label.
* Pairs with a {@code deliverable} false reading when the target is neither present, a lead,
* nor a collaborator.
*/
@Test
void listReportsAPaneRowWithANullLabelWhenHerdrHasNoneAndNotDeliverable() {
FakeHerdr h = new FakeHerdr();
FleetMcp.PaneSource panes = new FleetMcp.PaneSource(
() -> new PaneLocator(h).tabLabelsByTabId(),
() -> new PaneLocator(h).workspaceLabelsByWorkspaceId(),
Fleetd.deliverableTo(new MemberPresence(), Map::of, Map::of), _ -> false, _ -> false);
String out = textOf(listFleetWithPanes(h, panes, true));
assertTrue(out.contains("\"panes\":["), out);
assertTrue(out.contains("\"sessionId\":\"term_a\""), out);
assertTrue(out.contains("\"label\":null"), out);
assertTrue(out.contains("\"deliverable\":false"), out);
}
/**
* fleetd #771: a pane whose agent lives in a workspace that {@code workspace.list} does not
* report (an unknown {@code workspaceId}) still gets a row -- the lookup miss must never throw,
* and must never drop the pane, only report a {@code null} "workspaceLabel".
*/
@Test
void listReportsANullWorkspaceLabelForAnUnknownWorkspaceId() {
// withAgent seeds its pane under workspace_id "wQ", which the fake's workspace.list never
// reports (only "w1"/"w2") -- modelling a workspace the lookup has no entry for.
FakeHerdr h = new FakeHerdr().withAgent("claude-x", "term_unknown_ws", "wQ:p1", "wQ:t1");
FleetMcp.PaneSource panes = new FleetMcp.PaneSource(
() -> new PaneLocator(h).tabLabelsByTabId(),
() -> new PaneLocator(h).workspaceLabelsByWorkspaceId(),
_ -> false, _ -> false, _ -> false);
String out = textOf(listFleetWithPanes(h, panes, true));
assertTrue(out.contains("\"sessionId\":\"term_unknown_ws\""), out);
assertTrue(out.contains("\"workspaceId\":\"wQ\""), out);
assertTrue(out.contains("\"workspaceLabel\":null"),
"an unknown workspaceId must project a null workspaceLabel, not throw or drop the row: " + out);
}
/** A caller this role may not show the array to gets no {@code panes} key at all. */
@Test
void listOmitsThePanesArrayWhenTheCallerMayNotSeeIt() {
FakeHerdr h = new FakeHerdr();
String out = textOf(listFleetWithPanes(h, FleetMcp.PaneSource.none(), false));
assertFalse(out.contains("\"panes\""), out);
}
/**
* The tab-label scan behind {@code panes} shares no failure path with the rest of
* {@code listFleet} -- a {@code workspace.list}/{@code tab.list} failure costs only the
* labels in the {@code panes} row (each renders {@code null}), never the {@code leads}/
* {@code members} arrays, which never needed that scan at all. fleetd #771: the same
* {@code workspace.list} failure costs {@code workspaceLabel} the same way.
*/
@Test
void listStillReportsEveryOtherArrayWhenTheLabelScanFails() {
FakeHerdr h = new FakeHerdr().workspaceListFailsWith("unavailable");
FleetMcp.PaneSource panes = new FleetMcp.PaneSource(
() -> new PaneLocator(h).tabLabelsByTabId(),
() -> new PaneLocator(h).workspaceLabelsByWorkspaceId(),
Fleetd.deliverableTo(new MemberPresence(), Map::of, Map::of), _ -> false, _ -> false);
McpSchema.CallToolResult res = listFleetWithPanes(h, panes, true);
assertNotEquals(Boolean.TRUE, res.isError(), textOf(res));
String out = textOf(res);
assertTrue(out.contains("\"panes\":["), out);
assertTrue(out.contains("\"sessionId\":\"term_a\""), out);
assertTrue(out.contains("\"label\":null"), out);
assertTrue(out.contains("\"workspaceLabel\":null"),
"a workspace.list failure must not cost the leads/members arrays, only a null "
+ "workspaceLabel: " + out);
assertTrue(out.contains("\"leads\":[]"), "a label-scan failure must not cost the leads array: " + out);
assertTrue(out.contains("\"members\":[]"), "a label-scan failure must not cost the members array: " + out);
}
/**
* fleetd #756: a pane bound to a configured architect slot with no live member session must
* report {@code role: "architect"}, read from {@link CallerResolver#boundToArchitectSlot} —
* the same classifier {@link CallerResolver#sendableObserverTarget} refuses as a {@code SEND}
* target — rather than falling through to {@code "observer"}.
*/
@Test
void listReportsArchitectForASlotBoundPaneWithNoLiveMember() {
FakeHerdr h = new FakeHerdr()
.withAgent("claude-arch", "term_unoccupied_architect", "w2:pArch", "w2:tArch");
FleetMcp.PaneSource panes = new FleetMcp.PaneSource(
Map::of, Map::of, _ -> false, "term_unoccupied_architect"::equals, _ -> false);
String out = textOf(listFleetWithPanes(h, panes, true));
assertTrue(out.contains("\"sessionId\":\"term_unoccupied_architect\""), out);
assertTrue(out.contains("\"role\":\"architect\""),
"a slot-bound pane with no live session must read \"architect\", not the generic "
+ "\"observer\" fallback: " + out);
}
/**
* Calls the canonical {@code listFleet} overload directly with an explicit {@code leads}/
* {@code collaborators} payload and {@code callerIsObserver}, mirroring exactly what the real
* {@code fleet_list} handler computes for an observer caller: {@code panesVisible} true,
* {@code leadsVisible}/{@code membersVisible}/{@code collaboratorsVisible} false.
*/
private static McpSchema.CallToolResult listFleetAsObserver(FakeHerdr h, Map<String, String> leads,
Map<String, String> collaborators, FleetMcp.PaneSource panes) {
SessionManager sessions = new SessionManager(workerService(h, "http://gx00.gw:8000", Set.of("gx00.gw")));
return FleetMcp.listFleet(
workerService(h, "http://gx00.gw:8000", Set.of("gx00.gw")), sessions, null,
FleetMcp.CapacitySource.none(), new FleetMcp.HealthCoverageSource(() -> "off"),
FleetMcp.LoopHealthSource.none(), FleetMcp.QuarantineSource.none(), FleetMcp.OutageSource.none(),
FleetMcp.LeadSeatSource.none(), new LeadContextGauge(), FleetMcp.LeadConfigDirSource.none(),
leads, "", collaborators, false,
FleetMcp.CoordinationSource.none(), false, false, false, panes, true, true);
}
/**
* fleetd #758: an observer's {@code fleet_list} now carries a {@code panes} key, filtered to
* {@link CallerResolver#sendableObserverTarget} (so a lead's pane, a spawned member's pane, a
* collaborator's pane, and an unoccupied architect-slot pane are all absent) and every
* surviving row reduced to exactly {@code sessionId}, {@code label}, {@code status},
* {@code role}, {@code deliverable} — never {@code paneId}, {@code workspaceId},
* {@code workspaceLabel} (fleetd #771 — a space name is host shape, a stronger disclosure than
* a pane id, so it stays out of the reduced row too), {@code tabId}, {@code agentType}, or
* {@code cwd}.
*/
@Test
void listFiltersAndReducesThePanesArrayForAnObserver() {
MemberRegistry members = new MemberRegistry(new FleetConfig.Fleet(Map.of(),
Map.of("lead-designer", new FleetConfig.Slot("sonnet")), Map.of(), Map.of(), null));
assertTrue(members.bind("architect:lead-designer", "term_architect_pane"));
CallerResolver callers = CallerResolver.withLeadsAndMembers(null, false, null,
() -> Map.of("term_lead_pane", "fleet01-lead"), members,
t -> "term_member_pane".equals(t) ? MemberRole.DEV : null,
() -> Map.of("term_collab_pane", "ops"));
FakeHerdr h = new FakeHerdr()
.withAgent("claude-sendable", "term_sendable", "w2:pS", "w2:tS")
.withAgent("claude-lead", "term_lead_pane", "w2:pL", "w2:tL")
.withAgent("claude-member", "term_member_pane", "w2:pM", "w2:tM")
.withAgent("claude-collab", "term_collab_pane", "w2:pC", "w2:tC")
.withAgent("claude-arch", "term_architect_pane", "w2:pA", "w2:tA")
.withTab("w2", "w2:tS", "trinotes");
FleetMcp.PaneSource panes = new FleetMcp.PaneSource(
() -> new PaneLocator(h).tabLabelsByTabId(),
() -> new PaneLocator(h).workspaceLabelsByWorkspaceId(), _ -> true,
callers::boundToArchitectSlot, callers.sendableObserverTarget());
String out = textOf(listFleetAsObserver(h, callers.leads(),
Map.of("term_collab_pane", "ops"), panes));
assertTrue(out.contains("\"panes\":["), out);
assertTrue(out.contains("\"sessionId\":\"term_sendable\""),
"an ordinary unclassified pane must still be sendable and visible: " + out);
assertFalse(out.contains("term_lead_pane"), "a lead's pane must not be enumerated: " + out);
assertFalse(out.contains("term_member_pane"), "a spawned member's pane must not be enumerated: " + out);
assertFalse(out.contains("term_collab_pane"), "a collaborator's pane must not be enumerated: " + out);
assertFalse(out.contains("term_architect_pane"),
"an unoccupied architect-slot pane must not be enumerated: " + out);
assertFalse(out.contains("\"paneId\""), "an observer's row must never carry paneId: " + out);
assertFalse(out.contains("\"workspaceId\""), "an observer's row must never carry workspaceId: " + out);
assertFalse(out.contains("\"workspaceLabel\""),
"an observer's row must never carry workspaceLabel: " + out);
assertFalse(out.contains("\"tabId\""), "an observer's row must never carry tabId: " + out);
assertFalse(out.contains("\"agentType\""), "an observer's row must never carry agentType: " + out);
assertFalse(out.contains("\"cwd\""), "an observer's row must never carry cwd: " + out);
}
/**
* Control for the test above: a primary's {@code panes} row is unchanged by fleetd #758 —
* {@code callerIsObserver} false keeps every field, including {@code paneId} and a spawned
* member's {@code cwd}.
*/
@Test
void listKeepsTheFullPaneRowForAPrimaryIncludingCwdAndPaneId() {
FakeHerdr h = new FakeHerdr();
SessionManager sessions = new SessionManager(workerService(h, "http://gx00.gw:8000", Set.of("gx00.gw")));
MemberSession spawned = sessions.acquire("ltms-local", "/worktree/member-1", null, null);
h.withAgent("claude-member", spawned.terminalId(), "w9:pMember", "w9:tMember");
FleetMcp.PaneSource panes = new FleetMcp.PaneSource(Map::of, Map::of, _ -> true, _ -> false, _ -> false);
McpSchema.CallToolResult res = FleetMcp.listFleet(
workerService(h, "http://gx00.gw:8000", Set.of("gx00.gw")), sessions, null,
FleetMcp.CapacitySource.none(), new FleetMcp.HealthCoverageSource(() -> "off"),
FleetMcp.LoopHealthSource.none(), FleetMcp.QuarantineSource.none(), FleetMcp.OutageSource.none(),
FleetMcp.LeadSeatSource.none(), new LeadContextGauge(), FleetMcp.LeadConfigDirSource.none(),
Map.of(), "", Map.of(), false,
FleetMcp.CoordinationSource.none(), true, true, true, panes, true, false);
String out = textOf(res);
assertTrue(out.contains("\"sessionId\":\"" + spawned.terminalId() + "\""), out);
assertTrue(out.contains("\"paneId\":\"w9:pMember\""),
"a primary must still see the fleet_stop handle: " + out);
assertTrue(out.contains("\"cwd\":\"/worktree/member-1\""),
"a primary must still see a spawned member's worktree path: " + out);
assertTrue(out.contains("\"role\":\"dev\""), out);
}
/** /**
* fleetd #421: {@code mailbox.pending} counts only broker-ready messages, so a blocked lead's * fleetd #421: {@code mailbox.pending} counts only broker-ready messages, so a blocked lead's
* normal, healthy state is {@code "pending": 0} next to a non-empty {@code held[]} — which * normal, healthy state is {@code "pending": 0} next to a non-empty {@code held[]} — which
@@ -1920,8 +2170,8 @@ class FleetMcpTest {
Principal architect = Principal.architect("lead-designer", "term_design", 400); Principal architect = Principal.architect("lead-designer", "term_design", 400);
MemberPresence presence = new MemberPresence(); MemberPresence presence = new MemberPresence();
FleetMcp.markSpawnedMemberPresent(worker, presence); FleetMcp.markTrackedCallerPresent(worker, presence);
FleetMcp.markSpawnedMemberPresent(architect, presence); FleetMcp.markTrackedCallerPresent(architect, presence);
assertTrue(presence.isPresent("term_worker")); assertTrue(presence.isPresent("term_worker"));
assertTrue(presence.isPresent("term_design")); assertTrue(presence.isPresent("term_design"));
@@ -1932,12 +2182,27 @@ class FleetMcpTest {
Principal lead = Principal.leader("opus", "term_lead", 100); Principal lead = Principal.leader("opus", "term_lead", 100);
MemberPresence presence = new MemberPresence(); MemberPresence presence = new MemberPresence();
FleetMcp.markSpawnedMemberPresent(lead, presence); FleetMcp.markTrackedCallerPresent(lead, presence);
FleetMcp.markSpawnedMemberPresent(Principal.anonymous(), presence); FleetMcp.markTrackedCallerPresent(Principal.anonymous(), presence);
assertFalse(presence.isPresent("term_lead")); assertFalse(presence.isPresent("term_lead"));
} }
/**
* The item whose absence would be silent: an observer's own MCP contact must still mark
* presence, or a pane resolving to the unconfigured-pane floor would sit on the injector
* readiness gate forever once something addresses it.
*/
@Test
void anObserverContactMarksPresence() {
Principal observer = Principal.observer("term_observer", 800);
MemberPresence presence = new MemberPresence();
FleetMcp.markTrackedCallerPresent(observer, presence);
assertTrue(presence.isPresent("term_observer"));
}
@Test @Test
void statusReportsLiveAgentStatus() { void statusReportsLiveAgentStatus() {
FakeHerdr blocked = new FakeHerdr().agentStatus("blocked"); FakeHerdr blocked = new FakeHerdr().agentStatus("blocked");
@@ -1996,13 +2261,15 @@ class FleetMcpTest {
/** /**
* {@code fleet_status}'s pending-ask block (the question, its {@code turnId} and its ticket) * {@code fleet_status}'s pending-ask block (the question, its {@code turnId} and its ticket)
* is shown only to the caller whose terminal created the delegation, or to a caller with no * is shown only to the caller whose owner key created the delegation. An unnamed primary is
* terminal at all (the unnamed primary) — a different terminal-bearing caller still sees the * held to the same rule: its owner key is {@code null}, which here does not match the named
* base status line, but none of the pending-ask fields. * worker that created the delegation, so it sees none of the pending-ask fields either — the
* same as any other non-creating caller.
*/ */
@Test @Test
void statusGatesThePendingAskFieldsByTheDelegationsCreatorTerminal() throws Exception { void statusGatesThePendingAskFieldsByTheDelegationsCreatorOwner() throws Exception {
String ticket = messages.sendAsync(T, "task that asks", null, "term_creator"); Principal creator = Principal.worker("term_creator", 1);
String ticket = messages.sendAsync(T, "task that asks", null, creator);
long deadline = System.currentTimeMillis() + 3000; long deadline = System.currentTimeMillis() + 3000;
while (!rendezvous.isWaiting(T) && System.currentTimeMillis() < deadline) { while (!rendezvous.isWaiting(T) && System.currentTimeMillis() < deadline) {
Thread.sleep(5); Thread.sleep(5);
@@ -2020,7 +2287,7 @@ class FleetMcpTest {
} while (asking.phase() != MessageService.Phase.ASKING && System.currentTimeMillis() < deadline); } while (asking.phase() != MessageService.Phase.ASKING && System.currentTimeMillis() < deadline);
assertEquals(MessageService.Phase.ASKING, asking.phase()); assertEquals(MessageService.Phase.ASKING, asking.phase());
String other = textOf(FleetMcp.status(messages, T, "term_other")); String other = textOf(FleetMcp.status(messages, T, "worker:term_other"));
assertTrue(other.startsWith("idle"), "the base status must still be shown: " + other); assertTrue(other.startsWith("idle"), "the base status must still be shown: " + other);
assertFalse(other.contains("which config file?"), assertFalse(other.contains("which config file?"),
"a non-creating caller must not see the question text: " + other); "a non-creating caller must not see the question text: " + other);
@@ -2029,19 +2296,26 @@ class FleetMcpTest {
assertFalse(other.contains(ticket), assertFalse(other.contains(ticket),
"a non-creating caller must not see the ticket: " + other); "a non-creating caller must not see the ticket: " + other);
String creator = textOf(FleetMcp.status(messages, T, "term_creator")); String creatorStatus = textOf(FleetMcp.status(messages, T, creator.ownerKey()));
assertTrue(creator.contains("which config file?"), "the creator must see the question: " + creator); assertTrue(creatorStatus.contains("which config file?"),
assertTrue(creator.contains(asking.turnId()), "the creator must see the turnId: " + creator); "the creator must see the question: " + creatorStatus);
assertTrue(creator.contains(ticket), "the creator must see the ticket: " + creator); assertTrue(creatorStatus.contains(asking.turnId()),
"the creator must see the turnId: " + creatorStatus);
assertTrue(creatorStatus.contains(ticket), "the creator must see the ticket: " + creatorStatus);
String unnamed = textOf(FleetMcp.status(messages, T, null)); String unnamed = textOf(FleetMcp.status(messages, T, null));
assertTrue(unnamed.contains("which config file?"), assertTrue(unnamed.startsWith("idle"), "the base status must still be shown: " + unnamed);
"a caller with no terminal (the unnamed primary) must see the question: " + unnamed); assertFalse(unnamed.contains("which config file?"),
"an unnamed primary must not see a question on a delegation a named worker created: " + unnamed);
assertFalse(unnamed.contains(asking.turnId()),
"a non-creating unnamed primary must not see the turnId: " + unnamed);
assertFalse(unnamed.contains(ticket),
"a non-creating unnamed primary must not see the ticket: " + unnamed);
// Clean up the still-open ask so the background thread does not linger past the test. // Clean up the still-open ask so the background thread does not linger past the test.
String turnId = asking.turnId(); String turnId = asking.turnId();
CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync( CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync(
() -> messages.answer(turnId, "config.yaml", 5000, "term_creator")); () -> messages.answer(turnId, "config.yaml", 5000, creator.ownerKey()));
assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer()); assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer());
deadline = System.currentTimeMillis() + 3000; deadline = System.currentTimeMillis() + 3000;
while (!rendezvous.isWaiting(T) && System.currentTimeMillis() < deadline) { while (!rendezvous.isWaiting(T) && System.currentTimeMillis() < deadline) {
@@ -2149,6 +2423,27 @@ class FleetMcpTest {
assertTrue(leadOut.contains("\"leader\":\"opus\""), leadOut); assertTrue(leadOut.contains("\"leader\":\"opus\""), leadOut);
} }
/**
* An observer reports its own role and pane, never a {@code leader} key. Without an explicit
* branch it would reach the lead branch by elimination and look right only because the
* {@code leader} key is guarded on a non-null name — this pins the branch rather than the
* accident.
*/
@Test
void whoamiReportsAnObserverNotALead() {
FakeHerdr h = new FakeHerdr();
SessionManager sessions = sessionManager(h, "http://gx00.gw:8000", Set.of("gx00.gw"));
McpSchema.CallToolResult res = FleetMcp.whoami(
Principal.observer("term_observer", 900), sessions);
assertNotEquals(Boolean.TRUE, res.isError());
String out = textOf(res);
assertTrue(out.contains("\"role\":\"observer\""), out);
assertTrue(out.contains("\"sessionId\":\"term_observer\""), out);
assertFalse(out.contains("leader"), out);
}
/** /**
* CB-548: an architect SEND delegates as its own pane (recording the per-target delegation) but * CB-548: an architect SEND delegates as its own pane (recording the per-target delegation) but
* must NEVER become the legacy singleton "primary" fallback — the per-target map does not cure * must NEVER become the legacy singleton "primary" fallback — the per-target map does not cure
@@ -169,4 +169,101 @@ class PrimaryRegistryTest {
assertTrue(reg.nudgeTargetFor("term_worker").isEmpty()); assertTrue(reg.nudgeTargetFor("term_worker").isEmpty());
assertTrue(reg.nudgeTargetFor(null).isEmpty()); assertTrue(reg.nudgeTargetFor(null).isEmpty());
} }
// ── fleetd #737 unit 3: a named lead's terminal is resolved live, not just recorded ─────────
/**
* The whole point of carrying a name: a lead that has been rolled keeps its name but gets a
* fresh terminal. {@code currentTerminalForName} stands in for the live lead-tab scan here —
* it reports the lead now sits on a different terminal than the one that was recorded — and
* {@code nudgeTargetFor} must follow the name to that current terminal, not the stale one.
*/
@Test
void nudgeTargetForFollowsARolledLeadsNameToItsCurrentTerminal() {
var reg = new PrimaryRegistry(null, name -> "opus".equals(name) ? "term_opus_after_roll" : null);
reg.recordDelegation("term_worker", "term_opus_before_roll", "opus");
assertEquals("term_opus_after_roll", reg.nudgeTargetFor("term_worker").orElseThrow(),
"the name must be resolved to the lead's CURRENT terminal, not the one recorded "
+ "at delegation time");
}
/**
* The lookup cannot place every name — an architect/collaborator name (never a lead), or a lead
* whose tab the scan cannot currently see (just rolled, off-host, non-herdr). Either way the
* terminal actually recorded is still the right thing to try, exactly as before this unit.
*/
@Test
void nudgeTargetForFallsBackToTheRecordedTerminalWhenTheNameCannotBePlaced() {
var reg = new PrimaryRegistry(null, name -> null); // nothing is ever currently recognised
reg.recordDelegation("term_worker", "term_lead_recorded", "opus");
assertEquals("term_lead_recorded", reg.nudgeTargetFor("term_worker").orElseThrow());
}
/** The 2-arg {@code recordDelegation} overload records no name, so resolution never applies. */
@Test
void recordDelegationWithNoNameIsNeverResolvedByLookup() {
var reg = new PrimaryRegistry(null, name -> {
throw new AssertionError("a delegation recorded with no name must never consult the lookup");
});
reg.recordDelegation("term_worker", "term_lead");
assertEquals("term_lead", reg.nudgeTargetFor("term_worker").orElseThrow());
}
/** As {@link #nudgeTargetForFollowsARolledLeadsNameToItsCurrentTerminal}, for the singleton. */
@Test
void currentPrimaryTerminalFollowsARolledLeadsNameToItsCurrentTerminal() {
var reg = new PrimaryRegistry(null, name -> "sol".equals(name) ? "term_sol_after_roll" : null);
reg.record("term_sol_before_roll", "sol");
assertEquals("term_sol_after_roll", reg.currentPrimaryTerminal().orElseThrow());
assertEquals("term_sol_before_roll", reg.primaryTerminal().orElseThrow(),
"primaryTerminal() stays the raw recorded value — currentPrimaryTerminal() is the "
+ "one that resolves live");
}
@Test
void currentPrimaryTerminalFallsBackWhenTheNameCannotBePlaced() {
var reg = new PrimaryRegistry(null, name -> null);
reg.record("term_sol", "sol");
assertEquals("term_sol", reg.currentPrimaryTerminal().orElseThrow());
}
@Test
void currentPrimaryTerminalWithNoNameRecordedIsTheRawTerminal() {
var reg = new PrimaryRegistry(null, name -> {
throw new AssertionError("no name was ever recorded, the lookup must not be consulted");
});
reg.record("term_x");
assertEquals("term_x", reg.currentPrimaryTerminal().orElseThrow());
}
@Test
void currentPrimaryTerminalIsEmptyWhenNothingWasEverLearned() {
var reg = new PrimaryRegistry(null, name -> "anything");
assertTrue(reg.currentPrimaryTerminal().isEmpty());
}
/** A pin never carries a name, so a pinned registry's singleton resolution is always a no-op. */
@Test
void currentPrimaryTerminalForAPinIsNeverResolvedByLookup() {
var reg = new PrimaryRegistry("term_pinned", name -> {
throw new AssertionError("a pin carries no name, the lookup must not be consulted");
});
assertEquals("term_pinned", reg.currentPrimaryTerminal().orElseThrow());
}
/** {@code nudgeTargetFor}'s fallback to the singleton is the resolved one, not the raw one. */
@Test
void nudgeTargetForWithNoDelegationFallsBackToTheResolvedSingleton() {
var reg = new PrimaryRegistry(null, name -> "opus".equals(name) ? "term_opus_after_roll" : null);
reg.record("term_opus_before_roll", "opus");
assertEquals("term_opus_after_roll", reg.nudgeTargetFor("term_never_seen").orElseThrow());
}
} }
@@ -40,7 +40,7 @@ class LeadCoordLoopTest {
@Test @Test
void deliversAHeldMessageToTheLeadPaneAndAcksIt() { void deliversAHeldMessageToTheLeadPaneAndAcksIt() {
var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "the merge is blocked")); var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "the merge is blocked"));
var herdr = new FakeHerdr().agentStatus("idle"); var herdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET).agentStatus("idle");
loop(channel, herdr, Map.of(LEAD_TERM, SELF)).tick(); loop(channel, herdr, Map.of(LEAD_TERM, SELF)).tick();
@@ -57,7 +57,7 @@ class LeadCoordLoopTest {
@Test @Test
void redeliveryOfAMessageAlreadyWrittenToThePaneIsAckedWithoutAnotherPaneWrite() { void redeliveryOfAMessageAlreadyWrittenToThePaneIsAckedWithoutAnotherPaneWrite() {
var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "recover me")); var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "recover me"));
var herdr = new FakeHerdr().agentStatus("idle"); var herdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET).agentStatus("idle");
var loop = loop(channel, herdr, Map.of(LEAD_TERM, SELF)); var loop = loop(channel, herdr, Map.of(LEAD_TERM, SELF));
loop.tick(); loop.tick();
@@ -71,7 +71,7 @@ class LeadCoordLoopTest {
@Test @Test
void aRedeliveryIsAckedEvenWhileTheLeadIsMidTurn() { void aRedeliveryIsAckedEvenWhileTheLeadIsMidTurn() {
var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "recover me")); var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "recover me"));
var herdr = new FakeHerdr().agentStatus("idle"); var herdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET).agentStatus("idle");
var loop = loop(channel, herdr, Map.of(LEAD_TERM, SELF)); var loop = loop(channel, herdr, Map.of(LEAD_TERM, SELF));
loop.tick(); loop.tick();
@@ -91,7 +91,7 @@ class LeadCoordLoopTest {
@Test @Test
void leavesTheMessageUnackedWhenTheLeadIsMidTurn() { void leavesTheMessageUnackedWhenTheLeadIsMidTurn() {
var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "hello")); var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "hello"));
var herdr = new FakeHerdr().agentStatus("working"); var herdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET).agentStatus("working");
loop(channel, herdr, Map.of(LEAD_TERM, SELF)).tick(); loop(channel, herdr, Map.of(LEAD_TERM, SELF)).tick();
@@ -103,7 +103,7 @@ class LeadCoordLoopTest {
@Test @Test
void leavesTheMessageUnackedWhenNoLeadPaneIsKnown() { void leavesTheMessageUnackedWhenNoLeadPaneIsKnown() {
var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "hello")); var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "hello"));
var herdr = new FakeHerdr().agentStatus("idle"); var herdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET).agentStatus("idle");
loop(channel, herdr, Map.of()).tick(); loop(channel, herdr, Map.of()).tick();
@@ -115,7 +115,8 @@ class LeadCoordLoopTest {
@Test @Test
void leavesTheMessageUnackedWhenHerdrRefusesTheInjection() { void leavesTheMessageUnackedWhenHerdrRefusesTheInjection() {
var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "hello")); var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "hello"));
var herdr = new FakeHerdr().agentStatus("idle").agentSendFailsWith("agent_not_found"); var herdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET)
.agentStatus("idle").agentSendFailsWith("agent_not_found");
loop(channel, herdr, Map.of(LEAD_TERM, SELF)).tick(); loop(channel, herdr, Map.of(LEAD_TERM, SELF)).tick();
@@ -126,7 +127,7 @@ class LeadCoordLoopTest {
@Test @Test
void resolvesTheLeadByNameWhenSeveralAreKnown() { void resolvesTheLeadByNameWhenSeveralAreKnown() {
var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "hello")); var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "hello"));
var herdr = new FakeHerdr().agentStatus("idle"); var herdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET).agentStatus("idle");
// Two leads on this daemon; only one carries the coord-id the mailbox is owned as. // Two leads on this daemon; only one carries the coord-id the mailbox is owned as.
var leads = new java.util.LinkedHashMap<String, String>(); var leads = new java.util.LinkedHashMap<String, String>();
leads.put("term_other", "some-other-lead"); leads.put("term_other", "some-other-lead");
@@ -142,7 +143,7 @@ class LeadCoordLoopTest {
@Test @Test
void holdsWhenSeveralLeadsAreKnownAndNoneCarriesTheCoordId() { void holdsWhenSeveralLeadsAreKnownAndNoneCarriesTheCoordId() {
var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "hello")); var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "hello"));
var herdr = new FakeHerdr().agentStatus("idle"); var herdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET).agentStatus("idle");
var leads = new java.util.LinkedHashMap<String, String>(); var leads = new java.util.LinkedHashMap<String, String>();
leads.put("term_one", "lead-one"); leads.put("term_one", "lead-one");
leads.put("term_two", "lead-two"); leads.put("term_two", "lead-two");
@@ -159,7 +160,7 @@ class LeadCoordLoopTest {
var channel = new FakeLeadChannel(SELF) var channel = new FakeLeadChannel(SELF)
.hold(new LeadMessage("m1", PEER, SELF, "first")) .hold(new LeadMessage("m1", PEER, SELF, "first"))
.hold(new LeadMessage("m2", PEER, SELF, "second")); .hold(new LeadMessage("m2", PEER, SELF, "second"));
var herdr = new FakeHerdr().agentStatus("idle"); var herdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET).agentStatus("idle");
var loop = loop(channel, herdr, Map.of(LEAD_TERM, SELF)); var loop = loop(channel, herdr, Map.of(LEAD_TERM, SELF));
loop.tick(); loop.tick();
@@ -175,10 +176,51 @@ class LeadCoordLoopTest {
@Test @Test
void anEmptyMailboxNeverTouchesHerdr() { void anEmptyMailboxNeverTouchesHerdr() {
var channel = new FakeLeadChannel(SELF); var channel = new FakeLeadChannel(SELF);
var herdr = new FakeHerdr().agentStatus("idle"); var herdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET).agentStatus("idle");
loop(channel, herdr, Map.of(LEAD_TERM, SELF)).tick(); loop(channel, herdr, Map.of(LEAD_TERM, SELF)).tick();
assertEquals(0, herdr.calls.size(), "an idle fleet must not poll a pane's status every tick"); assertEquals(0, herdr.calls.size(), "an idle fleet must not poll a pane's status every tick");
} }
@Test
void aLeadWithUnsubmittedTextInItsPromptBoxKeepsTheMessageHeldAndUnacked() {
var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "the merge is blocked"));
var herdr = new FakeHerdr().detectionText(FakeHerdr.DRAFTED_PROMPT_CARET).agentStatus("idle");
loop(channel, herdr, Map.of(LEAD_TERM, SELF)).tick();
assertEquals(0, prompts(herdr).size(),
"delivery pastes and submits, so it must not land on a half-typed line");
assertEquals(List.of(), channel.acked(), "an undelivered message stays on the broker");
assertFalse(channel.peek().isEmpty(), "and is still held");
}
@Test
void aMessageHeldForADraftIsDeliveredOnALaterTick() {
var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "the merge is blocked"));
var herdr = new FakeHerdr().detectionText(FakeHerdr.DRAFTED_PROMPT_CARET).agentStatus("idle");
var loop = loop(channel, herdr, Map.of(LEAD_TERM, SELF));
loop.tick();
assertEquals(0, prompts(herdr).size());
herdr.detectionText(FakeHerdr.IDLE_PROMPT_CARET);
loop.tick();
assertEquals(1, prompts(herdr).size(), "the held message lands once the box is empty");
assertEquals(List.of("m1"), channel.acked());
}
@Test
void anUnreadablePaneKeepsTheMessageHeld() {
var channel = new FakeLeadChannel(SELF).hold(new LeadMessage("m1", PEER, SELF, "the merge is blocked"));
var herdr = new FakeHerdr().detectionText("garbled ansi noise with no input box").agentStatus("idle");
loop(channel, herdr, Map.of(LEAD_TERM, SELF)).tick();
assertEquals(0, prompts(herdr).size(), "a pane whose box cannot be found may be holding a draft");
assertEquals(List.of(), channel.acked());
}
} }
@@ -7,6 +7,7 @@ import ch.qos.logback.core.read.ListAppender;
import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.ObjectMapper;
import dev.ltms.fleet.herdr.AgentControl; import dev.ltms.fleet.herdr.AgentControl;
import dev.ltms.fleet.herdr.FakeHerdr;
import dev.ltms.fleet.herdr.AgentStatus; import dev.ltms.fleet.herdr.AgentStatus;
import dev.ltms.fleet.herdr.HerdrClient; import dev.ltms.fleet.herdr.HerdrClient;
import dev.ltms.fleet.lead.LeadContextGauge; import dev.ltms.fleet.lead.LeadContextGauge;
@@ -648,6 +649,42 @@ class LeadHeartbeatLoopTest {
"the notice text must appear exactly once across all three sends: " + herdr.sentTexts()); "the notice text must appear exactly once across all three sends: " + herdr.sentTexts());
} }
// ── fleetd #737 unit 3: tick() nudges the lead's CURRENT terminal, not the learned one ───────
/**
* {@code tick()} reads {@code primaryRegistry.currentPrimaryTerminal()} both to check the lead's
* status and to send the nudge. Here the registry learned the lead's terminal under its name
* before a roll; {@code currentTerminalForName} stands in for the live lead-tab scan and reports
* the lead now sits on a different terminal. A correct tick must follow the name and nudge the
* new terminal — nudging the old one would mean the heartbeat lost the lead across its own roll.
*/
@Test
void tickNudgesTheLeadsCurrentTerminalAfterARoll() {
var herdr = new FailableHerdrClient("term_lead_after_roll");
var now = new AtomicLong(NOW);
InMemoryReplyInbox inbox = new InMemoryReplyInbox();
inbox.own(WORKER);
inbox.publish(WORKER, "m1", "hello");
@SuppressWarnings("unchecked")
List<MemberSession>[] rosterBox = new List[]{List.of(new MemberSession("p1", WORKER, "prof",
MemberRole.DEV, "/cwd", null, 0, 0, 0, MemberSession.State.READY, null, null))};
AgentControl agents = new AgentControl(herdr);
PrimaryRegistry registry = new PrimaryRegistry(null,
name -> "opus".equals(name) ? "term_lead_after_roll" : null);
registry.record("term_lead_before_roll", "opus");
ReplyPushLoop pushLoop = new ReplyPushLoop(registry, agents, inbox, scheduler, 5, 100_000);
LeadHeartbeatLoop loop = new LeadHeartbeatLoop(registry, agents, inbox, () -> rosterBox[0], pushLoop,
scheduler, now::get, IDLE_AFTER_NANOS, 100_000L, 0);
loop.tick(); // opens the idle window
now.addAndGet(TimeUnit.SECONDS.toNanos(400));
loop.tick(); // past the quiet period, pending reply -> INJECT
assertEquals(List.of("term_lead_after_roll"), herdr.promptTargets(),
"the heartbeat must read and nudge the lead's CURRENT terminal, not the one learned "
+ "before the roll");
}
/** /**
* Fake herdr client for the four tests above: always reports {@code lead} as IDLE, records the * Fake herdr client for the four tests above: always reports {@code lead} as IDLE, records the
* {@code text} of every {@code agent.prompt} call, and can be told to throw on the very next * {@code text} of every {@code agent.prompt} call, and can be told to throw on the very next
@@ -657,7 +694,10 @@ class LeadHeartbeatLoopTest {
private static final ObjectMapper MAPPER = new ObjectMapper(); private static final ObjectMapper MAPPER = new ObjectMapper();
private final String lead; private final String lead;
private final List<String> sentTexts = new ArrayList<>(); private final List<String> sentTexts = new ArrayList<>();
private final List<String> promptTargets = new ArrayList<>();
private boolean throwOnNextSend = false; private boolean throwOnNextSend = false;
/** What {@code agent.read} reports — the loop reads the lead's input box before it nudges. */
private String paneTail = FakeHerdr.IDLE_PROMPT_CARET;
FailableHerdrClient(String lead) { FailableHerdrClient(String lead) {
this.lead = lead; this.lead = lead;
@@ -667,10 +707,19 @@ class LeadHeartbeatLoopTest {
throwOnNextSend = true; throwOnNextSend = true;
} }
void paneTail(String tail) {
this.paneTail = tail;
}
List<String> sentTexts() { List<String> sentTexts() {
return List.copyOf(sentTexts); return List.copyOf(sentTexts);
} }
/** Every terminal an {@code agent.prompt} call named, in call order. */
List<String> promptTargets() {
return List.copyOf(promptTargets);
}
@Override @Override
@SuppressWarnings("unchecked") @SuppressWarnings("unchecked")
public JsonNode call(String method, Object params) { public JsonNode call(String method, Object params) {
@@ -680,13 +729,18 @@ class LeadHeartbeatLoopTest {
.put("terminal_id", lead) .put("terminal_id", lead)
.put("agent_status", "idle")); .put("agent_status", "idle"));
} }
if ("agent.read".equals(method)) {
return MAPPER.createObjectNode()
.set("read", MAPPER.createObjectNode().put("text", paneTail));
}
if ("agent.prompt".equals(method)) { if ("agent.prompt".equals(method)) {
Map<String, Object> p = params instanceof Map ? (Map<String, Object>) params : Map.of();
if (throwOnNextSend) { if (throwOnNextSend) {
throwOnNextSend = false; throwOnNextSend = false;
throw new RuntimeException("simulated transient herdr send failure"); throw new RuntimeException("simulated transient herdr send failure");
} }
Map<String, Object> p = params instanceof Map ? (Map<String, Object>) params : Map.of();
sentTexts.add(String.valueOf(p.get("text"))); sentTexts.add(String.valueOf(p.get("text")));
promptTargets.add(String.valueOf(p.get("target")));
} }
return MAPPER.createObjectNode(); return MAPPER.createObjectNode();
} }
@@ -695,4 +749,49 @@ class LeadHeartbeatLoopTest {
public void close() { public void close() {
} }
} }
// ── the operator's own prompt box ────────────────────────────────────────────────────────────
@Test
void tickHoldsTheNudgeWhileTheLeadsPromptBoxHoldsUnsubmittedText() {
var herdr = new FailableHerdrClient(LEAD);
herdr.paneTail(FakeHerdr.DRAFTED_PROMPT_CARET);
var now = new AtomicLong(NOW);
@SuppressWarnings("unchecked")
List<MemberSession>[] rosterBox = new List[]{List.of()};
InMemoryReplyInbox inbox = new InMemoryReplyInbox();
LeadHeartbeatLoop loop = tickableLoop(herdr, now, rosterBox, inbox, 0, scheduler);
loop.tick(); // opens the idle window
now.addAndGet(TimeUnit.SECONDS.toNanos(400));
loop.tick(); // would INJECT, but the operator is mid-sentence
assertEquals(0, herdr.sentTexts().size(),
"a nudge pastes and submits, so it must not land on a half-typed line");
herdr.paneTail(FakeHerdr.IDLE_PROMPT_CARET);
loop.tick();
assertEquals(1, herdr.sentTexts().size(), "the held nudge lands once the box is empty");
assertTrue(herdr.sentTexts().get(0).contains("Your own context is nearly full"),
"and it still carries the notice the held tick did not spend: " + herdr.sentTexts().get(0));
}
@Test
void anUnreadablePaneHoldsTheHeartbeatNudge() {
var herdr = new FailableHerdrClient(LEAD);
herdr.paneTail("garbled ansi noise with no input box");
var now = new AtomicLong(NOW);
@SuppressWarnings("unchecked")
List<MemberSession>[] rosterBox = new List[]{List.of()};
InMemoryReplyInbox inbox = new InMemoryReplyInbox();
LeadHeartbeatLoop loop = tickableLoop(herdr, now, rosterBox, inbox, 0, scheduler);
loop.tick();
now.addAndGet(TimeUnit.SECONDS.toNanos(400));
loop.tick();
assertEquals(0, herdr.sentTexts().size(),
"a pane whose box cannot be found may be holding a draft");
}
} }
@@ -19,7 +19,7 @@ import static org.junit.jupiter.api.Assertions.assertTrue;
* {@link MessageService#poll(String)} overload. That overload skips the ownership check in * {@link MessageService#poll(String)} overload. That overload skips the ownership check in
* {@code MessageService}'s {@code ownsTicket} entirely, so a caller of it can read any session's * {@code MessageService}'s {@code ownsTicket} entirely, so a caller of it can read any session's
* ticket. Every production caller must go through {@link MessageService#poll(String, String)} * ticket. Every production caller must go through {@link MessageService#poll(String, String)}
* and pass a {@code callerTerminal} explicitly, even when it is {@code null}. * and pass a {@code callerOwner} explicitly, even when it is {@code null}.
* *
* <p>This reads each file's own source text rather than reflecting on compiled bytecode, because * <p>This reads each file's own source text rather than reflecting on compiled bytecode, because
* the risk is a future one-word edit at a call site, not a missing overload. * the risk is a future one-word edit at a call site, not a missing overload.
@@ -48,7 +48,7 @@ class MessageServicePollUsageTest {
+ "below proves nothing"); + "below proves nothing");
assertTrue(violations.isEmpty(), "found a call to the fail-open MessageService.poll(String) " assertTrue(violations.isEmpty(), "found a call to the fail-open MessageService.poll(String) "
+ "overload, which skips the ownership check entirely -- pass a callerTerminal " + "overload, which skips the ownership check entirely -- pass a callerOwner "
+ "explicitly (even if null) through poll(String, String) instead: " + violations); + "explicitly (even if null) through poll(String, String) instead: " + violations);
// CONTROL: the arity parser actually finds the two genuine two-argument call sites (the // CONTROL: the arity parser actually finds the two genuine two-argument call sites (the
@@ -58,7 +58,7 @@ class MessageServicePollUsageTest {
assertEquals(2, twoArgSites.size(), "control failed: expected exactly the two known " assertEquals(2, twoArgSites.size(), "control failed: expected exactly the two known "
+ "two-argument messages.poll(...) call sites, found: " + twoArgSites); + "two-argument messages.poll(...) call sites, found: " + twoArgSites);
assertTrue(twoArgSites.stream().anyMatch(s -> s.contains("FleetMcp.java")), assertTrue(twoArgSites.stream().anyMatch(s -> s.contains("FleetMcp.java")),
"control failed: did not find the FleetMcp.java messages.poll(ticket, callerTerminal) " "control failed: did not find the FleetMcp.java messages.poll(ticket, callerOwner) "
+ "site among: " + twoArgSites); + "site among: " + twoArgSites);
assertTrue(twoArgSites.stream().anyMatch(s -> s.contains("FleetApp.java")), assertTrue(twoArgSites.stream().anyMatch(s -> s.contains("FleetApp.java")),
"control failed: did not find the FleetApp.java messages.poll(...) site among: " "control failed: did not find the FleetApp.java messages.poll(...) site among: "
@@ -4,6 +4,7 @@ import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.Logger; import ch.qos.logback.classic.Logger;
import ch.qos.logback.classic.spi.ILoggingEvent; import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.read.ListAppender; import ch.qos.logback.core.read.ListAppender;
import dev.ltms.fleet.auth.Principal;
import dev.ltms.fleet.herdr.AgentControl; import dev.ltms.fleet.herdr.AgentControl;
import dev.ltms.fleet.herdr.AgentStatus; import dev.ltms.fleet.herdr.AgentStatus;
import dev.ltms.fleet.herdr.FakeHerdr; import dev.ltms.fleet.herdr.FakeHerdr;
@@ -921,59 +922,128 @@ class MessageServiceTest {
assertNull(other.poll(ticket), "a ticket minted by a different instance must not resolve here"); assertNull(other.poll(ticket), "a ticket minted by a different instance must not resolve here");
} }
// --- fleetd #705: a ticket's creator terminal gates who may poll it ------------------------- // --- ticket ownership -----------------------------------------------------------------------
@Test @Test
void pollByAnotherTerminalIsRefused() throws Exception { void leadBIsRefusedFromLeadAsTicket() throws Exception {
String ticket = messages.sendAsync(T, "long task", null, "term_a"); Principal leadA = Principal.leader("opus", "term_a", 1);
Principal leadB = Principal.leader("sol", "term_b", 2);
String ticket = messages.sendAsync(T, "long task", null, leadA);
awaitWaiting(); awaitWaiting();
injector.onStatus(T, AgentStatus.IDLE); // deliver injector.onStatus(T, AgentStatus.IDLE); // deliver
injector.onStatus(T, AgentStatus.WORKING); // worker works injector.onStatus(T, AgentStatus.WORKING); // worker works
assertTrue(rendezvous.resolve(T, "secret async result"), "a reply resolves the async send"); assertTrue(rendezvous.resolve(T, "secret async result"), "a reply resolves the async send");
MessageService.TaskView owner = driveAsyncTicketToDone(ticket, "term_a"); MessageService.TaskView owner = driveAsyncTicketToDone(ticket, leadA.ownerKey());
assertNotNull(owner, "the creator must still be able to read its own ticket"); assertNotNull(owner, "the creator must still be able to read its own ticket");
assertEquals(MessageService.Phase.DONE, owner.phase()); assertEquals(MessageService.Phase.DONE, owner.phase());
MessageService.TaskView refused = messages.poll(ticket, "term_b"); MessageService.TaskView refused = messages.poll(ticket, leadB.ownerKey());
assertNotNull(refused, "a different terminal gets a refusal, not silence"); assertNotNull(refused, "a different lead gets a refusal, not silence");
assertNotEquals(MessageService.Phase.DONE, refused.phase(), assertNotEquals(MessageService.Phase.DONE, refused.phase(),
"a different terminal must never see the ticket as DONE"); "a different lead must never see the ticket as DONE");
assertNull(refused.reply(), "a refusal must never carry the reply text"); assertNull(refused.reply(), "a refusal must never carry the reply text");
assertFalse(String.valueOf(refused).contains("secret async result"), assertFalse(String.valueOf(refused).contains("secret async result"),
"the reply text must not appear anywhere in the refused view"); "the reply text must not appear anywhere in the refused view");
} }
@Test @Test
void unnamedPrimaryStillReadsAnyTicket() throws Exception { void unnamedPrimaryIsRefusedFromANamedLeadsTicket() throws Exception {
String ticket = messages.sendAsync(T, "long task", null, "term_lead"); String ticket = messages.sendAsync(T, "long task", null,
Principal.leader("opus", "term_lead", 1));
awaitWaiting(); awaitWaiting();
injector.onStatus(T, AgentStatus.IDLE); // deliver injector.onStatus(T, AgentStatus.IDLE); // deliver
injector.onStatus(T, AgentStatus.WORKING); // worker works injector.onStatus(T, AgentStatus.WORKING); // worker works
assertTrue(rendezvous.resolve(T, "primary-visible result"), "a reply resolves the async send"); assertTrue(rendezvous.resolve(T, "primary-visible result"), "a reply resolves the async send");
// callerTerminal == null is the unnamed primary (resolved by token or loopback trust, with MessageService.TaskView refused = messages.poll(ticket, Principal.primary(1).ownerKey());
// no herdr pane) — it must read a ticket a terminal-bearing lead created. assertNotNull(refused, "a different owner gets a refusal, not silence");
MessageService.TaskView view = driveAsyncTicketToDone(ticket, null); assertEquals(MessageService.Phase.FAILED, refused.phase());
assertNotNull(view, "the unnamed primary must be able to read any ticket"); assertEquals("forbidden: this ticket was created by a different session", refused.detail());
assertNull(refused.reply(), "a refusal must never carry the reply text");
assertFalse(String.valueOf(refused).contains("primary-visible result"),
"the reply text must not appear anywhere in the refused view");
}
/**
* Positive control for {@link #unnamedPrimaryIsRefusedFromANamedLeadsTicket}: without this,
* that test would pass just as well if {@code poll} refused every caller.
*/
@Test
void unnamedPrimaryReadsItsOwnTicket() throws Exception {
String ticket = messages.sendAsync(T, "long task", null, Principal.primary(1));
awaitWaiting();
injector.onStatus(T, AgentStatus.IDLE); // deliver
injector.onStatus(T, AgentStatus.WORKING); // worker works
assertTrue(rendezvous.resolve(T, "primary-visible result"), "a reply resolves the async send");
MessageService.TaskView view = driveAsyncTicketToDone(ticket, Principal.primary(2).ownerKey());
assertNotNull(view, "an unnamed primary must be able to read a ticket another unnamed primary created");
assertEquals(MessageService.Phase.DONE, view.phase()); assertEquals(MessageService.Phase.DONE, view.phase());
assertEquals("primary-visible result", view.reply()); assertEquals("primary-visible result", view.reply());
} }
@Test @Test
void creatorReadsItsOwnTicket() throws Exception { void theOneArgPollOverloadBypassesOwnershipEntirely() throws Exception {
String ticket = messages.sendAsync(T, "long task", null, "term_creator"); String ticket = messages.sendAsync(T, "long task", null,
Principal.leader("opus", "term_lead", 1));
awaitWaiting();
injector.onStatus(T, AgentStatus.IDLE); // deliver
injector.onStatus(T, AgentStatus.WORKING); // worker works
assertTrue(rendezvous.resolve(T, "primary-visible result"), "a reply resolves the async send");
MessageService.TaskView view = null;
long deadline = System.currentTimeMillis() + 2000;
while (view == null || view.phase() != MessageService.Phase.DONE) {
if (System.currentTimeMillis() >= deadline) break;
view = messages.poll(ticket); // the one-arg, no-check overload -- no caller owner key at all
//noinspection BusyWait
Thread.sleep(5);
}
assertNotNull(view, "the internal bypass must read a ticket owned by a named lead");
assertEquals(MessageService.Phase.DONE, view.phase());
assertEquals("primary-visible result", view.reply());
}
@Test
void namedLeadCanPollItsTicketAfterItsTerminalChanges() throws Exception {
Principal oldLead = Principal.leader("opus", "term_OLD", 1);
Principal newLead = Principal.leader("opus", "term_NEW", 2);
assertNotEquals(oldLead.terminal(), newLead.terminal(), "the test requires different terminals");
String ticket = messages.sendAsync(T, "long task", null, oldLead);
awaitWaiting(); awaitWaiting();
injector.onStatus(T, AgentStatus.IDLE); // deliver injector.onStatus(T, AgentStatus.IDLE); // deliver
injector.onStatus(T, AgentStatus.WORKING); // worker works injector.onStatus(T, AgentStatus.WORKING); // worker works
assertTrue(rendezvous.resolve(T, "own result"), "a reply resolves the async send"); assertTrue(rendezvous.resolve(T, "own result"), "a reply resolves the async send");
MessageService.TaskView view = driveAsyncTicketToDone(ticket, "term_creator"); MessageService.TaskView view = driveAsyncTicketToDone(ticket, newLead.ownerKey());
assertNotNull(view, "the ticket's own creator must be able to read it"); assertNotNull(view, "the same named lead must read the ticket from its new terminal");
assertEquals(MessageService.Phase.DONE, view.phase()); assertEquals(MessageService.Phase.DONE, view.phase());
assertEquals("own result", view.reply()); assertEquals("own result", view.reply());
} }
@Test
void anonymousOwnerKeyIsRefusedByTheTicketGateItself() {
Principal lead = Principal.leader("opus", "term_lead", 1);
String ticket = messages.sendAsync(T, "long task", null, lead);
MessageService.TaskView refused = messages.poll(ticket, Principal.anonymous().ownerKey());
assertNotNull(refused);
assertEquals(MessageService.Phase.FAILED, refused.phase());
assertEquals("forbidden: this ticket was created by a different session", refused.detail());
}
@Test
void architectOwnershipUsesTerminalRatherThanSlot() {
Principal oldArchitect = Principal.architect("opus", "term_OLD", 1);
Principal newArchitect = Principal.architect("opus", "term_NEW", 2);
String ticket = messages.sendAsync(T, "long task", null, oldArchitect);
assertEquals(MessageService.Phase.PENDING, messages.poll(ticket, oldArchitect.ownerKey()).phase());
assertEquals(MessageService.Phase.FAILED, messages.poll(ticket, newArchitect.ownerKey()).phase());
}
@Test @Test
void pollReportsACompletedTicket() throws Exception { void pollReportsACompletedTicket() throws Exception {
String ticket = messages.sendAsync(T, "long task"); String ticket = messages.sendAsync(T, "long task");
@@ -1950,13 +2020,15 @@ class MessageServiceTest {
} }
/** /**
* A caller's own terminal must match the terminal that created the delegation to see its * A caller's owner key must match the key that created the delegation to see its pending
* pending question; a different terminal-bearing caller sees nothing, and a caller with no * question. The unnamed primary is held to the same rule as everyone else: its key is
* terminal at all (the unnamed primary) always sees it. * {@code null}, which here does not match the named worker that created this delegation, so
* it is refused too.
*/ */
@Test @Test
void pendingAskGatesTheQuestionByTheDelegationsCreatorTerminal() throws Exception { void pendingAskGatesTheQuestionByTheDelegationsCreatorOwner() throws Exception {
String ticket = messages.sendAsync(T, "task that asks", null, "term_creator"); Principal creator = Principal.worker("term_creator", 1);
String ticket = messages.sendAsync(T, "task that asks", null, creator);
awaitWaiting(); awaitWaiting();
injectDelivery(); injectDelivery();
@@ -1964,19 +2036,18 @@ class MessageServiceTest {
CompletableFuture.supplyAsync(() -> messages.ask(T, "which config file?", 5000)); CompletableFuture.supplyAsync(() -> messages.ask(T, "which config file?", 5000));
MessageService.TaskView asking = awaitTicketPhase(ticket, MessageService.Phase.ASKING); MessageService.TaskView asking = awaitTicketPhase(ticket, MessageService.Phase.ASKING);
assertNull(messages.pendingAsk(T, "term_other"), assertNull(messages.pendingAsk(T, "worker:term_other"),
"a caller whose terminal did not create the delegation must not see the question"); "a caller whose key did not create the delegation must not see the question");
MessageService.PendingAsk own = messages.pendingAsk(T, "term_creator"); MessageService.PendingAsk own = messages.pendingAsk(T, creator.ownerKey());
assertNotNull(own, "the creating caller must see its own open question"); assertNotNull(own, "the creating caller must see its own open question");
assertEquals("which config file?", own.question()); assertEquals("which config file?", own.question());
MessageService.PendingAsk unnamed = messages.pendingAsk(T, null); assertNull(messages.pendingAsk(T, Principal.primary(1).ownerKey()),
assertNotNull(unnamed, "a caller with no terminal (the unnamed primary) must always see the question"); "an unnamed primary must not see a question on a delegation a named worker created");
assertEquals("which config file?", unnamed.question());
CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync( CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync(
() -> messages.answer(asking.turnId(), "config.yaml", 5000, "term_creator")); () -> messages.answer(asking.turnId(), "config.yaml", 5000, creator.ownerKey()));
assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer()); assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer());
awaitWaiting(); awaitWaiting();
assertTrue(rendezvous.resolve(T, "done")); assertTrue(rendezvous.resolve(T, "done"));
@@ -1984,13 +2055,42 @@ class MessageServiceTest {
} }
/** /**
* A task created with no recorded creator terminal (a short {@code sendAsync} overload) must * Positive control for {@link #pendingAskGatesTheQuestionByTheDelegationsCreatorOwner}:
* not hand its open question to any caller that does have a terminal — only a caller with no * without this, that test's refusal would pass just as well if {@code pendingAsk} refused
* terminal at all may still see it. * every caller. Here the delegation's creator is itself an unnamed primary (owner key
* {@code null}), so another unnamed primary's {@code null} key must still match it.
*/ */
@Test @Test
void pendingAskDeniesATerminalBearingCallerWhenTheTaskRecordsNoCreator() throws Exception { void unnamedPrimarySeesItsOwnDelegationsPendingQuestion() throws Exception {
String ticket = messages.sendAsync(T, "task that asks"); // no creatorTerminal recorded String ticket = messages.sendAsync(T, "task that asks", null, Principal.primary(1));
awaitWaiting();
injectDelivery();
CompletableFuture<MessageService.AskResult> ask =
CompletableFuture.supplyAsync(() -> messages.ask(T, "which config file?", 5000));
awaitTicketPhase(ticket, MessageService.Phase.ASKING);
MessageService.PendingAsk unnamed = messages.pendingAsk(T, Principal.primary(2).ownerKey());
assertNotNull(unnamed, "an unnamed primary must see the question on a delegation another unnamed primary created");
assertEquals("which config file?", unnamed.question());
CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync(
() -> messages.answer(unnamed.turnId(), "config.yaml", 5000, null));
assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer());
awaitWaiting();
assertTrue(rendezvous.resolve(T, "done"));
assertEquals(MessageService.Outcome.REPLIED, answer.get(5, TimeUnit.SECONDS).outcome());
}
/**
* {@code pendingAsk} has no public no-check overload the way {@link MessageService#poll}
* does, so this drives {@link MessageService#INTERNAL_NO_OWNER_CHECK} directly — the only way
* to exercise the bypass for this method.
*/
@Test
void pendingAskInternalBypassSeesAnyDelegationsPendingQuestion() throws Exception {
Principal creator = Principal.worker("term_creator", 1);
String ticket = messages.sendAsync(T, "task that asks", null, creator);
awaitWaiting(); awaitWaiting();
injectDelivery(); injectDelivery();
@@ -1998,8 +2098,34 @@ class MessageServiceTest {
CompletableFuture.supplyAsync(() -> messages.ask(T, "which config file?", 5000)); CompletableFuture.supplyAsync(() -> messages.ask(T, "which config file?", 5000));
MessageService.TaskView asking = awaitTicketPhase(ticket, MessageService.Phase.ASKING); MessageService.TaskView asking = awaitTicketPhase(ticket, MessageService.Phase.ASKING);
assertNull(messages.pendingAsk(T, "term_someone"), MessageService.PendingAsk bypassed = messages.pendingAsk(T, MessageService.INTERNAL_NO_OWNER_CHECK);
"a terminal-bearing caller must not see a question whose task records no creator"); assertNotNull(bypassed, "the internal bypass must see a question on a delegation a named worker created");
assertEquals("which config file?", bypassed.question());
CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync(
() -> messages.answer(asking.turnId(), "config.yaml", 5000, creator.ownerKey()));
assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer());
awaitWaiting();
assertTrue(rendezvous.resolve(T, "done"));
assertEquals(MessageService.Outcome.REPLIED, answer.get(5, TimeUnit.SECONDS).outcome());
}
/**
* A task created with no recorded owner (a short {@code sendAsync} overload) must not hand its
* open question to a caller with an owner key. Only the unnamed primary may still see it.
*/
@Test
void pendingAskDeniesATerminalBearingCallerWhenTheTaskRecordsNoCreator() throws Exception {
String ticket = messages.sendAsync(T, "task that asks");
awaitWaiting();
injectDelivery();
CompletableFuture<MessageService.AskResult> ask =
CompletableFuture.supplyAsync(() -> messages.ask(T, "which config file?", 5000));
MessageService.TaskView asking = awaitTicketPhase(ticket, MessageService.Phase.ASKING);
assertNull(messages.pendingAsk(T, "worker:term_someone"),
"a caller with an owner key must not see a question whose task records no creator");
assertNotNull(messages.pendingAsk(T, null), assertNotNull(messages.pendingAsk(T, null),
"the unnamed primary must still see it even with no recorded creator"); "the unnamed primary must still see it even with no recorded creator");
@@ -2011,6 +2137,118 @@ class MessageServiceTest {
assertEquals(MessageService.Outcome.REPLIED, answer.get(5, TimeUnit.SECONDS).outcome()); assertEquals(MessageService.Outcome.REPLIED, answer.get(5, TimeUnit.SECONDS).outcome());
} }
// --- outstanding(): fleet_handover{open}'s list of open tickets and asks -------------------
@Test
void outstandingReportsOwnedPendingTicketsWithPhases() {
Principal lead = Principal.leader("opus", "term_lead", 1);
String ticket1 = messages.sendAsync(T, "task one", null, lead);
String ticket2 = messages.sendAsync(T, "task two", null, lead);
MessageService.Outstanding outstanding = messages.outstanding(lead.ownerKey());
assertEquals(2, outstanding.tickets().size());
assertTrue(outstanding.tickets().stream().anyMatch(t ->
ticket1.equals(t.ticket()) && t.phase() == MessageService.Phase.PENDING && T.equals(t.target())),
"ticket1 must be reported PENDING: " + outstanding.tickets());
assertTrue(outstanding.tickets().stream().anyMatch(t ->
ticket2.equals(t.ticket()) && t.phase() == MessageService.Phase.PENDING && T.equals(t.target())),
"ticket2 must be reported PENDING: " + outstanding.tickets());
assertTrue(outstanding.asks().isEmpty(), "neither ticket has an open question");
}
@Test
void outstandingReportsAnOpenAsksTurnId() throws Exception {
Principal lead = Principal.leader("opus", "term_lead", 1);
String ticket = messages.sendAsync(T, "task that asks", null, lead);
awaitWaiting();
injectDelivery();
CompletableFuture<MessageService.AskResult> ask =
CompletableFuture.supplyAsync(() -> messages.ask(T, "which config?", 5000));
MessageService.TaskView asking = awaitTicketPhase(ticket, MessageService.Phase.ASKING);
MessageService.Outstanding outstanding = messages.outstanding(lead.ownerKey());
assertEquals(1, outstanding.tickets().size());
assertEquals(MessageService.Phase.ASKING, outstanding.tickets().get(0).phase());
assertEquals(1, outstanding.asks().size());
MessageService.OutstandingAsk openAsk = outstanding.asks().get(0);
assertEquals(ticket, openAsk.ticket());
assertEquals(asking.turnId(), openAsk.turnId());
assertEquals(T, openAsk.workerSession());
CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync(
() -> messages.answer(asking.turnId(), "config.yaml", 5000, lead.ownerKey()));
assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer());
awaitWaiting();
assertTrue(rendezvous.resolve(T, "done"));
assertEquals(MessageService.Outcome.REPLIED, answer.get(5, TimeUnit.SECONDS).outcome());
}
/**
* Positive control: {@code leadA} must still see its own ticket and ask, so {@code leadB}
* seeing neither is the owner filter at work and not {@code outstanding} refusing everyone.
*/
@Test
void outstandingDoesNotLeakAcrossNamedLeads() throws Exception {
Principal leadA = Principal.leader("opus", "term_a", 1);
Principal leadB = Principal.leader("sol", "term_b", 2);
String ticket = messages.sendAsync(T, "task that asks", null, leadA);
awaitWaiting();
injectDelivery();
CompletableFuture<MessageService.AskResult> ask =
CompletableFuture.supplyAsync(() -> messages.ask(T, "which config?", 5000));
MessageService.TaskView asking = awaitTicketPhase(ticket, MessageService.Phase.ASKING);
MessageService.Outstanding seenByA = messages.outstanding(leadA.ownerKey());
assertEquals(1, seenByA.tickets().size(), "lead A must see its own ticket");
assertEquals(1, seenByA.asks().size(), "lead A must see its own open ask");
MessageService.Outstanding seenByB = messages.outstanding(leadB.ownerKey());
assertTrue(seenByB.tickets().isEmpty(), "lead B must not see lead A's ticket");
assertTrue(seenByB.asks().isEmpty(), "lead B must not see lead A's open ask");
CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync(
() -> messages.answer(asking.turnId(), "config.yaml", 5000, leadA.ownerKey()));
assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer());
awaitWaiting();
assertTrue(rendezvous.resolve(T, "done"));
assertEquals(MessageService.Outcome.REPLIED, answer.get(5, TimeUnit.SECONDS).outcome());
}
@Test
void outstandingIsEmptyCollectionsNotNullForACallerWithNothing() {
MessageService.Outstanding outstanding =
messages.outstanding(Principal.leader("opus", "term_lead", 1).ownerKey());
assertNotNull(outstanding.tickets(), "a caller with no delegations still gets a list, not null");
assertNotNull(outstanding.asks(), "a caller with no delegations still gets a list, not null");
assertTrue(outstanding.tickets().isEmpty());
assertTrue(outstanding.asks().isEmpty());
}
/**
* A ticket is destroyed on a timer once it goes terminal ({@link #pruneTerminalTickets}'s TTL
* runs from completion), so it is the one case where carrying the id forward actually matters
* — a still-PENDING ticket is in no such danger, its worker is still running.
*/
@Test
void outstandingReportsACompletedUncollectedTicketWithATerminalPhase() throws Exception {
Principal lead = Principal.leader("opus", "term_lead", 1);
String ticket = messages.sendAsync(T, "long task", null, lead);
awaitWaiting();
injectDelivery();
assertTrue(rendezvous.resolve(T, "async result"), "a reply resolves the async send");
awaitTicketPhase(ticket, MessageService.Phase.DONE);
MessageService.Outstanding outstanding = messages.outstanding(lead.ownerKey());
assertEquals(1, outstanding.tickets().size());
MessageService.OutstandingTicket done = outstanding.tickets().get(0);
assertEquals(ticket, done.ticket());
assertEquals(MessageService.Phase.DONE, done.phase());
assertEquals(T, done.target());
}
// --- fleetd #715: answer() is gated on the caller that owns the turn ----------------------- // --- fleetd #715: answer() is gated on the caller that owns the turn -----------------------
/** /**
@@ -2055,7 +2293,8 @@ class MessageServiceTest {
*/ */
@Test @Test
void aDifferentCallersAnswerIsRefusedForAnAsyncSendDelegationButTheRealOwnerSucceeds() throws Exception { void aDifferentCallersAnswerIsRefusedForAnAsyncSendDelegationButTheRealOwnerSucceeds() throws Exception {
String ticket = messages.sendAsync(T, "task that asks", null, "term_owner"); Principal owner = Principal.worker("term_owner", 1);
String ticket = messages.sendAsync(T, "task that asks", null, owner);
awaitWaiting(); awaitWaiting();
injectDelivery(); injectDelivery();
@@ -2063,7 +2302,8 @@ class MessageServiceTest {
CompletableFuture.supplyAsync(() -> messages.ask(T, "which config file?", 5000)); CompletableFuture.supplyAsync(() -> messages.ask(T, "which config file?", 5000));
MessageService.TaskView asking = awaitTicketPhase(ticket, MessageService.Phase.ASKING); MessageService.TaskView asking = awaitTicketPhase(ticket, MessageService.Phase.ASKING);
MessageService.Reply hijacked = messages.answer(asking.turnId(), "evil.yaml", 500, "term_attacker"); MessageService.Reply hijacked = messages.answer(asking.turnId(), "evil.yaml", 500,
"worker:term_attacker");
assertEquals(MessageService.Outcome.NOT_TURN_OWNER, hijacked.outcome(), assertEquals(MessageService.Outcome.NOT_TURN_OWNER, hijacked.outcome(),
"a caller that did not create this delegation must be refused, not served"); "a caller that did not create this delegation must be refused, not served");
assertFalse(ask.isDone(), "a refused answer must not resolve the worker's blocked fleet_ask"); assertFalse(ask.isDone(), "a refused answer must not resolve the worker's blocked fleet_ask");
@@ -2071,7 +2311,38 @@ class MessageServiceTest {
"a refused answer must not advance the async ticket's phase"); "a refused answer must not advance the async ticket's phase");
CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync( CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync(
() -> messages.answer(asking.turnId(), "config.yaml", 5000, "term_owner")); () -> messages.answer(asking.turnId(), "config.yaml", 5000, owner.ownerKey()));
assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer());
awaitWaiting();
assertTrue(rendezvous.resolve(T, "done"));
assertEquals(MessageService.Outcome.REPLIED, answer.get(5, TimeUnit.SECONDS).outcome());
assertEquals("done", awaitTicketPhase(ticket, MessageService.Phase.DONE).reply());
}
@Test
void namedLeadCanSeeAndAnswerAnAskAfterItsTerminalChangesWhileLeadBIsRefused() throws Exception {
Principal oldLead = Principal.leader("opus", "term_OLD", 1);
Principal newLead = Principal.leader("opus", "term_NEW", 2);
Principal leadB = Principal.leader("sol", "term_SOL", 3);
assertNotEquals(oldLead.terminal(), newLead.terminal(), "the test requires different terminals");
String ticket = messages.sendAsync(T, "task that asks", null, oldLead);
awaitWaiting();
injectDelivery();
CompletableFuture<MessageService.AskResult> ask =
CompletableFuture.supplyAsync(() -> messages.ask(T, "which config file?", 5000));
MessageService.TaskView asking = awaitTicketPhase(ticket, MessageService.Phase.ASKING);
assertNotNull(messages.pendingAsk(T, newLead.ownerKey()),
"the same lead at its new terminal must see the pending ask");
assertNull(messages.pendingAsk(T, leadB.ownerKey()),
"another lead must not see the pending ask");
assertEquals(MessageService.Outcome.NOT_TURN_OWNER,
messages.answer(asking.turnId(), "evil.yaml", 500, leadB.ownerKey()).outcome());
assertFalse(ask.isDone(), "another lead must not resolve the worker's ask");
CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync(
() -> messages.answer(asking.turnId(), "config.yaml", 5000, newLead.ownerKey()));
assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer()); assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer());
awaitWaiting(); awaitWaiting();
assertTrue(rendezvous.resolve(T, "done")); assertTrue(rendezvous.resolve(T, "done"));
@@ -2105,7 +2376,7 @@ class MessageServiceTest {
private PushWiring wireWithPushLoop(int maxReminders, long backoffMs, java.util.function.LongSupplier nowNanos) { private PushWiring wireWithPushLoop(int maxReminders, long backoffMs, java.util.function.LongSupplier nowNanos) {
PrimaryRegistry registry = new PrimaryRegistry(null); PrimaryRegistry registry = new PrimaryRegistry(null);
registry.recordDelegation(T, LEAD); registry.recordDelegation(T, LEAD);
FakeHerdr leadHerdr = new FakeHerdr(); FakeHerdr leadHerdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET);
AgentControl leadAgents = new AgentControl(leadHerdr); AgentControl leadAgents = new AgentControl(leadHerdr);
var scheduler = java.util.concurrent.Executors.newSingleThreadScheduledExecutor(); var scheduler = java.util.concurrent.Executors.newSingleThreadScheduledExecutor();
ReplyPushLoop pushLoop = new ReplyPushLoop(registry, leadAgents, inbox, scheduler, maxReminders, backoffMs); ReplyPushLoop pushLoop = new ReplyPushLoop(registry, leadAgents, inbox, scheduler, maxReminders, backoffMs);
@@ -2142,7 +2413,7 @@ class MessageServiceTest {
java.util.function.LongSupplier nowNanos) { java.util.function.LongSupplier nowNanos) {
PrimaryRegistry registry = new PrimaryRegistry(null); PrimaryRegistry registry = new PrimaryRegistry(null);
registry.recordDelegation(T, LEAD); registry.recordDelegation(T, LEAD);
FakeHerdr leadHerdr = new FakeHerdr(); FakeHerdr leadHerdr = new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET);
AgentControl leadAgents = new AgentControl(leadHerdr); AgentControl leadAgents = new AgentControl(leadHerdr);
ManualScheduler scheduler = new ManualScheduler(); ManualScheduler scheduler = new ManualScheduler();
ReplyPushLoop pushLoop = new ReplyPushLoop(registry, leadAgents, inbox, scheduler, maxReminders, backoffMs); ReplyPushLoop pushLoop = new ReplyPushLoop(registry, leadAgents, inbox, scheduler, maxReminders, backoffMs);
@@ -2401,7 +2672,8 @@ class MessageServiceTest {
var scheduler = java.util.concurrent.Executors.newSingleThreadScheduledExecutor(); var scheduler = java.util.concurrent.Executors.newSingleThreadScheduledExecutor();
// A backoff far longer than the test: the schedule is started but no tick ever fires, so // A backoff far longer than the test: the schedule is started but no tick ever fires, so
// decide() is read directly and nothing here depends on timing. // decide() is read directly and nothing here depends on timing.
ReplyPushLoop pushLoop = new ReplyPushLoop(registry, new AgentControl(new FakeHerdr()), inbox, ReplyPushLoop pushLoop = new ReplyPushLoop(registry,
new AgentControl(new FakeHerdr().detectionText(FakeHerdr.IDLE_PROMPT_CARET)), inbox,
scheduler, 5, 60_000); scheduler, 5, 60_000);
MessageService service = new MessageService(agents, injector, rendezvous, inbox, pushLoop); MessageService service = new MessageService(agents, injector, rendezvous, inbox, pushLoop);
try { try {
@@ -191,21 +191,68 @@ class RendezvousTest {
assertNull(rendezvous.askOwner(t.turnId()), "a closed ask no longer reports an owner"); assertNull(rendezvous.askOwner(t.turnId()), "a closed ask no longer reports an owner");
} }
// ── fleetd #729: per-boot nonce guards turnId against cross-instance reuse ────────────────
@Test
void twoInstancesMintDisjointTurnIds() {
Rendezvous other = new Rendezvous();
Rendezvous.AskTicket fromThis = rendezvous.openAsk(W);
Rendezvous.AskTicket fromOther = other.openAsk(W);
assertNotEquals(fromThis.turnId(), fromOther.turnId(),
"each instance mints its own id space, so even a first ask from each must differ");
}
@Test
void foreignInstanceTurnIdDoesNotResolve() {
Rendezvous other = new Rendezvous();
// `other` must reach the same sequence number as `rendezvous` (two asks each, the first
// closed so the second mints fresh), or this test passes against an empty map instead of
// against a colliding id.
Rendezvous.AskTicket firstFromThis = rendezvous.openAsk(W);
rendezvous.closeAsk(firstFromThis.turnId());
Rendezvous.AskTicket secondFromThis = rendezvous.openAsk(W);
Rendezvous.AskTicket firstFromOther = other.openAsk(W);
other.closeAsk(firstFromOther.turnId());
other.openAsk(W);
// control: the id resolves in the instance that minted it, so a false below cannot be
// explained by broken plumbing — only by the turnId being foreign to `other`.
assertTrue(rendezvous.answerAsk(secondFromThis.turnId(), "answer from this instance"),
"the minting instance must still resolve its own turnId");
assertFalse(other.answerAsk(secondFromThis.turnId(), "answer from other instance"),
"a turnId minted by a different instance must not resolve here");
}
@Test
void openAskStillCoalescesDuplicatesAndStillMintsDistinctIdsPerAsk() {
Rendezvous.AskTicket t1 = rendezvous.openAsk(W);
Rendezvous.AskTicket t2 = rendezvous.openAsk(W);
assertEquals(t1.turnId(), t2.turnId(),
"a second openAsk while one is open still coalesces onto the same turn");
assertFalse(t2.fresh(), "the coalesced ask is still reported as not fresh");
rendezvous.closeAsk(t1.turnId());
Rendezvous.AskTicket t3 = rendezvous.openAsk(W);
assertNotEquals(t1.turnId(), t3.turnId(), "two asks from the same session still get different turnIds");
}
@Test @Test
void ownerPermitsIsFailClosedOnARecordAndThreeStatesAreDistinct() { void ownerPermitsIsFailClosedOnARecordAndThreeStatesAreDistinct() {
assertFalse(Rendezvous.Owner.permits(null, null), assertFalse(Rendezvous.Owner.permits(null, null),
"no owner on record refuses even a caller with no terminal"); "no owner on record refuses even the unnamed primary");
assertFalse(Rendezvous.Owner.permits(null, "term_a"), assertFalse(Rendezvous.Owner.permits(null, "worker:term_a"),
"no owner on record refuses a terminal-bearing caller too"); "no owner on record refuses a caller with an owner key too");
assertTrue(Rendezvous.Owner.permits(Rendezvous.Owner.UNNAMED_PRIMARY, null), assertTrue(Rendezvous.Owner.permits(Rendezvous.Owner.UNNAMED_PRIMARY, null),
"the unnamed primary owner matches a caller with no terminal"); "the recorded unnamed primary matches a caller with a null owner key");
assertFalse(Rendezvous.Owner.permits(Rendezvous.Owner.UNNAMED_PRIMARY, "term_a"), assertFalse(Rendezvous.Owner.permits(Rendezvous.Owner.UNNAMED_PRIMARY, "worker:term_a"),
"the unnamed primary owner does not match a terminal-bearing caller"); "the recorded unnamed primary does not match another owner key");
assertTrue(Rendezvous.Owner.permits(Rendezvous.Owner.of("term_a"), "term_a"), assertTrue(Rendezvous.Owner.permits(Rendezvous.Owner.of("worker:term_a"), "worker:term_a"),
"a named owner matches the same terminal"); "an owner matches the same key");
assertFalse(Rendezvous.Owner.permits(Rendezvous.Owner.of("term_a"), "term_b"), assertFalse(Rendezvous.Owner.permits(Rendezvous.Owner.of("worker:term_a"), "worker:term_b"),
"a named owner refuses a different terminal"); "an owner refuses a different key");
assertFalse(Rendezvous.Owner.permits(Rendezvous.Owner.of("term_a"), null), assertFalse(Rendezvous.Owner.permits(Rendezvous.Owner.of("worker:term_a"), null),
"a named owner refuses the unnamed primary"); "a named owner refuses the unnamed primary");
} }
} }
@@ -3,6 +3,7 @@ package dev.ltms.fleet.msg;
import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.ObjectMapper;
import dev.ltms.fleet.herdr.AgentControl; import dev.ltms.fleet.herdr.AgentControl;
import dev.ltms.fleet.herdr.FakeHerdr;
import dev.ltms.fleet.herdr.HerdrClient; import dev.ltms.fleet.herdr.HerdrClient;
import dev.ltms.fleet.herdr.HerdrException; import dev.ltms.fleet.herdr.HerdrException;
import dev.ltms.fleet.mcp.PrimaryRegistry; import dev.ltms.fleet.mcp.PrimaryRegistry;
@@ -304,6 +305,40 @@ class ReplyPushLoopTest {
+ "still live"); + "still live");
} }
// --- fleetd #737 unit 3: the fallback nudgeTargetFor returns must be probed too --------------
/**
* {@code resolveLiveLead} forgets a dead per-target binding and asks {@code PrimaryRegistry}
* again for a fallback. That fallback can be dead too — here PRIMARY, the pinned singleton, is
* affirmatively gone alongside DEAD_LEAD. A correct {@code resolveLiveLead} probes it exactly
* like the first lead and gives up for this tick rather than trust it unchecked.
*
* <p>This is checked through {@code onReplyQueued}/{@code isActive} rather than a send count:
* {@link ReplyPushLoop#onReplyQueued} only registers pending work and starts a schedule once
* {@code resolveLiveLead} returns a present value — a dead fallback that was trusted unprobed
* would already make this true, synchronously, with no tick or send needed to observe it. Pairs
* with {@link #aStaleLeadBindingFallsBackToTheLiveLeadInsteadOfNudgingADeadTerminal} as the
* positive control: same stale-DEAD_LEAD setup, but there the fallback (PRIMARY) is live and the
* nudge does fire — proving this test's "nothing happens" result comes from the fallback being
* dead, not from the assertion being unable to observe a nudge at all.
*/
@Test
void aDoublyDeadFallbackIsNeverTrustedAndStartsNoSchedule() {
registry.recordDelegation(WORKER, DEAD_LEAD);
var rec = new AllDeadHerdrClient(Set.of(DEAD_LEAD, PRIMARY));
agents = new AgentControl(rec);
inbox.publish(WORKER, "m1", "hello");
var loop = loop(1, 50);
loop.onReplyQueued(WORKER);
assertFalse(loop.isActive(),
"both the per-target binding and the fallback are dead, so resolveLiveLead must "
+ "return empty and onReplyQueued must never register pending work or start "
+ "a schedule — an unprobed fallback would start one here");
assertEquals(0, rec.promptTargets().size(), "nobody live was found, so nothing was ever sent");
}
// --- nudge format -------------------------------------------------------------------------- // --- nudge format --------------------------------------------------------------------------
@Test @Test
@@ -1045,6 +1080,76 @@ class ReplyPushLoopTest {
"hitting the ticket reminder cap must count as exhausted"); "hitting the ticket reminder cap must count as exhausted");
} }
// --- the operator's own prompt box ----------------------------------------------------------
@Test
void aLeadWithUnsubmittedTextInItsPromptBoxIsNotNudged() {
var herdr = new PaneTextHerdrClient(FakeHerdr.DRAFTED_PROMPT_CARET);
agents = new AgentControl(herdr);
inbox.publish(WORKER, "m1", "hello");
var loop = loop(5, 100_000);
loop.onReplyQueued(WORKER);
assertEquals(ReplyPushLoop.Action.WAIT_BUSY, loop.decide(PRIMARY, 0, 0),
"a nudge pastes and submits, so an idle lead mid-sentence must not be nudged");
assertEquals(0, herdr.promptCount(), "nothing reached the pane");
assertFalse(inbox.peek(WORKER).isEmpty(), "and the reply is still waiting to be collected");
}
@Test
void theSameLeadIsNudgedOnceItsPromptBoxIsEmpty() {
var herdr = new PaneTextHerdrClient(FakeHerdr.DRAFTED_PROMPT_CARET);
agents = new AgentControl(herdr);
inbox.publish(WORKER, "m1", "hello");
var loop = loop(5, 100_000);
loop.onReplyQueued(WORKER);
assertEquals(ReplyPushLoop.Action.WAIT_BUSY, loop.decide(PRIMARY, 0, 0));
herdr.paneText(FakeHerdr.IDLE_PROMPT_CARET);
assertEquals(ReplyPushLoop.Action.INJECT, loop.decide(PRIMARY, 0, 0),
"the box emptied, so the held nudge is due");
}
@Test
void anUnrecognisablePaneHoldsTheNudge() {
var herdr = new PaneTextHerdrClient("garbled ansi noise with no input box");
agents = new AgentControl(herdr);
inbox.publish(WORKER, "m1", "hello");
var loop = loop(5, 100_000);
loop.onReplyQueued(WORKER);
assertEquals(ReplyPushLoop.Action.WAIT_BUSY, loop.decide(PRIMARY, 0, 0),
"a pane whose box cannot be found may be holding a draft");
assertEquals(0, herdr.promptCount());
}
@Test
void aFailedPaneReadHoldsTheNudge() {
var herdr = new PaneTextHerdrClient(FakeHerdr.IDLE_PROMPT_CARET).failReads();
agents = new AgentControl(herdr);
inbox.publish(WORKER, "m1", "hello");
var loop = loop(5, 100_000);
loop.onReplyQueued(WORKER);
assertEquals(ReplyPushLoop.Action.WAIT_BUSY, loop.decide(PRIMARY, 0, 0),
"an unreadable box is treated as a draft, never as an empty one");
assertEquals(0, herdr.promptCount());
}
@Test
void aNudgeHeldForADraftIsSentOnALaterTick() throws Exception {
var herdr = new PaneTextHerdrClient(FakeHerdr.DRAFTED_PROMPT_CARET);
agents = new AgentControl(herdr);
loop(5, 50).onTicketTerminal("task-1", WORKER, false);
Thread.sleep(300);
assertEquals(0, herdr.promptCount(), "every tick holds while the operator is typing");
herdr.paneText(FakeHerdr.IDLE_PROMPT_CARET);
assertTrue(herdr.sendLatch.await(3, TimeUnit.SECONDS),
"the nudge lands on the first tick after the box empties");
}
// --- helpers ------------------------------------------------------------------------------- // --- helpers -------------------------------------------------------------------------------
private ReplyPushLoop loop() { private ReplyPushLoop loop() {
@@ -1063,6 +1168,15 @@ class ReplyPushLoopTest {
return new AgentControl(new FakeHerdrClient(status)); return new AgentControl(new FakeHerdrClient(status));
} }
/**
* The {@code agent.read} frame every fake here returns: a lead settled at an empty input box. The
* loop reads the box before it nudges, so a fake that answered nothing would read as a pane it
* cannot classify and hold every nudge.
*/
private static JsonNode emptyPromptBoxRead() {
return MAPPER.createObjectNode().set("read", MAPPER.createObjectNode().put("text", FakeHerdr.IDLE_PROMPT_CARET));
}
/** Non-recording (single-threaded) fake — safe for decide() tests. */ /** Non-recording (single-threaded) fake — safe for decide() tests. */
private static final class FakeHerdrClient implements HerdrClient { private static final class FakeHerdrClient implements HerdrClient {
private final String agentStatus; private final String agentStatus;
@@ -1079,6 +1193,9 @@ class ReplyPushLoopTest {
.put("terminal_id", PRIMARY) .put("terminal_id", PRIMARY)
.put("agent_status", agentStatus)); .put("agent_status", agentStatus));
} }
if ("agent.read".equals(method)) {
return emptyPromptBoxRead();
}
return MAPPER.createObjectNode(); return MAPPER.createObjectNode();
} }
@@ -1108,6 +1225,9 @@ class ReplyPushLoopTest {
calls.add(Map.entry(method, params)); calls.add(Map.entry(method, params));
sendLatch.countDown(); sendLatch.countDown();
} }
if ("agent.read".equals(method)) {
return emptyPromptBoxRead();
}
return MAPPER.createObjectNode(); return MAPPER.createObjectNode();
} }
@@ -1145,6 +1265,9 @@ class ReplyPushLoopTest {
if ("agent.prompt".equals(method)) { if ("agent.prompt".equals(method)) {
sendLatch.countDown(); sendLatch.countDown();
} }
if ("agent.read".equals(method)) {
return emptyPromptBoxRead();
}
return MAPPER.createObjectNode(); return MAPPER.createObjectNode();
} }
@@ -1182,6 +1305,9 @@ class ReplyPushLoopTest {
calls.add(Map.entry(method, params)); calls.add(Map.entry(method, params));
sendLatch.countDown(); sendLatch.countDown();
} }
if ("agent.read".equals(method)) {
return emptyPromptBoxRead();
}
return MAPPER.createObjectNode(); return MAPPER.createObjectNode();
} }
@@ -1232,6 +1358,9 @@ class ReplyPushLoopTest {
promptTargets.add(String.valueOf(p.get("target"))); promptTargets.add(String.valueOf(p.get("target")));
sendLatch.countDown(); sendLatch.countDown();
} }
if ("agent.read".equals(method)) {
return emptyPromptBoxRead();
}
return MAPPER.createObjectNode(); return MAPPER.createObjectNode();
} }
@@ -1248,6 +1377,55 @@ class ReplyPushLoopTest {
} }
} }
/**
* Fake herdr client for fleetd #737 unit 3: every terminal named in {@code deadTargets} reports
* {@code agent_not_found} from {@code agent.get} — unlike {@link DeadLeadHerdrClient}, which can
* only make one terminal dead, this can make a per-target binding AND its fallback dead in the
* same test. {@code agent.prompt} is recorded unconditionally (no liveness check of its own),
* so a test can tell "resolveLiveLead probed and correctly found nobody live" (no prompt call)
* apart from "resolveLiveLead trusted a dead fallback and sent into it anyway" (a prompt call to
* a terminal this fake has already declared gone).
*/
private static final class AllDeadHerdrClient implements HerdrClient {
private final Set<String> deadTargets;
private final List<String> promptTargets = Collections.synchronizedList(new ArrayList<>());
AllDeadHerdrClient(Set<String> deadTargets) {
this.deadTargets = deadTargets;
}
@Override
@SuppressWarnings("unchecked")
public JsonNode call(String method, Object params) {
Map<String, Object> p = params instanceof Map ? (Map<String, Object>) params : Map.of();
if ("agent.get".equals(method)) {
String target = String.valueOf(p.get("target"));
if (deadTargets.contains(target)) {
throw new HerdrException("no such agent: " + target, "agent_not_found", null);
}
return MAPPER.createObjectNode()
.set("agent", MAPPER.createObjectNode()
.put("terminal_id", target)
.put("agent_status", "idle"));
}
if ("agent.prompt".equals(method)) {
promptTargets.add(String.valueOf(p.get("target")));
}
if ("agent.read".equals(method)) {
return emptyPromptBoxRead();
}
return MAPPER.createObjectNode();
}
List<String> promptTargets() {
return List.copyOf(promptTargets);
}
@Override
public void close() {
}
}
/** /**
* Fake herdr client for fleetd #368 review: {@code flakyTarget}'s FIRST {@code agent.get} call * Fake herdr client for fleetd #368 review: {@code flakyTarget}'s FIRST {@code agent.get} call
* fails with a transient, non-{@code agent_not_found} {@code HerdrException} — a transport-level * fails with a transient, non-{@code agent_not_found} {@code HerdrException} — a transport-level
@@ -1283,6 +1461,9 @@ class ReplyPushLoopTest {
promptTargets.add(String.valueOf(p.get("target"))); promptTargets.add(String.valueOf(p.get("target")));
sendLatch.countDown(); sendLatch.countDown();
} }
if ("agent.read".equals(method)) {
return emptyPromptBoxRead();
}
return MAPPER.createObjectNode(); return MAPPER.createObjectNode();
} }
@@ -1294,4 +1475,60 @@ class ReplyPushLoopTest {
public void close() { public void close() {
} }
} }
/**
* Thread-safe fake that always reports {@code idle} and serves a mutable pane tail, so a test can
* change what the lead's input box holds between ticks. Records every {@code agent.prompt}.
*/
private static final class PaneTextHerdrClient implements HerdrClient {
private final List<Map.Entry<String, Object>> prompts =
Collections.synchronizedList(new ArrayList<>());
private volatile String paneText;
private volatile boolean failReads = false;
volatile CountDownLatch sendLatch = new CountDownLatch(1);
PaneTextHerdrClient(String paneText) {
this.paneText = paneText;
}
void paneText(String text) {
this.paneText = text;
}
PaneTextHerdrClient failReads() {
this.failReads = true;
return this;
}
int promptCount() {
return prompts.size();
}
@Override
public JsonNode call(String method, Object params) {
if ("agent.get".equals(method)) {
return MAPPER.createObjectNode()
.set("agent", MAPPER.createObjectNode()
.put("terminal_id", PRIMARY)
.put("agent_status", "idle"));
}
if ("agent.read".equals(method)) {
if (failReads) {
throw new HerdrException("herdr socket read timed out");
}
return MAPPER.createObjectNode()
.set("read", MAPPER.createObjectNode().put("text", paneText));
}
if ("agent.prompt".equals(method)) {
prompts.add(Map.entry(method, params));
sendLatch.countDown();
}
return MAPPER.createObjectNode();
}
@Override
public void close() {
}
}
} }
@@ -8,6 +8,7 @@ import dev.ltms.fleet.auth.CallerResolver;
import dev.ltms.fleet.auth.Authz; import dev.ltms.fleet.auth.Authz;
import dev.ltms.fleet.auth.MemberRegistry; import dev.ltms.fleet.auth.MemberRegistry;
import dev.ltms.fleet.auth.Principal; import dev.ltms.fleet.auth.Principal;
import dev.ltms.fleet.auth.Role;
import dev.ltms.fleet.config.FleetConfig; import dev.ltms.fleet.config.FleetConfig;
import dev.ltms.fleet.guard.SubscriptionGuard; import dev.ltms.fleet.guard.SubscriptionGuard;
import dev.ltms.fleet.herdr.AgentControl; import dev.ltms.fleet.herdr.AgentControl;
@@ -20,6 +21,7 @@ import dev.ltms.fleet.metrics.FleetMetrics;
import dev.ltms.fleet.metrics.Metrics; import dev.ltms.fleet.metrics.Metrics;
import dev.ltms.fleet.msg.MessageService; import dev.ltms.fleet.msg.MessageService;
import dev.ltms.fleet.msg.Rendezvous; import dev.ltms.fleet.msg.Rendezvous;
import dev.ltms.fleet.peer.MemberRole;
import dev.ltms.fleet.session.FakeWorktrees; import dev.ltms.fleet.session.FakeWorktrees;
import dev.ltms.fleet.session.SessionManager; import dev.ltms.fleet.session.SessionManager;
import dev.ltms.fleet.member.ClaudeCodeLauncher; import dev.ltms.fleet.member.ClaudeCodeLauncher;
@@ -89,8 +91,13 @@ class FleetAppAuthTest {
MessageService messages = new MessageService(agents, injector, new Rendezvous()); MessageService messages = new MessageService(agents, injector, new Rendezvous());
ConnectionIdentity identity = new ConnectionIdentity(new PaneLocator(herdr), _ -> pid); ConnectionIdentity identity = new ConnectionIdentity(new PaneLocator(herdr), _ -> pid);
// term_a is the only herdr-owned pane this fixture's PID can resolve to (FakeHerdr's canned
// pane list), and this helper's own contract above says that pane is the worker -- so it
// must be recognised as a live spawned member here, the same way a real roster would,
// rather than falling through to the observer floor.
CallerResolver callers = CallerResolver.withLeadsAndMembers(identity, tokenMode, token, CallerResolver callers = CallerResolver.withLeadsAndMembers(identity, tokenMode, token,
Map::of, new MemberRegistry(null)); Map::of, new MemberRegistry(null), t -> "term_a".equals(t) ? MemberRole.DEV : null,
Map::of);
metrics = FleetMetrics.create(sessions, new dev.ltms.fleet.msg.InMemoryReplyInbox()); metrics = FleetMetrics.create(sessions, new dev.ltms.fleet.msg.InMemoryReplyInbox());
app = new FleetApp(herdr, workers, sessions, messages, sessions.asPresence(), null, app = new FleetApp(herdr, workers, sessions, messages, sessions.asPresence(), null,
@@ -146,11 +153,11 @@ class FleetAppAuthTest {
/** /**
* {@code GET /tasks/{ticket}} must resolve its caller the same way {@code allow(...)} does * {@code GET /tasks/{ticket}} must resolve its caller the same way {@code allow(...)} does
* and thread that terminal into {@link MessageService#poll(String, String)}, not the * and thread that owner key into {@link MessageService#poll(String, String)}, not the
* no-check overload that ignores who is asking. * no-check overload that ignores who is asking.
*/ */
@Test @Test
void theTaskStatusRouteActuallyThreadsTheCallersTerminalIntoPoll() throws Exception { void theTaskStatusRouteActuallyThreadsTheCallersOwnerKeyIntoPoll() throws Exception {
String source = Files.readString(REST_SOURCE); String source = Files.readString(REST_SOURCE);
int start = source.indexOf("private void taskStatus(Context ctx) {"); int start = source.indexOf("private void taskStatus(Context ctx) {");
@@ -166,8 +173,8 @@ class FleetAppAuthTest {
"control failed: the scraped taskStatus block contains no messages.poll( call at all " "control failed: the scraped taskStatus block contains no messages.poll( call at all "
+ "-- the anchors have drifted, this test is not testing what it claims to"); + "-- the anchors have drifted, this test is not testing what it claims to");
assertTrue(handlerBlock.contains("caller.terminal()"), assertTrue(handlerBlock.contains("caller.ownerKey()"),
"the taskStatus route must thread the resolved caller's terminal into messages.poll(...), " "the taskStatus route must thread the resolved caller's owner key into messages.poll(...), "
+ "not the no-check overload -- block: " + handlerBlock); + "not the no-check overload -- block: " + handlerBlock);
assertTrue(handlerBlock.contains("ctx.attribute(CALLER)"), assertTrue(handlerBlock.contains("ctx.attribute(CALLER)"),
"the taskStatus route must resolve its caller the same way allow(...) does, not via a " "the taskStatus route must resolve its caller the same way allow(...) does, not via a "
@@ -176,11 +183,11 @@ class FleetAppAuthTest {
/** /**
* {@code GET /sessions/{id}/status} must resolve its caller the same way {@code allow(...)} * {@code GET /sessions/{id}/status} must resolve its caller the same way {@code allow(...)}
* does and thread that terminal into {@link MessageService#pendingAsk(String, String)}, not * does and thread that owner key into {@link MessageService#pendingAsk(String, String)}, not
* the no-check overload that ignores who is asking. * the no-check overload that ignores who is asking.
*/ */
@Test @Test
void theSessionStatusRouteActuallyThreadsTheCallersTerminalIntoPendingAsk() throws Exception { void theSessionStatusRouteActuallyThreadsTheCallersOwnerKeyIntoPendingAsk() throws Exception {
String source = Files.readString(REST_SOURCE); String source = Files.readString(REST_SOURCE);
int start = source.indexOf("private void sessionStatus(Context ctx) {"); int start = source.indexOf("private void sessionStatus(Context ctx) {");
@@ -196,8 +203,8 @@ class FleetAppAuthTest {
"control failed: the scraped sessionStatus block contains no messages.pendingAsk( " "control failed: the scraped sessionStatus block contains no messages.pendingAsk( "
+ "call at all -- the anchors have drifted, this test is not testing what it claims to"); + "call at all -- the anchors have drifted, this test is not testing what it claims to");
assertTrue(handlerBlock.contains("caller.terminal()"), assertTrue(handlerBlock.contains("caller.ownerKey()"),
"the sessionStatus route must thread the resolved caller's terminal into " "the sessionStatus route must thread the resolved caller's owner key into "
+ "messages.pendingAsk(...), not the no-check overload -- block: " + handlerBlock); + "messages.pendingAsk(...), not the no-check overload -- block: " + handlerBlock);
assertTrue(handlerBlock.contains("ctx.attribute(CALLER)"), assertTrue(handlerBlock.contains("ctx.attribute(CALLER)"),
"the sessionStatus route must resolve its caller the same way allow(...) does, not via " "the sessionStatus route must resolve its caller the same way allow(...) does, not via "
@@ -205,14 +212,15 @@ class FleetAppAuthTest {
} }
/** /**
* {@code GET /tasks/{ticket}} refuses a worker whose terminal did not create the ticket, * {@code GET /tasks/{ticket}} refuses a worker whose terminal did not create the ticket, and
* while the creating worker and the unnamed primary both still read it. The ticket is minted * refuses an unnamed primary just the same: a named worker's ticket is not anyone else's to
* directly on the shared {@link MessageService}, the same way {@code MessageServiceTest} * read, caller rank included. The ticket is minted directly on the shared
* drives {@link MessageService#poll(String, String)}, so this exercises only the REST poll * {@link MessageService}, the same way {@code MessageServiceTest} drives
* route's own handling of the ownership already recorded on the ticket. * {@link MessageService#poll(String, String)}, so this exercises only the REST poll route's
* own handling of the ownership already recorded on the ticket.
*/ */
@Test @Test
void restPollRefusesADifferentWorkerButAllowsTheCreatorAndTheUnnamedPrimary() throws Exception { void restPollRefusesADifferentWorkerAndAnUnnamedPrimaryOnANamedWorkersTicket() throws Exception {
FakeHerdr herdr = new FakeHerdr(); FakeHerdr herdr = new FakeHerdr();
AgentControl agents = new AgentControl(herdr); AgentControl agents = new AgentControl(herdr);
Injector injector = new Injector(agents); Injector injector = new Injector(agents);
@@ -222,7 +230,8 @@ class FleetAppAuthTest {
Javalin otherWorkerApp = startOnSharedService(messages, herdr, 9001L); // -> term_shell Javalin otherWorkerApp = startOnSharedService(messages, herdr, 9001L); // -> term_shell
Javalin primaryApp = startOnSharedService(messages, herdr, 999_999L); // no pane -> primary Javalin primaryApp = startOnSharedService(messages, herdr, 999_999L); // no pane -> primary
try { try {
String ticket = messages.sendAsync("term_a", "long task", null, "term_a"); String ticket = messages.sendAsync("term_a", "long task", null,
Principal.worker("term_a", FakeHerdr.WORKER_PID));
HttpResponse<String> refused = send(otherWorkerApp.port(), "GET", "/tasks/" + ticket, null, null); HttpResponse<String> refused = send(otherWorkerApp.port(), "GET", "/tasks/" + ticket, null, null);
assertEquals(200, refused.statusCode()); assertEquals(200, refused.statusCode());
@@ -238,8 +247,10 @@ class FleetAppAuthTest {
HttpResponse<String> primary = send(primaryApp.port(), "GET", "/tasks/" + ticket, null, null); HttpResponse<String> primary = send(primaryApp.port(), "GET", "/tasks/" + ticket, null, null);
assertEquals(200, primary.statusCode()); assertEquals(200, primary.statusCode());
assertFalse(primary.body().contains("forbidden"), assertTrue(primary.body().contains("forbidden"),
"the unnamed primary must read any ticket: " + primary.body()); "an unnamed primary must not read a ticket a named worker created: " + primary.body());
assertFalse(primary.body().contains("\"reply\""),
"a refusal must never carry reply text: " + primary.body());
} finally { } finally {
creatorApp.stop(); creatorApp.stop();
otherWorkerApp.stop(); otherWorkerApp.stop();
@@ -247,6 +258,33 @@ class FleetAppAuthTest {
} }
} }
/**
* Positive control for
* {@link #restPollRefusesADifferentWorkerAndAnUnnamedPrimaryOnANamedWorkersTicket}: without
* this, that test's refusal would pass just as well if the route refused every caller. Here
* the ticket's creator is itself an unnamed primary, so another unnamed primary reading it
* over REST must still succeed.
*/
@Test
void restPollAllowsAnUnnamedPrimaryItsOwnTicket() throws Exception {
FakeHerdr herdr = new FakeHerdr();
AgentControl agents = new AgentControl(herdr);
Injector injector = new Injector(agents);
MessageService messages = new MessageService(agents, injector, new Rendezvous());
Javalin primaryApp = startOnSharedService(messages, herdr, 999_999L); // no pane -> primary
try {
String ticket = messages.sendAsync("term_a", "long task", null, Principal.primary(FakeHerdr.WORKER_PID));
HttpResponse<String> own = send(primaryApp.port(), "GET", "/tasks/" + ticket, null, null);
assertEquals(200, own.statusCode());
assertFalse(own.body().contains("forbidden"),
"an unnamed primary must read a ticket another unnamed primary created: " + own.body());
} finally {
primaryApp.stop();
}
}
/** /**
* {@code POST /sessions/{id}/message} with {@code wait:false} must record the creating * {@code POST /sessions/{id}/message} with {@code wait:false} must record the creating
* caller's own terminal on the ticket it returns, so that caller can still poll its own * caller's own terminal on the ticket it returns, so that caller can still poll its own
@@ -287,12 +325,13 @@ class FleetAppAuthTest {
/** /**
* {@code GET /sessions/{id}/status} shows a worker's pending {@code fleet_ask} question, its * {@code GET /sessions/{id}/status} shows a worker's pending {@code fleet_ask} question, its
* {@code turnId} and its ticket only to the caller whose terminal created that delegation, or * {@code turnId} and its ticket only to the caller whose owner key created that delegation. An
* to a caller with no terminal at all (the unnamed primary) — a different terminal-bearing * unnamed primary is held to the same rule: its owner key is {@code null}, which here does not
* caller still sees the base status line, but none of the pending-ask fields. * match the named worker that created the delegation, so it sees none of the pending-ask
* fields either — the same as any other non-creating caller.
*/ */
@Test @Test
void restStatusGatesThePendingAskFieldsByTheDelegationsCreatorTerminal() throws Exception { void restStatusGatesThePendingAskFieldsByTheDelegationsCreatorOwner() throws Exception {
FakeHerdr herdr = new FakeHerdr(); FakeHerdr herdr = new FakeHerdr();
AgentControl agents = new AgentControl(herdr); AgentControl agents = new AgentControl(herdr);
Injector injector = new Injector(agents); Injector injector = new Injector(agents);
@@ -304,7 +343,8 @@ class FleetAppAuthTest {
Javalin primaryApp = startOnSharedService(messages, herdr, 999_999L); // no pane -> primary Javalin primaryApp = startOnSharedService(messages, herdr, 999_999L); // no pane -> primary
try { try {
ObjectMapper mapper = new ObjectMapper(); ObjectMapper mapper = new ObjectMapper();
String ticket = messages.sendAsync("term_target", "task that asks", null, "term_a"); String ticket = messages.sendAsync("term_target", "task that asks", null,
Principal.worker("term_a", FakeHerdr.WORKER_PID));
long deadline = System.currentTimeMillis() + 3000; long deadline = System.currentTimeMillis() + 3000;
while (!rendezvous.isWaiting("term_target") && System.currentTimeMillis() < deadline) { while (!rendezvous.isWaiting("term_target") && System.currentTimeMillis() < deadline) {
Thread.sleep(5); Thread.sleep(5);
@@ -317,7 +357,8 @@ class FleetAppAuthTest {
MessageService.TaskView asking; MessageService.TaskView asking;
deadline = System.currentTimeMillis() + 3000; deadline = System.currentTimeMillis() + 3000;
do { do {
asking = messages.poll(ticket, null); // the no-check overload: this is test plumbing waiting for ASKING, not the gate under test
asking = messages.poll(ticket);
Thread.sleep(5); Thread.sleep(5);
} while (asking.phase() != MessageService.Phase.ASKING && System.currentTimeMillis() < deadline); } while (asking.phase() != MessageService.Phase.ASKING && System.currentTimeMillis() < deadline);
assertEquals(MessageService.Phase.ASKING, asking.phase()); assertEquals(MessageService.Phase.ASKING, asking.phase());
@@ -338,12 +379,15 @@ class FleetAppAuthTest {
JsonNode primary = mapper.readTree( JsonNode primary = mapper.readTree(
send(primaryApp.port(), "GET", "/sessions/term_target/status", null, null).body()); send(primaryApp.port(), "GET", "/sessions/term_target/status", null, null).body());
assertEquals("which config file?", primary.get("question").asText(), assertEquals("idle", primary.get("status").asText(), "the base status must still be shown");
"a caller with no terminal (the unnamed primary) must see the question"); assertFalse(primary.has("question"),
"an unnamed primary must not see a question on a delegation a named worker created: " + primary);
assertFalse(primary.has("turnId"), "a non-creating unnamed primary must not see the turnId: " + primary);
assertFalse(primary.has("ticket"), "a non-creating unnamed primary must not see the ticket: " + primary);
// Clean up the still-open ask so the background thread does not linger past the test. // Clean up the still-open ask so the background thread does not linger past the test.
CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync( CompletableFuture<MessageService.Reply> answer = CompletableFuture.supplyAsync(
() -> messages.answer(turnId, "config.yaml", 5000, "term_a")); () -> messages.answer(turnId, "config.yaml", 5000, "worker:term_a"));
assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer()); assertEquals("config.yaml", ask.get(5, TimeUnit.SECONDS).answer());
deadline = System.currentTimeMillis() + 3000; deadline = System.currentTimeMillis() + 3000;
while (!rendezvous.isWaiting("term_target") && System.currentTimeMillis() < deadline) { while (!rendezvous.isWaiting("term_target") && System.currentTimeMillis() < deadline) {
@@ -394,7 +438,8 @@ class FleetAppAuthTest {
MessageService.TaskView asking; MessageService.TaskView asking;
deadline = System.currentTimeMillis() + 3000; deadline = System.currentTimeMillis() + 3000;
do { do {
asking = messages.poll(ticket, null); // the no-check overload: this is test plumbing waiting for ASKING, not the gate under test
asking = messages.poll(ticket);
Thread.sleep(5); Thread.sleep(5);
} while (asking.phase() != MessageService.Phase.ASKING && System.currentTimeMillis() < deadline); } while (asking.phase() != MessageService.Phase.ASKING && System.currentTimeMillis() < deadline);
assertEquals(MessageService.Phase.ASKING, asking.phase()); assertEquals(MessageService.Phase.ASKING, asking.phase());
@@ -520,6 +565,11 @@ class FleetAppAuthTest {
* As {@link #startOnSharedService(MessageService, FakeHerdr, long)}, but {@code leadTerminals} * As {@link #startOnSharedService(MessageService, FakeHerdr, long)}, but {@code leadTerminals}
* resolves the given pid's terminal to a named lead (a caller with SEND permission) instead of * resolves the given pid's terminal to a named lead (a caller with SEND permission) instead of
* a plain worker, for a test that needs a terminal-bearing caller able to create a ticket. * a plain worker, for a test that needs a terminal-bearing caller able to create a ticket.
*
* <p>Every connecting pane not already claimed by {@code leadTerminals} is wired into the live
* roster as a spawned worker, so a caller's resolved role matches what its own test expects:
* a {@link Role#WORKER}, never the unconfigured-pane {@link Role#OBSERVER} floor a roster-less
* resolver would otherwise fall to.
*/ */
private Javalin startOnSharedService(MessageService messages, FakeHerdr herdr, long pid, private Javalin startOnSharedService(MessageService messages, FakeHerdr herdr, long pid,
Map<String, String> leadTerminals) { Map<String, String> leadTerminals) {
@@ -534,7 +584,8 @@ class FleetAppAuthTest {
SessionManager sessions = new SessionManager(workers, new FakeWorktrees()); SessionManager sessions = new SessionManager(workers, new FakeWorktrees());
ConnectionIdentity identity = new ConnectionIdentity(new PaneLocator(herdr), _ -> pid); ConnectionIdentity identity = new ConnectionIdentity(new PaneLocator(herdr), _ -> pid);
CallerResolver callers = CallerResolver.withLeadsAndMembers(identity, false, null, CallerResolver callers = CallerResolver.withLeadsAndMembers(identity, false, null,
() -> leadTerminals, new MemberRegistry(null)); () -> leadTerminals, new MemberRegistry(null),
t -> leadTerminals.containsKey(t) ? null : MemberRole.DEV, Map::of);
Metrics appMetrics = FleetMetrics.create(sessions, new dev.ltms.fleet.msg.InMemoryReplyInbox()); Metrics appMetrics = FleetMetrics.create(sessions, new dev.ltms.fleet.msg.InMemoryReplyInbox());
return new FleetApp(herdr, workers, sessions, messages, sessions.asPresence(), null, return new FleetApp(herdr, workers, sessions, messages, sessions.asPresence(), null,
@@ -223,6 +223,37 @@ class FleetAppTest {
assertTrue(body.has("detail"), res.body()); assertTrue(body.has("detail"), res.body());
} }
@Test
void agentsReportsTheAgentsTabLabel() throws Exception {
FakeHerdr herdr = new FakeHerdr().withTab("w2", "w2:t7", "trinotes");
int port = start(herdr, "http://gx00.gw:8000", Set.of("gx00.gw"));
HttpResponse<String> res = req(port, "GET", "/agents");
assertEquals(200, res.statusCode(), res.body());
JsonNode agents = mapper.readTree(res.body()).get("agents");
assertEquals(1, agents.size());
assertEquals("sess-1111", agents.get(0).get("sessionId").asText());
assertEquals("trinotes", agents.get(0).get("label").asText());
}
/**
* The tab-label scan ({@code workspace.list}/{@code tab.list}) is decoration on top of
* {@code workers.list()}'s own agent roster, so its failure must not cost that roster: a row
* reports a {@code null} label instead, never the {@code herdr_error} envelope.
*/
@Test
void agentsStillReportsTheRosterWhenTheLabelScanFails() throws Exception {
FakeHerdr herdr = new FakeHerdr().workspaceListFailsWith("unavailable");
int port = start(herdr, "http://gx00.gw:8000", Set.of("gx00.gw"));
HttpResponse<String> res = req(port, "GET", "/agents");
assertEquals(200, res.statusCode(), res.body());
JsonNode agents = mapper.readTree(res.body()).get("agents");
assertEquals(1, agents.size());
assertEquals("sess-1111", agents.get(0).get("sessionId").asText());
assertTrue(agents.get(0).get("label").isNull(), "a failed label scan must report a null label, not fail the roster: " + res.body());
}
@Test @Test
void spawnWorkerLandsInOwnTabInWorkerSpaceAndInjectsBaseUrl() throws Exception { void spawnWorkerLandsInOwnTabInWorkerSpaceAndInjectsBaseUrl() throws Exception {
FakeHerdr herdr = new FakeHerdr(); FakeHerdr herdr = new FakeHerdr();
@@ -0,0 +1,92 @@
package dev.ltms.fleet.session;
import dev.ltms.fleet.inject.MemberPresence;
import dev.ltms.fleet.peer.Capability;
import dev.ltms.fleet.peer.PeerHandle;
import dev.ltms.fleet.peer.PeerLauncher;
import dev.ltms.fleet.peer.SpawnRequest;
import dev.ltms.fleet.placement.PlacementDecision;
import java.util.List;
import java.util.Set;
/**
* {@link PeerLauncher} decorator that marks presence for a spawned terminal before returning its
* handle to the caller — the contact-then-register ordering fleetd #722 covers, where the
* terminal's MCP contact lands before {@link SessionManager#acquire} runs its own
* {@code registry.put}. The presence view is set after construction, via {@link #presence},
* because it is owned by the {@link SessionManager} this launcher is passed into.
*/
final class PresenceRacingLauncher implements PeerLauncher {
private final PeerLauncher delegate;
volatile MemberPresence presence;
PresenceRacingLauncher(PeerLauncher delegate) {
this.delegate = delegate;
}
@Override
public PeerHandle spawn(SpawnRequest req) {
PeerHandle handle = delegate.spawn(req);
presence.markPresent(handle.terminalId());
return handle;
}
@Override
public PeerHandle spawn(SpawnRequest req, PlacementDecision decision) {
PeerHandle handle = delegate.spawn(req, decision);
presence.markPresent(handle.terminalId());
return handle;
}
@Override
public Set<Capability> capabilities() {
return delegate.capabilities();
}
@Override
public Set<Capability> capabilitiesFor(String profileName) {
return delegate.capabilitiesFor(profileName);
}
@Override
public Set<String> profiles() {
return delegate.profiles();
}
@Override
public String defaultProfile() {
return delegate.defaultProfile();
}
@Override
public String effectiveCwd(SpawnRequest req) {
return delegate.effectiveCwd(req);
}
@Override
public List<String> parityOverlay(String profileName) {
return delegate.parityOverlay(profileName);
}
@Override
public List<?> list() {
return delegate.list();
}
@Override
public int reapOrphanWorkers() {
return delegate.reapOrphanWorkers();
}
@Override
public void stop(String id) {
delegate.stop(id);
}
@Override
public boolean clearContext(String id) {
return delegate.clearContext(id);
}
}
@@ -422,6 +422,53 @@ class SessionManagerTest {
"turn completion moves BUSY → DONE"); "turn completion moves BUSY → DONE");
} }
// --- fleetd #722: registration and presence must reach READY whichever lands first --------
@Test
void registerThenContactReachesReadyForPlainSpawn() {
FakeHerdr herdr = new FakeHerdr();
SessionManager sessions = sessionManager(herdr);
MemberSession session = sessions.acquire("ltms-local", null, "/caller", "term_primary");
sessions.asPresence().markPresent(session.terminalId());
assertEquals(MemberSession.State.READY, sessions.get(session.paneId()).orElseThrow().state(),
"a presence contact that arrives after registration reaches READY");
}
@Test
void contactThenRegisterStillReachesReadyForPlainSpawn() {
// The racing launcher marks presence for the spawned terminal from inside spawn() —
// before SessionManager.acquire's own registry.put runs — modeling an MCP contact that
// lands in that window.
FakeHerdr herdr = new FakeHerdr();
FleetConfig.Profile cfg = new FleetConfig.Profile(
"ltms-local", "http://gx00.gw:8000", "coder", null, "FLEETD_WORKER_TOKEN",
List.of("ccs", "ltms-local"), "tab", "fleetd-workers",
"worker: {profile} #{n}", null, null, null);
ClaudeCodeLauncher workers = new ClaudeCodeLauncher(new AgentControl(herdr), new WorkspaceControl(herdr),
new SubscriptionGuard(Set.of("gx00.gw")), Map.of(cfg.profile(), cfg), cfg.profile(), _ -> null);
PresenceRacingLauncher race = new PresenceRacingLauncher(workers);
SessionManager sessions = new SessionManager(race);
race.presence = sessions.asPresence();
MemberSession session = sessions.acquire("ltms-local", null, "/caller", "term_primary");
assertEquals(MemberSession.State.READY, sessions.get(session.paneId()).orElseThrow().state(),
"a presence contact that lands before registry.put must still reach READY");
}
@Test
void aTerminalNeverMarkedPresentStaysSpawningAfterRegistration() {
FakeHerdr herdr = new FakeHerdr();
SessionManager sessions = sessionManager(herdr);
MemberSession session = sessions.acquire("ltms-local", null, "/caller", "term_primary");
assertEquals(MemberSession.State.SPAWNING, sessions.get(session.paneId()).orElseThrow().state(),
"registration alone must not advance a terminal that was never marked present");
}
@Test @Test
void releaseTearsDownWorkerAndRemovesFromRosterAndIsIdempotent() { void releaseTearsDownWorkerAndRemovesFromRosterAndIsIdempotent() {
FakeHerdr herdr = new FakeHerdr(); FakeHerdr herdr = new FakeHerdr();
@@ -1405,6 +1452,105 @@ class SessionManagerTest {
+ "dirty check threw"); + "dirty check threw");
} }
// --- fleetd #736: a release must forget the member's presence entry, not just its registry
// row ---------------------------------------------------------------------------------------
@Test
void releaseByPaneIdForgetsThePresenceEntry() {
FakeHerdr herdr = new FakeHerdr();
SessionManager sessions = sessionManager(herdr);
MemberSession session = sessions.acquire("ltms-local", null, "/caller", "term_primary");
String terminal = session.terminalId();
sessions.asPresence().markPresent(terminal);
assertTrue(sessions.asPresence().isPresent(terminal), "present before the release");
sessions.release(session.paneId());
assertFalse(sessions.asPresence().isPresent(terminal),
"release must forget the terminal's presence, not just remove its registry row");
}
@Test
void reapIdleForgetsThePresenceEntryToo() {
long[] clock = {0};
FakeHerdr herdr = new FakeHerdr();
SessionManager sessions = sessionManager(herdr, () -> clock[0]);
MemberSession session = sessions.acquire("ltms-local", null, "/caller", "term_primary");
String terminal = session.terminalId();
sessions.asPresence().markPresent(terminal);
assertTrue(sessions.asPresence().isPresent(terminal), "present before the reap");
clock[0] = 11;
assertEquals(1, sessions.reapIdle(10), "READY session past TTL is reaped");
assertFalse(sessions.asPresence().isPresent(terminal),
"the idle-reap release path (releaseIfCurrent) goes through the same teardown "
+ "funnel as an explicit release, so it must forget presence too");
}
@Test
void shutdownDrainAlsoForgetsThePresenceEntry() {
FakeHerdr herdr = new FakeHerdr();
SessionManager sessions = sessionManager(herdr);
MemberSession session = sessions.acquire("ltms-local", null, "/caller", "term_primary");
String terminal = session.terminalId();
sessions.asPresence().markPresent(terminal);
assertTrue(sessions.asPresence().isPresent(terminal), "present before the drain");
sessions.drainAll(TimeUnit.MILLISECONDS.toNanos(100));
assertFalse(sessions.asPresence().isPresent(terminal),
"a shutdown drain still ends the member's process, so presence must be cleared "
+ "exactly as it is for any other release cause");
}
@Test
void releaseOfAnUnknownPaneIdDoesNotThrow() {
FakeHerdr herdr = new FakeHerdr();
SessionManager sessions = sessionManager(herdr);
assertDoesNotThrow(() -> sessions.release("no-such-pane"),
"releasing a pane id that was never registered must be a no-op, not a throw");
}
@Test
void releaseStillForgetsPresenceWhenDirtyCheckThrows() {
FakeHerdr herdr = new FakeHerdr();
RecordingWorktrees worktrees = new RecordingWorktrees();
SessionManager sessions = sessionManager(herdr, worktrees);
MemberSession s = sessions.acquire("ltms-local", null, "/caller/proj", null,
new WorktreeRequest("fleetd-736", null));
String terminal = s.terminalId();
sessions.asPresence().markPresent(terminal);
worktrees.failHasUncommittedWith(new WorktreeException("git status exited 128"));
assertDoesNotThrow(() -> sessions.release(s.paneId()),
"a throwing dirty check must not abort the release");
assertFalse(sessions.asPresence().isPresent(terminal),
"presence must be forgotten even when the dirty check throws, which pins the "
+ "forget call to the finally block that runs no matter what happened above");
}
@Test
void releaseLeavesADifferentStillLiveMembersPresenceUntouched() {
FakeHerdr herdr = new FakeHerdr();
SessionManager sessions = sessionManager(herdr);
MemberSession released = sessions.acquire("ltms-local", null, "/caller/a", "ownerA");
MemberSession stillLive = sessions.acquire("ltms-local", null, "/caller/b", "ownerB");
sessions.asPresence().markPresent(released.terminalId());
sessions.asPresence().markPresent(stillLive.terminalId());
assertTrue(sessions.asPresence().isPresent(stillLive.terminalId()),
"present before the release of the other member");
sessions.release(released.paneId());
assertFalse(sessions.asPresence().isPresent(released.terminalId()),
"the released terminal is forgotten");
assertTrue(sessions.asPresence().isPresent(stillLive.terminalId()),
"a still-live member's presence must survive an unrelated release");
}
// --- fleetd #316: the dirty check must be re-taken after the worker is stopped, not trusted // --- fleetd #316: the dirty check must be re-taken after the worker is stopped, not trusted
// stale from before it ------------------------------------------------------------------------ // stale from before it ------------------------------------------------------------------------
@@ -159,6 +159,40 @@ class WorktreeSessionManagerTest {
assertEquals(expectedPath, s.cwd(), "session cwd is the worktree path"); assertEquals(expectedPath, s.cwd(), "session cwd is the worktree path");
} }
// --- fleetd #722: registration and presence must reach READY whichever lands first --------
@Test
void registerThenContactReachesReadyForWorktreeSpawn() {
FakeHerdr herdr = new FakeHerdr();
FakeWorktrees worktrees = new FakeWorktrees().withRepoRoot("/repo").withPrefix("/wt");
SessionManager sessions = new SessionManager(workerService(herdr), worktrees);
MemberSession session = sessions.acquire("ltms-local", null, "/caller/proj", "term_primary",
new WorktreeRequest("cb-722", null));
sessions.asPresence().markPresent(session.terminalId());
assertEquals(MemberSession.State.READY, sessions.get(session.paneId()).orElseThrow().state(),
"a presence contact that arrives after worktree registration reaches READY");
}
@Test
void contactThenRegisterStillReachesReadyForWorktreeSpawn() {
// The racing launcher marks presence for the spawned terminal from inside spawn() —
// before SessionManager.acquireWithWorktree's own registry.put runs — modeling an MCP
// contact that lands in that window.
FakeHerdr herdr = new FakeHerdr();
FakeWorktrees worktrees = new FakeWorktrees().withRepoRoot("/repo").withPrefix("/wt");
PresenceRacingLauncher race = new PresenceRacingLauncher(workerService(herdr));
SessionManager sessions = new SessionManager(race, worktrees);
race.presence = sessions.asPresence();
MemberSession session = sessions.acquire("ltms-local", null, "/caller/proj", "term_primary",
new WorktreeRequest("cb-722", null));
assertEquals(MemberSession.State.READY, sessions.get(session.paneId()).orElseThrow().state(),
"a presence contact that lands before worktree registration must still reach READY");
}
@Test @Test
void worktreeArchitectAcquireAlsoBindsItsSlot() { void worktreeArchitectAcquireAlsoBindsItsSlot() {
FakeHerdr herdr = new FakeHerdr(); FakeHerdr herdr = new FakeHerdr();
+3 -2
View File
@@ -1507,6 +1507,7 @@ else
grep -E ' (ERROR|SEVERE) ' "$FRESH_LOG" | tail -5 | sed 's/^/ /' grep -E ' (ERROR|SEVERE) ' "$FRESH_LOG" | tail -5 | sed 's/^/ /'
fi fi
echo echo
echo " Next: call fleet_whoami and confirm it still answers 'primary'. A lead whose tab label" echo " Next: call fleet_whoami and confirm it still answers 'primary'. A lead is found by its"
echo " no longer matches fleet.leaders.*.tab is demoted to worker and refuses orchestration." echo " tab being labelled 'lead' AND sitting in fleet.leaders.<name>.workspace; if either stops"
echo " matching, the lead is demoted to worker and refuses orchestration."
echo echo