Compare commits
19 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f687046450 | |||
| c6058652be | |||
| 7f9137fcb6 | |||
| 3bf3968bc7 | |||
| 261aa056f9 | |||
| 042b8c99dd | |||
| 4bfab6b718 | |||
| 008a457557 | |||
| 9494a6b99a | |||
| eb0557621e | |||
| a0eed6f01b | |||
| e2a91e883e | |||
| 62646957ea | |||
| 802c0ab701 | |||
| 4bab23e241 | |||
| a94262271b | |||
| 5c12865c25 | |||
| 3f8c956325 | |||
| 2051ceeb26 |
@@ -1,13 +1,23 @@
|
||||
---
|
||||
name: handover
|
||||
description: Procedure for an outgoing lead to write the handover file before a fresh lead session replaces it (fleetd #480). Load this when your context is full and fleetd is about to clear your pane. The file is the new lead's only inheritance — follow it exactly.
|
||||
description: Procedure for an outgoing lead to write the handover file that a fresh lead session inherits. Load this when your context is filling up and you are about to be replaced, whether you hand off by hand or fleetd does it for you. The file is the new lead's only inheritance — follow it exactly.
|
||||
---
|
||||
|
||||
# Handover — write the file the next lead depends on
|
||||
|
||||
fleetd ticket #480 lets a lead session hand off to a fresh one. The outgoing lead writes a
|
||||
handover file, fleetd checks it, clears the pane, and tells the new session to read that file
|
||||
and carry on.
|
||||
A lead session fills up its context and has to be replaced by a fresh one. The outgoing lead
|
||||
writes a handover file, and the new session reads that file and carries on.
|
||||
|
||||
**There are two ways to hand off, and the file is the same either way.**
|
||||
|
||||
- **By hand.** You write the file, then tell the operator where it is. The operator starts the new
|
||||
session and points it at the file. This always works.
|
||||
- **With `fleet_handover`** (fleetd #480, merged 2026-09-11). You ask fleetd to do the swap: it
|
||||
checks the file, clears your pane, and tells the fresh session to read it. This needs
|
||||
`leadRollover:` in `fleetd.yaml`; without it every action answers a clean refusal naming
|
||||
`NOT_CONFIGURED`, and you fall back to the manual path. Section 11 below is the procedure.
|
||||
|
||||
Nothing else in this skill changes between the two. Only who performs the swap changes.
|
||||
|
||||
**The new lead's only inheritance is that file.** It does not see your conversation, your plan,
|
||||
or your screen. If the file is thin or wrong, the new lead re-derives what you already knew, and
|
||||
@@ -118,6 +128,47 @@ Do not include:
|
||||
|
||||
A handover file is a record of state and decisions. It is not a diary.
|
||||
|
||||
## 11. Using `fleet_handover` (only if `leadRollover:` is configured)
|
||||
|
||||
**Run the three steps in this order. The order is not a style choice — the wrong order is
|
||||
refused.**
|
||||
|
||||
1. **`fleet_handover{action: "open", reason: "<why now>"}`.** It returns a `token` and the
|
||||
`handoverPath` you must write to. Nothing has happened to your pane yet.
|
||||
|
||||
**Write to exactly that path, and do not resolve it yourself.** It is always absolute, even when
|
||||
the operator configured a relative `handoverPath`: fleetd resolves a relative one against your
|
||||
own workspace before it hands it to you. The daemon and your pane can run in different
|
||||
directories, so a path you resolve yourself can point at a different file from the one the daemon
|
||||
will check.
|
||||
2. **Write the handover file at that path**, following sections 1–10 above.
|
||||
3. **Ask the operator, then `fleet_handover{action: "confirm", token, operatorConfirmed: true}`.**
|
||||
|
||||
Why that order: `confirm` refuses with `HANDOVER_STALE` unless the file was modified **after** the
|
||||
`open` request. That check stops a leftover file from an earlier session being accepted as this
|
||||
one's handover. So writing the file first and then calling `open` — the obvious order — always
|
||||
fails.
|
||||
|
||||
`{action: "cancel", token}` drops a pending request without rolling.
|
||||
|
||||
**Things that will surprise you:**
|
||||
|
||||
- **`accepted` does not mean your pane has been cleared.** It means every gate passed and the roll
|
||||
is scheduled to run once your current turn ends. Say your goodbye in the same turn — you will not
|
||||
get another one.
|
||||
- **There is no terminal or session parameter, on purpose.** The pane is always your own, resolved
|
||||
from your connection, so you can only ever roll yourself.
|
||||
- **`operatorConfirmed` is your report of what a human told you.** Do not pass `true` because you
|
||||
are confident. Ask, wait for the answer, then pass what they said. `requireOperatorConfirm`
|
||||
defaults to `true` and this is the only thing standing between a judgement call and a wiped
|
||||
session.
|
||||
- **The roll can still refuse after `confirm` returns**, and by then there is no caller to tell.
|
||||
Those outcomes are logged only, as `lead-rollover:` lines in the daemon log.
|
||||
- **If the bootstrap prompt never lands, your context is gone and no fresh session starts.** This
|
||||
has not yet been proven end-to-end (see fleetd #480). The recovery is the manual path: the file
|
||||
is already written, so the operator starts a session and points it at the file. That is why you
|
||||
write the file before you confirm, and never the other way round.
|
||||
|
||||
## Writing style
|
||||
|
||||
Write in plain English. Use everyday words, one idea per sentence, and active voice. Keep every
|
||||
|
||||
@@ -19,3 +19,9 @@
|
||||
fleetd.out
|
||||
fleetd/fleetd.out
|
||||
logs/
|
||||
|
||||
# fleetd #480: the lead rollover handover file. `leadRollover.handoverPath` points here, and the
|
||||
# outgoing lead rewrites it on every rollover. It is a snapshot of one moment's live state —
|
||||
# unpushed branches, running builds, open questions — so it is stale the moment it is written and
|
||||
# has no business in git history.
|
||||
.handover/
|
||||
|
||||
@@ -142,6 +142,7 @@ the merge — and merging on a reviewer's word is delegating it by proxy.
|
||||
| Read your own held lead-to-lead mail (no ack) | `fleet_poll{coordId: <your own coord-id, from fleet_list's coordinator.selfId>}` — primary-only; never acks, so `fleet_list`'s `held[]` still shows it after. `fleet_list`'s `held[]` gives only a truncated preview — this is the only way to read the full body |
|
||||
| Collect a held reply | `fleet_poll{target}` · then `fleet_ack{target, msgId}` |
|
||||
| Tear down a member | `fleet_stop{paneId}` |
|
||||
| Replace your OWN lead session when its context is full | `fleet_handover{action:"open", reason?}` → write the handover file it names → `fleet_handover{action:"confirm", token, operatorConfirmed}`. Primary-only. **In that order**: the file must be modified *after* `open`, or `confirm` refuses it as stale. There is no terminal parameter — the pane is always your own, so you can never roll another lead. `{action:"cancel", token}` drops a pending request |
|
||||
|
||||
### Lead ↔ lead — coordinate, never delegate
|
||||
|
||||
|
||||
@@ -88,6 +88,47 @@ bind:
|
||||
# backoffMs: 60000
|
||||
# quietNudgeCap: 3
|
||||
|
||||
# Lead rollover (fleetd #480): replace a lead session that has decided it is ready to be replaced,
|
||||
# without an operator doing it by hand. A lead writes a handover file, then asks fleetd to clear its
|
||||
# own pane and bootstrap a fresh session against that file.
|
||||
#
|
||||
# Opt-in on purpose — it clears the lead's own pane on request, so upgrading the daemon must never
|
||||
# acquire that ability for you. Absent block = feature off, and nothing is constructed at all. Even
|
||||
# once present, nothing but an explicit confirm() call — one that passes every check — can ever
|
||||
# cause a /clear: there is no recurring timer, heartbeat or scheduler anywhere in this feature that
|
||||
# fires one on its own initiative. confirm() itself is called FROM the calling lead's own turn, so
|
||||
# it cannot clear the pane inline (that pane is still WORKING); instead it schedules a one-shot
|
||||
# continuation that waits for the SAME confirm() call's turn to end, then does the actual work. See
|
||||
# dev.ltms.fleet.lead.LeadRollover's class javadoc for the exact order (fleetd #480 correction).
|
||||
#
|
||||
# handoverPath: REQUIRED when this block is present — where the handover file a fresh lead session
|
||||
# reads must live. No default (an operator-specific path); a present block with no
|
||||
# handoverPath refuses to start. May be relative: it then resolves against the
|
||||
# CALLING lead's own fleet.leaders.<name>.cwd (falling back to the daemon's own
|
||||
# working directory when that lead has none configured) — never against whatever
|
||||
# directory the daemon process happens to have been started in. An absolute path is
|
||||
# used unchanged. Prefer an absolute path if the daemon and the lead's pane might not
|
||||
# share a working directory (fleetd #480 follow-up).
|
||||
# requireOperatorConfirm: true # default true — confirm() refuses unless the caller also passes
|
||||
# # operatorConfirmed: true
|
||||
# maxDocAgeSeconds: 3600 # default 3600 — refuse a handover file older than this
|
||||
# turnSettleSeconds: 20 # default 20 — how long the deferred roll waits for the CALLING
|
||||
# # lead's own turn to end (its pane to report injectable again)
|
||||
# # before sending /clear at all. If this elapses, /clear is NEVER
|
||||
# # sent — a lead that never goes idle is still doing real work.
|
||||
# clearSettleSeconds: 20 # default 20 — how long to wait for the pane to become injectable
|
||||
# # again AFTER /clear before giving up (never sends bootstrapText
|
||||
# # if this elapses). A separate, second wait from turnSettleSeconds.
|
||||
# bootstrapText: "..." # default names the RESOLVED (absolute) handoverPath — sent to
|
||||
# # the lead once its pane settles after /clear
|
||||
# leadRollover:
|
||||
# handoverPath: /path/to/handover.md
|
||||
# requireOperatorConfirm: true
|
||||
# maxDocAgeSeconds: 3600
|
||||
# turnSettleSeconds: 20
|
||||
# clearSettleSeconds: 20
|
||||
# bootstrapText: "Fresh lead session: read the handover file and carry on."
|
||||
|
||||
# Fleet health detection is dormant unless enabled (CB-573). It reads one whole-fleet agent list
|
||||
# per tick.
|
||||
# intervalSeconds → how often a tick runs (default 30). ENFORCED floor of 15: the code computes
|
||||
|
||||
@@ -10,6 +10,7 @@ import dev.ltms.fleet.herdr.HerdrException;
|
||||
import dev.ltms.fleet.herdr.HerdrRouter;
|
||||
import dev.ltms.fleet.herdr.LeadTabScanner;
|
||||
import dev.ltms.fleet.lead.LeadLauncher;
|
||||
import dev.ltms.fleet.lead.LeadRollover;
|
||||
import dev.ltms.fleet.herdr.PaneLocator;
|
||||
import dev.ltms.fleet.herdr.UnixSocketHerdrClient;
|
||||
import dev.ltms.fleet.herdr.WorkspaceControl;
|
||||
@@ -590,6 +591,16 @@ public final class Fleetd {
|
||||
heartbeat = null;
|
||||
heartbeatScheduler.shutdownNow();
|
||||
}
|
||||
// fleetd #480: lead rollover. Opt-in; absent `leadRollover:` this is never constructed, so
|
||||
// an upgraded daemon cannot silently acquire the ability to clear the lead's own pane.
|
||||
// Unlike heartbeat above, this has no recurring scheduler of its own — nothing but an
|
||||
// explicit confirm() call (wired to an MCP tool by a later ticket; nothing calls it yet)
|
||||
// that passes every gate can ever schedule a roll. It does not take primaryRegistry: the
|
||||
// lead terminal to roll comes from the caller of open()/confirm() (resolved by the MCP
|
||||
// layer from the connection, the same way auth/CallerResolver#resolve builds a
|
||||
// Principal.leader(...)), never from a single-slot lookup — see LeadRollover's class
|
||||
// javadoc, fleetd #480 correction 2.
|
||||
LeadRollover leadRollover = leadRollover(cfg, router.leadAgents(), config, leads);
|
||||
MessageService messages = new MessageService(router, injector, rendezvous, replyInbox,
|
||||
pushLoop, metrics);
|
||||
|
||||
@@ -700,7 +711,10 @@ public final class Fleetd {
|
||||
// reach. Read from the SAME snapshot leadMailbox itself opened from (cfg.coordinator()),
|
||||
// not the live config.get() — coordinator wiring is already boot-time-fixed (see
|
||||
// leadMailbox above), so peers follows the same rule rather than half hot-reloading.
|
||||
cfg.coordinator() == null ? List.of() : cfg.coordinator().peers());
|
||||
cfg.coordinator() == null ? List.of() : cfg.coordinator().peers(),
|
||||
// fleetd #480 Unit C: the executor behind fleet_handover — null whenever
|
||||
// leadRollover: is not configured (see the leadRollover local above).
|
||||
leadRollover);
|
||||
|
||||
// CB-637: the receive half. Only constructed when a lead mailbox actually opened — with no
|
||||
// coordinator (or an unreachable one) there is nothing to deliver, so no scheduler is
|
||||
@@ -1053,6 +1067,68 @@ public final class Fleetd {
|
||||
.orElse(null);
|
||||
}
|
||||
|
||||
/**
|
||||
* fleetd #480: construct the {@link LeadRollover} executor only when {@code leadRollover:} is
|
||||
* present at startup — the same presence gate {@code leadHeartbeat:} uses just above this
|
||||
* call site in {@code main}. Extracted to its own factory, the same reason
|
||||
* {@link #worktreeBranchLookup} and {@link #exhaustionSink} are: a unit test can call this
|
||||
* directly with a fabricated {@link FleetConfig} (absent block ⇒ {@code null}, present block ⇒
|
||||
* constructed) without needing the whole of {@code main}, and a source-text test on the real
|
||||
* call site proves {@code main} still calls this factory rather than inlining a copy that could
|
||||
* silently diverge.
|
||||
*
|
||||
* <p>The returned object reads every {@code leadRollover:} field fresh on each {@code open()}/
|
||||
* {@code confirm()} call through {@code () -> config.get().leadRollover()} — see {@code
|
||||
* ConfigRef}'s class doc Hot bullet and {@link FleetConfig.LeadRollover}'s javadoc for why that
|
||||
* makes the block's fields HOT despite this presence gate being evaluated once, at startup.
|
||||
*
|
||||
* <p>Deliberately does NOT take {@link PrimaryRegistry}: fleetd #480 correction 2 found that a
|
||||
* single-slot lookup lets one lead's {@code confirm()} clear a DIFFERENT lead's pane on a
|
||||
* daemon with more than one labelled lead tab. The lead terminal to roll instead comes from
|
||||
* whoever calls {@code open()}/{@code confirm()} — the later MCP-tool unit must resolve it from
|
||||
* the connection and pass it in, never take it as a request field. See {@link LeadRollover}'s
|
||||
* class javadoc.
|
||||
*
|
||||
* <p><strong>fleetd #480 follow-up:</strong> also builds the terminal → lead-workspace lookup
|
||||
* {@link LeadRollover#open} needs to resolve a relative {@code handoverPath} against the
|
||||
* CALLING lead's own {@code cwd} rather than the daemon's — the daemon and a lead's pane can
|
||||
* have different working directories (this repo nests {@code fleetd/} inside its own root, so
|
||||
* they already differ on this host). The lookup is terminal → lead name (via {@code
|
||||
* liveLeadTerminals}) → that lead's {@code cwd} (via {@code config.get().fleet().leaders()}),
|
||||
* and both hops are read LIVE on every call, never off a snapshot taken here: leads are
|
||||
* discovered by a live tab scan ({@code LeadTabScanner}), so a map captured at construction
|
||||
* time could be empty (no lead has been scanned yet) or stale (a lead added since).
|
||||
*
|
||||
* @param cfg the startup config snapshot — read ONCE here, only to decide whether
|
||||
* to construct the object at all, exactly like {@code
|
||||
* cfg.leadHeartbeat()}
|
||||
* @param leadAgents the {@link AgentControl} instance that reaches the LEAD's pane (not
|
||||
* {@code memberAgents}), normally {@code router.leadAgents()}
|
||||
* @param config the live {@link ConfigRef}, captured only inside the returned
|
||||
* supplier and the workspace lookup — never dereferenced here
|
||||
* @param liveLeadTerminals terminal id → lead NAME for every CURRENTLY recognised lead, normally
|
||||
* the same {@code leads} supplier {@code main} already builds for
|
||||
* {@code HerdrRouter}/{@link #leadSeatLookup} — never a value snapshot
|
||||
* @return a constructed {@link LeadRollover}, or {@code null} when {@code leadRollover:} is
|
||||
* absent from the startup config
|
||||
*/
|
||||
static LeadRollover leadRollover(FleetConfig cfg, AgentControl leadAgents, ConfigRef config,
|
||||
Supplier<Map<String, String>> liveLeadTerminals) {
|
||||
if (cfg.leadRollover() == null) {
|
||||
return null;
|
||||
}
|
||||
Function<String, String> leadWorkspace = terminal -> {
|
||||
String leadName = liveLeadTerminals.get().get(terminal);
|
||||
if (leadName == null) {
|
||||
return null;
|
||||
}
|
||||
FleetConfig.Fleet fleet = config.get().fleet();
|
||||
FleetConfig.Leader leader = fleet == null ? null : fleet.leaders().get(leadName);
|
||||
return leader == null ? null : leader.cwd();
|
||||
};
|
||||
return new LeadRollover(leadAgents, () -> config.get().leadRollover(), leadWorkspace);
|
||||
}
|
||||
|
||||
/**
|
||||
* fleetd #176: per-profile factory for {@link FleetMcp.LeadSeatSource} — how many seats a
|
||||
* profile's own live LEAD session(s) hold on the same Claude subscription.
|
||||
|
||||
@@ -41,7 +41,16 @@ public final class Authz {
|
||||
*/
|
||||
COORD_READ,
|
||||
/** Scrape the metrics endpoint. */
|
||||
METRICS
|
||||
METRICS,
|
||||
/**
|
||||
* Drive the lead-rollover executor ({@code fleet_handover}: open/confirm/cancel a
|
||||
* self-replace, fleetd #480 Unit C). Primary-only, same as {@link #SPAWN}/{@link #STOP}/
|
||||
* {@link #DRAIN} — and, unlike those, the terminal it acts on is never even an argument:
|
||||
* {@code LeadRollover#open}/{@code #confirm} are always called with the CALLER's own
|
||||
* connection-resolved terminal (see {@code dev.ltms.fleet.lead.LeadRollover}'s class
|
||||
* javadoc, fleetd #480 correction 2), so a primary can only ever roll itself.
|
||||
*/
|
||||
HANDOVER
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -60,7 +69,7 @@ public final class Authz {
|
||||
// deliberately does NOT get these (CB-548), so it cannot tear down or stand up workers
|
||||
// even though it coordinates them; and a worker driving any of these would be a worker
|
||||
// escalating into the orchestrator role.
|
||||
case SPAWN, STOP, DRAIN -> caller.isPrimary();
|
||||
case SPAWN, STOP, DRAIN, HANDOVER -> caller.isPrimary();
|
||||
|
||||
// Delivering a turn is open to the primary and the architect: an architect delegates
|
||||
// to workers (that is the role's point) but still has no lifecycle rights. A worker is
|
||||
|
||||
@@ -62,7 +62,22 @@ import java.util.function.Supplier;
|
||||
* both read, so a reload that arms or disarms a profile's usage-limit detection takes effect
|
||||
* on the next check with no restart. {@code errorPattern}, {@code exhaustedPattern}'s sibling
|
||||
* key for backend-error (not usage-limit) classification, was deliberately left OUT of this
|
||||
* fleetd #446 change and stays deferred below — the ticket scoped it out explicitly.</li>
|
||||
* fleetd #446 change and stays deferred below — the ticket scoped it out explicitly.
|
||||
* {@code leadRollover:} (fleetd #480) joined this class whole, the same shape as
|
||||
* {@code models:} above: {@code dev.ltms.fleet.lead.LeadRollover} holds a
|
||||
* {@code Supplier<FleetConfig.LeadRollover>} (the same {@code () -> config.get().x()} shape)
|
||||
* and reads {@code handoverPath}/{@code requireOperatorConfirm}/{@code maxDocAgeSeconds}/
|
||||
* {@code turnSettleSeconds}/{@code clearSettleSeconds}/{@code bootstrapText} fresh on every
|
||||
* {@code open()}/{@code confirm()} call (and on the deferred post-{@code confirm()}
|
||||
* continuation fleetd #480's correction added — see {@code LeadRollover}'s class doc) rather
|
||||
* than capturing them into fields at construction — unlike its closest
|
||||
* structural cousin {@code leadHeartbeat:}, whose {@code LeadHeartbeatLoop} bakes
|
||||
* {@code idleAfterNanos}/{@code backoffMs}/{@code quietNudgeCap} into final fields. The one
|
||||
* restart-only edge is structural, not a stale value: {@code Fleetd.java} decides whether to
|
||||
* construct the {@code LeadRollover} object at all off the startup snapshot (the same
|
||||
* presence gate {@code leadHeartbeat:} uses), so a block ADDED where it was absent at boot
|
||||
* needs a restart before anything exists to call — the same fact already true of adding a
|
||||
* brand-new {@code profiles:} entry.</li>
|
||||
* <li><strong>Deferred</strong> — accepted into the new snapshot, but the wiring built at startup
|
||||
* keeps the old value until a restart: {@code lifecycle:}, {@code leadHeartbeat:},
|
||||
* {@code idleSleepGuard:} ({@code Fleetd.java} reads it once, at startup, to decide whether
|
||||
@@ -166,17 +181,20 @@ import java.util.function.Supplier;
|
||||
*
|
||||
* <p><strong>The denominator, measured on 2026-09-04 (fleetd #330; recounted for fleetd #333);
|
||||
* recounted again for fleetd #362, again after {@code idleSleepGuard:} was added, again after
|
||||
* {@code models:} was added as deferred, and again for fleetd #422, which moved {@code models:}
|
||||
* from deferred to hot-excluded once its on/off half was read live everywhere.</strong>
|
||||
* {@code FleetConfig} has 25 top-level record components: 5 cold, 13 deferred, 3 split, 4
|
||||
* hot-excluded. Four of them are named nowhere in this file, and the reason is the same for all
|
||||
* four: {@code placement}, {@code memberCredentials}, {@code memberLoginShell} and {@code models}
|
||||
* are <strong>hot</strong> and correctly absent — all four are read live off {@code config.get()}
|
||||
* (placement through the {@code CompositePeerLauncher} supplier the Hot bullet names;
|
||||
* {@code memberCredentials}/{@code memberLoginShell} at spawn time, {@code Fleetd.java:198, 205, 729}
|
||||
* and {@code HerdrPeerLauncher#configuredMemberLoginShell}; {@code models} the same way, through the
|
||||
* Hot bullet's {@code models:} paragraph), so a reload takes effect on the next spawn (or, for
|
||||
* {@code models}, the next reported status) with no entry needed here.
|
||||
* {@code models:} was added as deferred, again for fleetd #422, which moved {@code models:}
|
||||
* from deferred to hot-excluded once its on/off half was read live everywhere, and again after
|
||||
* {@code leadRollover:} was added (fleetd #480).</strong>
|
||||
* {@code FleetConfig} has 26 top-level record components: 5 cold, 13 deferred, 3 split, 5
|
||||
* hot-excluded. Five of them are named nowhere in this file, and the reason is the same for all
|
||||
* five: {@code placement}, {@code memberCredentials}, {@code memberLoginShell}, {@code models} and
|
||||
* {@code leadRollover} are <strong>hot</strong> and correctly absent — all five are read live off
|
||||
* {@code config.get()} (placement through the {@code CompositePeerLauncher} supplier the Hot bullet
|
||||
* names; {@code memberCredentials}/{@code memberLoginShell} at spawn time, {@code Fleetd.java:198,
|
||||
* 205, 729} and {@code HerdrPeerLauncher#configuredMemberLoginShell}; {@code models} the same way,
|
||||
* through the Hot bullet's {@code models:} paragraph; {@code leadRollover} through the Hot bullet's
|
||||
* {@code leadRollover:} paragraph), so a reload takes effect on the next spawn (or, for
|
||||
* {@code models}, the next reported status; for {@code leadRollover}, the next {@code open()}/
|
||||
* {@code confirm()} call) with no entry needed here.
|
||||
* {@code health} and {@code coordinator} used to be a third kind — <strong>undecided</strong>, not
|
||||
* hot — until fleetd #330 added the <strong>split</strong> class above and gave them a home. A
|
||||
* reload touching either used to report a bare "config reloaded", which under-claimed; now it names
|
||||
|
||||
@@ -139,6 +139,9 @@ import java.util.regex.PatternSyntaxException;
|
||||
* fleetd #422 added the separate on/off question — whether a configured model
|
||||
* may be spawned onto RIGHT NOW ({@link Models.ModelEntry#enabled}) — enforced
|
||||
* live at spawn by {@code CompositePeerLauncher}, not here. See {@link Models}.
|
||||
* @param leadRollover opt-in lead rollover (fleetd #480): {@code null} ⇒ off, and no
|
||||
* {@code dev.ltms.fleet.lead.LeadRollover} is constructed at all — an upgraded
|
||||
* daemon never clears a lead's pane on its own initiative. See {@link LeadRollover}.
|
||||
*/
|
||||
@JsonIgnoreProperties(ignoreUnknown = true)
|
||||
public record FleetConfig(
|
||||
@@ -166,7 +169,23 @@ public record FleetConfig(
|
||||
String memberLoginShell,
|
||||
String memberSkills,
|
||||
IdleSleepGuard idleSleepGuard,
|
||||
Models models) {
|
||||
Models models,
|
||||
LeadRollover leadRollover) {
|
||||
|
||||
/** Back-compat form before the {@code leadRollover:} block was added. */
|
||||
public FleetConfig(Bind bind, String herdrSocket, String memberHerdrSocket, Map<String, Profile> profiles,
|
||||
Guard guard, String worktreeRoot, Lifecycle lifecycle, Integer spawnReadyTimeoutMs,
|
||||
Integer spawnReadyPollMs, Broker broker, Primary primary, Fleet fleet,
|
||||
LeadHeartbeat leadHeartbeat, Health health, String placement, Auth auth,
|
||||
ConfigReload configReload, Integer quarantineCooldownSeconds,
|
||||
MemberCredentials memberCredentials, Coordinator coordinator, String worktreeGroup,
|
||||
String memberLoginShell, String memberSkills, IdleSleepGuard idleSleepGuard,
|
||||
Models models) {
|
||||
this(bind, herdrSocket, memberHerdrSocket, profiles, guard, worktreeRoot, lifecycle, spawnReadyTimeoutMs,
|
||||
spawnReadyPollMs, broker, primary, fleet, leadHeartbeat, health, placement, auth,
|
||||
configReload, quarantineCooldownSeconds, memberCredentials, coordinator, worktreeGroup,
|
||||
memberLoginShell, memberSkills, idleSleepGuard, models, null);
|
||||
}
|
||||
|
||||
/** Back-compat form before the {@code models:} block was added. */
|
||||
public FleetConfig(Bind bind, String herdrSocket, String memberHerdrSocket, Map<String, Profile> profiles,
|
||||
@@ -1310,6 +1329,104 @@ public record FleetConfig(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Opt-in lead rollover (fleetd #480): a lead that decides it is ready to be replaced writes a
|
||||
* handover file, then asks fleetd to clear its own pane and bootstrap a fresh session against
|
||||
* that file. Config + a pure decision/verification layer only — see
|
||||
* {@code dev.ltms.fleet.lead.LeadRollover} for the executor this block feeds, and
|
||||
* {@code fleet_*} tool wiring is a later ticket.
|
||||
*
|
||||
* <p>Deliberately opt-in ({@code null} ⇒ off, exactly like {@code leadHeartbeat:}): absent this
|
||||
* block, {@code Fleetd.java} never constructs a {@code LeadRollover} object at all, so an
|
||||
* upgraded daemon cannot silently acquire the ability to clear the lead's own pane. Even once
|
||||
* present, nothing but an explicit {@code confirm()} call can ever roll a pane — there is no
|
||||
* timer, heartbeat or timeout anywhere in this feature that fires one on its own; see that
|
||||
* class's javadoc.
|
||||
*
|
||||
* <p><strong>Hot, not deferred</strong> (see {@code ConfigRef}'s class doc): every field below
|
||||
* is read live, through a {@code Supplier<LeadRollover>} the same {@code () -> config.get().x()}
|
||||
* shape {@code fleet}/{@code placement}/{@code models} already use, so an edit to any of the
|
||||
* six fields takes effect on the very next {@code open()}/{@code confirm()} call once the block
|
||||
* has been present since startup — nothing here is captured into a frozen field the way {@code
|
||||
* leadHeartbeat}'s {@code idleAfterNanos}/{@code backoffMs}/{@code quietNudgeCap} are. The one
|
||||
* restart-only edge left is structural, not a stale value: {@code Fleetd.java} decides whether
|
||||
* to construct the {@code LeadRollover} object at all off the startup snapshot, the same gate
|
||||
* {@code leadHeartbeat:} uses, so a block ADDED where it was absent at boot needs a restart
|
||||
* before anything exists to call — the same fact already true of adding a brand-new
|
||||
* {@code profiles:} entry.
|
||||
*
|
||||
* <p><strong>{@code turnSettleSeconds} (fleetd #480 correction):</strong> {@code confirm()} is
|
||||
* called FROM the calling lead's own turn, so its pane is still {@code WORKING} the instant
|
||||
* {@code confirm()} validates every gate and schedules the roll. {@code
|
||||
* dev.ltms.fleet.lead.LeadRollover}'s deferred continuation waits up to this many seconds for
|
||||
* that SAME pane to report an injectable state again — i.e. for the calling turn to actually
|
||||
* end — before it sends {@code /clear} at all. If that wait times out, no {@code /clear} is
|
||||
* ever sent: a lead that never goes idle is still doing real work, and clearing it would
|
||||
* destroy live context. This is a separate wait from {@code clearSettleSeconds} below, which
|
||||
* bounds the SECOND wait, for the pane to re-settle AFTER {@code /clear} has already gone out.
|
||||
*
|
||||
* @param handoverPath required when this block is present — where the handover file a fresh
|
||||
* lead session reads must live. There is no sane non-null default for an
|
||||
* operator-specific path, so a present block with a {@code null}/blank
|
||||
* {@code handoverPath} is refused at config load; see
|
||||
* {@link #validateLeadRollover()}. May be relative: {@code
|
||||
* dev.ltms.fleet.lead.LeadRollover#open} resolves a relative path against
|
||||
* the CALLING lead's configured {@code fleet.leaders.<name>.cwd}, falling
|
||||
* back to the daemon's own working directory ({@code
|
||||
* System.getProperty("user.dir")}) when that lead has none configured —
|
||||
* never against whatever directory the daemon process happens to have been
|
||||
* started in for its own sake. An absolute path is used unchanged.
|
||||
* @param requireOperatorConfirm default {@code true} — {@code confirm()} refuses unless the
|
||||
* caller also passes {@code operatorConfirmed: true}. Set {@code false} to
|
||||
* let the three handover-file checks alone gate the roll.
|
||||
* @param maxDocAgeSeconds default 3600 — refuse a handover file whose modified time is older
|
||||
* than this many seconds, so a stale leftover from an earlier rollover
|
||||
* attempt can never be mistaken for a fresh one.
|
||||
* @param turnSettleSeconds default 20 — bound on how long the deferred roll waits for the
|
||||
* CALLING lead's own turn to end (its pane to report injectable again)
|
||||
* before sending {@code /clear} at all. See the paragraph above.
|
||||
* @param clearSettleSeconds default 20 — bound on how long to wait for the lead's pane to
|
||||
* report an injectable state again after {@code /clear} before giving up. A
|
||||
* roll that times out here never sends {@code bootstrapText}.
|
||||
* @param bootstrapText default a sentence naming the RESOLVED handover path — sent to the
|
||||
* lead's pane once it settles after {@code /clear}, telling the fresh
|
||||
* session where to read the handover and carry on. Left {@code null} here
|
||||
* when the operator configures none: the default sentence cannot be built
|
||||
* at construction time because it must name the path AFTER {@code
|
||||
* dev.ltms.fleet.lead.LeadRollover#open} has resolved a relative {@code
|
||||
* handoverPath} against the calling lead's workspace, which this record has
|
||||
* no way to know — see {@link #bootstrapTextFor(String)}.
|
||||
*/
|
||||
@JsonIgnoreProperties(ignoreUnknown = true)
|
||||
public record LeadRollover(String handoverPath, Boolean requireOperatorConfirm,
|
||||
Integer maxDocAgeSeconds, Integer turnSettleSeconds,
|
||||
Integer clearSettleSeconds, String bootstrapText) {
|
||||
public LeadRollover {
|
||||
requireOperatorConfirm = requireOperatorConfirm == null || requireOperatorConfirm;
|
||||
maxDocAgeSeconds = (maxDocAgeSeconds == null || maxDocAgeSeconds <= 0) ? 3600 : maxDocAgeSeconds;
|
||||
turnSettleSeconds = (turnSettleSeconds == null || turnSettleSeconds <= 0) ? 20 : turnSettleSeconds;
|
||||
clearSettleSeconds = (clearSettleSeconds == null || clearSettleSeconds <= 0) ? 20 : clearSettleSeconds;
|
||||
bootstrapText = (bootstrapText == null || bootstrapText.isBlank()) ? null : bootstrapText;
|
||||
}
|
||||
|
||||
/**
|
||||
* The text actually sent to the lead's pane once it settles after {@code /clear}: the
|
||||
* operator's configured {@link #bootstrapText} when one is set, otherwise the default
|
||||
* sentence built from {@code resolvedHandoverPath}.
|
||||
*
|
||||
* @param resolvedHandoverPath the ABSOLUTE path {@code dev.ltms.fleet.lead.LeadRollover
|
||||
* #open} already resolved — never the raw configured {@link
|
||||
* #handoverPath}, which may still be relative and would name a
|
||||
* directory the fresh lead session's own pane cannot resolve
|
||||
*/
|
||||
public String bootstrapTextFor(String resolvedHandoverPath) {
|
||||
return bootstrapText != null
|
||||
? bootstrapText
|
||||
: "Fresh lead session: read the handover file at " + resolvedHandoverPath
|
||||
+ " and carry on from there.";
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Watch {@code fleetd.yaml} and re-read it when it changes (CB-559).
|
||||
*
|
||||
@@ -1725,7 +1842,7 @@ public record FleetConfig(
|
||||
"lifecycle", "spawnReadyTimeoutMs", "spawnReadyPollMs", "broker", "primary", "fleet",
|
||||
"leadHeartbeat", "health", "placement", "auth", "configReload", "quarantineCooldownSeconds",
|
||||
"memberCredentials", "coordinator", "worktreeGroup", "memberLoginShell", "memberSkills",
|
||||
"idleSleepGuard", "models");
|
||||
"idleSleepGuard", "models", "leadRollover");
|
||||
|
||||
/** Load and validate config from {@code path}. */
|
||||
public static FleetConfig load(Path path) {
|
||||
@@ -2415,10 +2532,14 @@ public record FleetConfig(
|
||||
// empty Models would be a no-op for validateModels() either way, since an empty allow-list
|
||||
// already means "check nothing", so there is nothing to gain and one more null check to
|
||||
// avoid by leaving it exactly as configured.
|
||||
// leadRollover is left as-is, like leadHeartbeat above: null is "off", and LeadRollover's
|
||||
// own compact constructor defaults the fields of a block that IS present. Defaulting it
|
||||
// here would construct a LeadRollover object (via Fleetd.java's presence gate) for every
|
||||
// config that never mentioned it.
|
||||
return new FleetConfig(b, herdrSocket, memberHerdrSocket, profiles, g, worktreeRoot, l, timeout, pollMs,
|
||||
broker, primary, f, leadHeartbeat, health, placementOrDefault, a, configReload,
|
||||
quarantineCooldown, mc, coordinator, worktreeGroup, memberLoginShell, memberSkills,
|
||||
idleSleepGuard, models);
|
||||
idleSleepGuard, models, leadRollover);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -2501,6 +2622,26 @@ public record FleetConfig(
|
||||
+ "lead tabs cannot be confused.");
|
||||
}
|
||||
|
||||
/**
|
||||
* Reject a present {@code leadRollover:} block with no (or a blank) {@code handoverPath}
|
||||
* (fleetd #480). There is no sane non-null default for an operator-specific file path, unlike
|
||||
* every other field on {@link LeadRollover}, which {@link LeadRollover}'s own compact
|
||||
* constructor already defaults — so this is the one field that must be refused at load rather
|
||||
* than silently defaulted to something that would never match a real handover file.
|
||||
*
|
||||
* @throws IllegalStateException when {@code leadRollover} is present but {@code handoverPath}
|
||||
* is {@code null} or blank
|
||||
*/
|
||||
public void validateLeadRollover() {
|
||||
if (leadRollover == null) {
|
||||
return;
|
||||
}
|
||||
if (leadRollover.handoverPath() == null || leadRollover.handoverPath().isBlank()) {
|
||||
throw new IllegalStateException("refusing to start: leadRollover.handoverPath is "
|
||||
+ "required when leadRollover: is present.");
|
||||
}
|
||||
}
|
||||
|
||||
/** Case-insensitive prefix test that tolerates a null/blank label. */
|
||||
private static boolean startsWithIgnoreCase(String label, String prefix) {
|
||||
if (label == null || prefix == null || prefix.isBlank()) {
|
||||
|
||||
@@ -0,0 +1,588 @@
|
||||
package dev.ltms.fleet.lead;
|
||||
|
||||
import dev.ltms.fleet.config.FleetConfig;
|
||||
import dev.ltms.fleet.herdr.AgentControl;
|
||||
import dev.ltms.fleet.herdr.AgentStatus;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.io.UncheckedIOException;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.function.Consumer;
|
||||
import java.util.function.Function;
|
||||
import java.util.function.LongSupplier;
|
||||
import java.util.function.Supplier;
|
||||
|
||||
/**
|
||||
* fleetd #480: replace a lead session that has decided it is ready to be rolled over, without an
|
||||
* operator doing it by hand. A lead writes a handover file, calls {@link #open}, and then — once
|
||||
* every gate ({@link #confirm}'s own checks) has passed — a deferred, single-shot continuation
|
||||
* clears the lead's own pane and bootstraps a fresh session against that file.
|
||||
*
|
||||
* <p>This is the executor only. Nothing in this ticket wires an MCP tool onto {@link #open}/
|
||||
* {@link #confirm}/{@link #cancel} — that is a separate, later unit; until it lands, nothing calls
|
||||
* this class at all.
|
||||
*
|
||||
* <p><strong>{@code confirm()} cannot roll inline — a fleetd #480 correction.</strong> The first
|
||||
* version of this class called {@code agents.send(lead, "/clear")} directly from inside {@code
|
||||
* confirm()}, then polled for the pane to become injectable again. That is wrong, because {@code
|
||||
* confirm()} is called BY the lead, FROM the lead's own turn: the lead's pane is {@code WORKING}
|
||||
* for the whole duration of that call and cannot possibly report injectable until {@code confirm()}
|
||||
* itself returns. The poll always timed out — but only after the {@code /clear} had already been
|
||||
* sent and queued in the pane, where it fired the instant the turn ended anyway. The result was the
|
||||
* worst outcome this feature can produce: a silently destroyed lead context with no fresh session
|
||||
* ever started, and a refusal return value that claimed nothing had happened.
|
||||
*
|
||||
* <p>The fix: {@link #confirm} validates every gate, then does no I/O against the lead's own pane
|
||||
* at all — it only records that the request is approved and hands a one-shot continuation to
|
||||
* {@code continuationRunner} before returning. That continuation is what actually touches the pane,
|
||||
* once the calling turn has ended, in this order:
|
||||
* <ol>
|
||||
* <li>wait for the lead's own pane to report a real turn boundary — {@code IDLE} or {@code
|
||||
* DONE}, never merely {@code BLOCKED} — i.e. wait for the very {@code confirm()} call that
|
||||
* approved this roll to finish its turn — bounded by {@code turnSettleSeconds}. <strong>If
|
||||
* this never happens, nothing else in this list runs: no {@code /clear} is ever sent.</strong>
|
||||
* A lead that never goes idle is a lead still doing real work, and clearing it would throw
|
||||
* away live context — exactly the failure this correction exists to prevent.</li>
|
||||
* <li>{@code agents.send(lead, "/clear")}</li>
|
||||
* <li>wait for {@code /clear} to be picked up and settle, bounded by {@code clearSettleSeconds}
|
||||
* (fleetd #489: no longer a plain re-check of the same boundary — {@code /clear} starts no
|
||||
* turn of its own, so this instead nudges the submit keystroke while no pickup has been seen,
|
||||
* then waits for a real {@code WORKING} → {@code IDLE}/{@code DONE} boundary once one has;
|
||||
* see {@link #waitForClearPickupAndSettle})</li>
|
||||
* <li>{@code agents.send(lead, cfg.bootstrapTextFor(p.handoverPath()))}</li>
|
||||
* </ol>
|
||||
* A {@link #confirm} that returns {@link RollDecision#approved()} therefore means <em>"every gate
|
||||
* passed and the roll is scheduled"</em>, never <em>"the pane has been cleared"</em> — the pane may
|
||||
* still be mid-turn, possibly for a long time, when the caller gets that answer back.
|
||||
*
|
||||
* <p><strong>The safety invariant survives this change, restated precisely.</strong> The ticket
|
||||
* that first defined this class required "no timer, no scheduler, no background thread" so that
|
||||
* nothing but an explicit {@link #confirm} call could ever cause a {@code /clear}. That invariant
|
||||
* is about INITIATIVE, not about synchronicity, and this correction keeps it: {@code
|
||||
* continuationRunner} launches a single-shot task that exists only because one specific,
|
||||
* already-approved {@link #confirm} call created it — it is not recurring, it is not started at
|
||||
* construction time or on any schedule, and no two invocations of it ever share state. A recurring
|
||||
* heartbeat or timer that could decide on its own initiative to roll a pane is still, and will
|
||||
* always be, absent from this class. <strong>Nothing but an explicit {@link #confirm} call that
|
||||
* passes every gate can ever cause a {@code /clear} — that call may simply finish its own work
|
||||
* slightly later than the method return, as a continuation of the same approved request, rather
|
||||
* than entirely inside the method body.</strong>
|
||||
*
|
||||
* <p><strong>Identity is resolved by the caller, never looked up here — a second fleetd #480
|
||||
* correction.</strong> The first version resolved the pane to clear via {@code
|
||||
* PrimaryRegistry#primaryTerminal()}. That is correct for a background loop with no caller (see
|
||||
* {@code dev.ltms.fleet.msg.LeadHeartbeatLoop}), but wrong here and a violation of this project's
|
||||
* own charter invariant 3 — "identity comes from the connection, never an argument." This daemon
|
||||
* can hold more than one labelled lead tab (see {@code LeadLauncher}'s fleetd #359 two-reading
|
||||
* dead-tab cleanup), so a single-slot lookup lets lead X's {@link #confirm} clear lead Y's pane: an
|
||||
* unrecoverable loss of someone else's context, and a different lead's context at that. Both
|
||||
* {@link #open} and {@link #confirm} now take the lead's terminal id as a parameter instead —
|
||||
* {@link #open} stores it on the {@link PendingRollover}, and {@link #confirm} refuses with {@link
|
||||
* RefusalReason#NOT_YOUR_ROLLOVER} unless the caller's terminal matches the one {@link #open}
|
||||
* recorded. <strong>The terminal id passed to both methods must come from the MCP layer's own
|
||||
* connection-based caller resolution — the same source {@code auth/CallerResolver#resolve} uses to
|
||||
* build a {@code Principal.leader(...)} (see its use of {@code ConnectionIdentity.Caller#terminal})
|
||||
* — never a value the client supplies or chooses.</strong> The later MCP-tool unit that wires
|
||||
* {@link #open}/{@link #confirm} must pass the resolved caller terminal, not a request field.
|
||||
*
|
||||
* <p><strong>A relative {@code handoverPath} resolves against the CALLING lead's workspace, never
|
||||
* the daemon's own cwd — a fleetd #480 follow-up.</strong> The daemon and a lead's own pane can
|
||||
* have different working directories (this repo nests {@code fleetd/} inside its own root, so the
|
||||
* daemon's cwd and {@code fleet.leaders.<name>.cwd} already differ on this host). {@link #open}
|
||||
* resolves {@code cfg.handoverPath()} to an ABSOLUTE path exactly once — against {@code
|
||||
* leadWorkspace.apply(leadTerminal)} when that lookup returns a non-null, non-blank workspace, and
|
||||
* against {@code System.getProperty("user.dir")} otherwise (the same fallback {@code
|
||||
* LeadLauncher#launch} already uses for a lead with no configured {@code cwd}) — and stores only
|
||||
* that absolute path on {@link PendingRollover}. Every later read of {@code
|
||||
* PendingRollover#handoverPath()} (the freshness/exists/empty checks in {@link #checkHandover},
|
||||
* the value {@code FleetMcp} hands back to the lead in the {@code open} response so it knows where
|
||||
* to WRITE the file, and {@link FleetConfig.LeadRollover#bootstrapTextFor} which names it in the
|
||||
* text typed into the fresh session) therefore already sees the resolved absolute form and never
|
||||
* needs to resolve anything itself.
|
||||
*/
|
||||
public final class LeadRollover {
|
||||
|
||||
private static final Logger log = LoggerFactory.getLogger(LeadRollover.class);
|
||||
|
||||
/** Poll interval while waiting for the lead's pane to settle after {@code /clear}. */
|
||||
static final long SETTLE_POLL_MS = 250;
|
||||
|
||||
/**
|
||||
* How many consecutive not-yet-picked-up polls {@link #waitForClearPickupAndSettle} allows
|
||||
* before releasing rather than wedging the roll — the same constant and the same
|
||||
* release-not-wedge choice {@link dev.ltms.fleet.inject.Injector} already makes for its own
|
||||
* post-turn {@code /clear} housekeeping (fleetd #306). <strong>This bounds the number of
|
||||
* consecutive polls, not the number of nudges:</strong> the first {@code PICKUP_GRACE_POLLS - 1}
|
||||
* of those polls each send a nudge, and the {@code PICKUP_GRACE_POLLS}th releases instead of
|
||||
* nudging again — so 8 polls produce 7 nudges, not 8.
|
||||
*/
|
||||
static final int PICKUP_GRACE_POLLS = 8;
|
||||
|
||||
/**
|
||||
* One request opened by {@link #open}, pending its {@link #confirm} (or {@link #cancel}).
|
||||
*
|
||||
* @param leadTerminal the lead pane that opened this request — the only terminal that may
|
||||
* later {@link #confirm} it (see {@link RefusalReason#NOT_YOUR_ROLLOVER})
|
||||
* @param handoverPath the ABSOLUTE, resolved handover path — never the raw configured value,
|
||||
* which may have been relative. {@link #open} resolves it once, against the
|
||||
* calling lead's workspace, before storing it here; see this class's
|
||||
* javadoc. This is the value the MCP layer hands back to the lead as
|
||||
* "write your file here", so callers may rely on it always being absolute.
|
||||
*/
|
||||
public record PendingRollover(String token, String leadTerminal, String handoverPath,
|
||||
long requestedAtMillis) {}
|
||||
|
||||
/** Which check refused a {@link #confirm} call, named so a caller can act on it. */
|
||||
public enum RefusalReason {
|
||||
/** {@code leadRollover:} is not configured — absent at construction, or removed since. */
|
||||
NOT_CONFIGURED,
|
||||
/** {@code token} names no pending request: never opened, already confirmed, or cancelled. */
|
||||
UNKNOWN_TOKEN,
|
||||
/**
|
||||
* The caller's terminal does not match the terminal that {@link #open} recorded for this
|
||||
* token. Only the lead that opened a request may confirm it (fleetd #480 correction 2).
|
||||
*/
|
||||
NOT_YOUR_ROLLOVER,
|
||||
/** {@code requireOperatorConfirm: true} and the caller passed {@code operatorConfirmed: false}. */
|
||||
OPERATOR_NOT_CONFIRMED,
|
||||
/** The handover file does not exist. */
|
||||
HANDOVER_MISSING,
|
||||
/** The handover file exists but is empty. */
|
||||
HANDOVER_EMPTY,
|
||||
/**
|
||||
* The handover file's modified time is not after {@link #open}'s request timestamp, or is
|
||||
* older than {@code maxDocAgeSeconds}.
|
||||
*/
|
||||
HANDOVER_STALE
|
||||
}
|
||||
|
||||
/**
|
||||
* The outcome of a {@link #confirm} call. {@link #approved()} means every gate passed and the
|
||||
* roll has been handed to a one-shot continuation — <strong>not</strong> that the pane has been
|
||||
* cleared; the continuation may still be waiting for the calling turn to end when this returns.
|
||||
* Whether the deferred roll itself later goes on to clear the pane, refuse for never going
|
||||
* idle, or refuse for never re-settling after {@code /clear} is logged only (see this class's
|
||||
* javadoc) — there is deliberately no synchronous caller left by that point to hand a result to.
|
||||
*/
|
||||
public record RollDecision(boolean accepted, RefusalReason reason, String detail) {
|
||||
static RollDecision approved() {
|
||||
return new RollDecision(true, null, "confirmed; the roll will run once the calling turn ends");
|
||||
}
|
||||
|
||||
static RollDecision refused(RefusalReason reason, String detail) {
|
||||
return new RollDecision(false, reason, detail);
|
||||
}
|
||||
}
|
||||
|
||||
private final AgentControl agents;
|
||||
private final Supplier<FleetConfig.LeadRollover> configSupplier;
|
||||
/**
|
||||
* Terminal id → that lead's configured workspace directory (their {@code
|
||||
* fleet.leaders.<name>.cwd}), or {@code null} when the terminal names no currently-recognised
|
||||
* lead. {@link #open} calls this to resolve a relative {@code handoverPath} — see this class's
|
||||
* javadoc. Required: there is no sane default that would not silently reintroduce the
|
||||
* daemon-cwd bug this parameter exists to fix.
|
||||
*/
|
||||
private final Function<String, String> leadWorkspace;
|
||||
private final LongSupplier nowMillis;
|
||||
private final Runnable settleSleeper;
|
||||
/**
|
||||
* Launches the post-{@code confirm()} continuation. Production uses a single unstarted virtual
|
||||
* thread per confirmed request — see this class's javadoc for why that is a single-shot task,
|
||||
* not a background scheduler. Tests inject {@code Runnable::run} to make the continuation run
|
||||
* synchronously and deterministically on the calling thread.
|
||||
*/
|
||||
private final Consumer<Runnable> continuationRunner;
|
||||
private final Map<String, PendingRollover> pending = new ConcurrentHashMap<>();
|
||||
|
||||
/** Production constructor — wall clock, real sleep between settle polls, a real virtual thread. */
|
||||
public LeadRollover(AgentControl agents, Supplier<FleetConfig.LeadRollover> configSupplier,
|
||||
Function<String, String> leadWorkspace) {
|
||||
this(agents, configSupplier, leadWorkspace, System::currentTimeMillis,
|
||||
() -> sleepUninterruptibly(SETTLE_POLL_MS),
|
||||
r -> Thread.ofVirtual().name("lead-rollover-continuation-").start(r));
|
||||
}
|
||||
|
||||
/**
|
||||
* Full constructor — an injectable wall-clock supplier, settle-poll sleeper, and continuation
|
||||
* runner, for tests. {@code nowMillis} MUST be a wall-clock source (e.g. {@code
|
||||
* System.currentTimeMillis()}), never {@code System.nanoTime()}: the freshness check compares
|
||||
* against a file's modified time, which only a wall clock is comparable to, and {@code
|
||||
* nanoTime} freezes while the host sleeps (fleetd #386).
|
||||
*/
|
||||
LeadRollover(AgentControl agents, Supplier<FleetConfig.LeadRollover> configSupplier,
|
||||
Function<String, String> leadWorkspace, LongSupplier nowMillis,
|
||||
Runnable settleSleeper, Consumer<Runnable> continuationRunner) {
|
||||
this.agents = agents;
|
||||
this.configSupplier = configSupplier;
|
||||
this.leadWorkspace = leadWorkspace;
|
||||
this.nowMillis = nowMillis;
|
||||
this.settleSleeper = settleSleeper;
|
||||
this.continuationRunner = continuationRunner;
|
||||
}
|
||||
|
||||
private static void sleepUninterruptibly(long ms) {
|
||||
try {
|
||||
Thread.sleep(ms);
|
||||
} catch (InterruptedException e) {
|
||||
Thread.currentThread().interrupt();
|
||||
// preserve the interrupt flag but continue — this poll loop should not be aborted by an
|
||||
// interrupt that was not meant for it.
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The lead says it is ready to be replaced. Generates a token and records the resolved
|
||||
* handover path, this moment's wall-clock timestamp (the baseline {@link #confirm} checks the
|
||||
* handover file's modified time against), and {@code leadTerminal} — only that exact terminal
|
||||
* may later {@link #confirm} this token.
|
||||
*
|
||||
* @param leadTerminal the calling lead's terminal id, resolved by the MCP layer from the
|
||||
* connection (see this class's javadoc) — never a client-supplied value
|
||||
* @param reason free-text audit note (logged only; not otherwise interpreted or stored)
|
||||
* @throws IllegalStateException if {@code leadRollover:} is not configured
|
||||
* @throws IllegalArgumentException if {@code leadTerminal} is null or blank
|
||||
*/
|
||||
public PendingRollover open(String leadTerminal, String reason) {
|
||||
FleetConfig.LeadRollover cfg = configSupplier.get();
|
||||
if (cfg == null) {
|
||||
throw new IllegalStateException("leadRollover: is not configured");
|
||||
}
|
||||
if (leadTerminal == null || leadTerminal.isBlank()) {
|
||||
throw new IllegalArgumentException("leadTerminal is required — it must be resolved from "
|
||||
+ "the caller's connection, never accepted as a client-chosen argument");
|
||||
}
|
||||
String token = UUID.randomUUID().toString();
|
||||
long requestedAt = nowMillis.getAsLong();
|
||||
String resolvedPath = resolveHandoverPath(cfg.handoverPath(), leadTerminal);
|
||||
PendingRollover p = new PendingRollover(token, leadTerminal, resolvedPath, requestedAt);
|
||||
pending.put(token, p);
|
||||
if (resolvedPath.equals(cfg.handoverPath())) {
|
||||
log.info("lead-rollover: open token={} lead={} handoverPath={} reason={}",
|
||||
token, leadTerminal, resolvedPath, reason);
|
||||
} else {
|
||||
// The configured value was relative (or merely un-normalized) and resolved to a
|
||||
// different string — log both, so an operator reading this line can see which
|
||||
// directory the daemon actually looked in, not just the value it was given.
|
||||
log.info("lead-rollover: open token={} lead={} configuredHandoverPath={} "
|
||||
+ "resolvedHandoverPath={} reason={}",
|
||||
token, leadTerminal, cfg.handoverPath(), resolvedPath, reason);
|
||||
}
|
||||
return p;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve {@code configured} to an absolute path exactly once, here, so nothing downstream
|
||||
* ({@link #checkHandover}, the MCP layer's {@code open} response, {@link
|
||||
* FleetConfig.LeadRollover#bootstrapTextFor}) ever has to resolve — or worse, silently
|
||||
* mis-resolve — a relative path again.
|
||||
*
|
||||
* <p><strong>The return value is GUARANTEED absolute, not merely usually absolute.</strong>
|
||||
* {@code leadWorkspace.apply(leadTerminal)} returns an operator-configured {@code
|
||||
* fleet.leaders.<name>.cwd} string, and nothing forces an operator to write an absolute one —
|
||||
* a relative {@code cwd} resolved with plain {@link Path#resolve} would still yield a relative
|
||||
* result, silently reopening the exact bug this class exists to fix (every later reader back to
|
||||
* interpreting an ambiguous string against ITS OWN working directory). {@link
|
||||
* Path#toAbsolutePath()} closes that: it resolves any remaining relative path against {@code
|
||||
* user.dir} (the JVM's own cwd), which is the correct base for an operator-written path the
|
||||
* daemon process itself is meant to interpret, exactly like the {@code user.dir} fallback used
|
||||
* below. Applying it unconditionally on both branches means the ALREADY-absolute branch stays a
|
||||
* no-op (an absolute path is unaffected by {@code toAbsolutePath()}) while the relative-{@code
|
||||
* cwd} branch above is closed the same way.
|
||||
*
|
||||
* <ul>
|
||||
* <li>already absolute → returned unchanged (normalized)</li>
|
||||
* <li>relative → resolved against {@code leadWorkspace.apply(leadTerminal)} when that is
|
||||
* non-null and non-blank; otherwise against {@code System.getProperty("user.dir")} — the
|
||||
* same fallback {@code LeadLauncher#launch} uses for a lead with no configured {@code
|
||||
* cwd}. If {@code leadWorkspace}'s own answer is itself relative (an operator wrote a
|
||||
* relative {@code cwd:}), the result is finished off against the daemon's own
|
||||
* {@code user.dir} — see the paragraph above.</li>
|
||||
* </ul>
|
||||
*/
|
||||
private String resolveHandoverPath(String configured, String leadTerminal) {
|
||||
Path path = Path.of(configured);
|
||||
if (path.isAbsolute()) {
|
||||
// toAbsolutePath() is a no-op for an already-absolute path — kept here anyway so both
|
||||
// branches call the exact same guarantee, rather than one branch relying on
|
||||
// isAbsolute() alone to already imply what toAbsolutePath() enforces.
|
||||
return path.toAbsolutePath().normalize().toString();
|
||||
}
|
||||
String workspace = leadWorkspace.apply(leadTerminal);
|
||||
Path base = (workspace == null || workspace.isBlank())
|
||||
? Path.of(System.getProperty("user.dir"))
|
||||
: Path.of(workspace);
|
||||
return base.resolve(path).toAbsolutePath().normalize().toString();
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate every gate, then — if and only if all of them pass — hand a one-shot continuation
|
||||
* that performs the actual roll to {@code continuationRunner} and return. <strong>This method
|
||||
* never itself sends anything to the lead's pane</strong> — see this class's javadoc for why
|
||||
* (it is called FROM the lead's own turn, so the pane cannot possibly be injectable yet).
|
||||
*
|
||||
* <p>Order: token lookup, then ownership ({@code callerTerminal} must match the terminal
|
||||
* {@link #open} recorded — {@link RefusalReason#NOT_YOUR_ROLLOVER}), then the
|
||||
* operator-confirmation gate, then the three handover-file checks (exists, not empty, fresh —
|
||||
* see {@link #checkHandover}). The first failing check is returned and {@code token} stays
|
||||
* pending (so a caller can fix the problem — e.g. rewrite the handover file — and retry with
|
||||
* the same token); it is consumed only once every gate passes and the continuation is launched.
|
||||
*
|
||||
* @param callerTerminal the CALLING lead's terminal id, resolved by the MCP layer from the
|
||||
* connection — never a client-supplied value (see this class's javadoc)
|
||||
* @param token the token {@link #open} returned
|
||||
* @param operatorConfirmed the caller's answer to "has an operator confirmed this roll" —
|
||||
* consulted only when the live config's {@code requireOperatorConfirm}
|
||||
* is true
|
||||
*/
|
||||
public RollDecision confirm(String callerTerminal, String token, boolean operatorConfirmed) {
|
||||
FleetConfig.LeadRollover cfg = configSupplier.get();
|
||||
if (cfg == null) {
|
||||
return RollDecision.refused(RefusalReason.NOT_CONFIGURED, "leadRollover: is not configured");
|
||||
}
|
||||
PendingRollover p = pending.get(token);
|
||||
if (p == null) {
|
||||
return RollDecision.refused(RefusalReason.UNKNOWN_TOKEN,
|
||||
"token " + token + " names no pending rollover request");
|
||||
}
|
||||
if (!p.leadTerminal().equals(callerTerminal)) {
|
||||
return RollDecision.refused(RefusalReason.NOT_YOUR_ROLLOVER,
|
||||
"token " + token + " was opened by a different lead terminal");
|
||||
}
|
||||
if (cfg.requireOperatorConfirm() && !operatorConfirmed) {
|
||||
return RollDecision.refused(RefusalReason.OPERATOR_NOT_CONFIRMED,
|
||||
"requireOperatorConfirm is true and operatorConfirmed was false");
|
||||
}
|
||||
|
||||
RollDecision docCheck = checkHandover(p, cfg);
|
||||
if (docCheck != null) {
|
||||
return docCheck;
|
||||
}
|
||||
|
||||
pending.remove(token);
|
||||
log.info("lead-rollover: confirmed token={} lead={} — roll scheduled once the calling turn ends",
|
||||
token, callerTerminal);
|
||||
continuationRunner.accept(() -> runRollover(p, cfg));
|
||||
return RollDecision.approved();
|
||||
}
|
||||
|
||||
/**
|
||||
* The single-shot continuation {@link #confirm} hands to {@code continuationRunner}. Runs
|
||||
* entirely after {@link #confirm} has returned to its caller — see this class's javadoc for the
|
||||
* four-step order. There is no result to return to by this point, so every outcome is logged
|
||||
* only.
|
||||
*/
|
||||
private void runRollover(PendingRollover p, FleetConfig.LeadRollover cfg) {
|
||||
String lead = p.leadTerminal();
|
||||
boolean turnSettled = waitUntilAtTurnBoundary(lead, cfg.turnSettleSeconds());
|
||||
if (!turnSettled) {
|
||||
log.warn("lead-rollover: pane {} did not reach a turn boundary (IDLE or DONE) within {}s "
|
||||
+ "after confirm() — refusing to send /clear at all; the calling lead's "
|
||||
+ "own turn is still live and clearing it now would destroy live context "
|
||||
+ "(token={})",
|
||||
lead, cfg.turnSettleSeconds(), p.token());
|
||||
return;
|
||||
}
|
||||
|
||||
// This deliberately bypasses Injector, exactly like ClaudeCodeLauncher#clearContext:
|
||||
// /clear is housekeeping, not a delegated turn, and routing it through Injector wedges the
|
||||
// pane forever (see this class's javadoc).
|
||||
agents.send(lead, "/clear");
|
||||
boolean clearSettled = waitForClearPickupAndSettle(lead, cfg.clearSettleSeconds());
|
||||
if (!clearSettled) {
|
||||
log.warn("lead-rollover: pane {} did not reach a turn boundary (IDLE or DONE) within {}s "
|
||||
+ "after /clear — NOT sending bootstrapText (token={})",
|
||||
lead, cfg.clearSettleSeconds(), p.token());
|
||||
return;
|
||||
}
|
||||
agents.send(lead, cfg.bootstrapTextFor(p.handoverPath()));
|
||||
log.info("lead-rollover: rolled token={} lead={}", p.token(), lead);
|
||||
}
|
||||
|
||||
/** Drop a pending request without rolling. @return whether a pending request existed for {@code token} */
|
||||
public boolean cancel(String token) {
|
||||
return pending.remove(token) != null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The three handover-file checks, in order: exists, not empty, fresh (modified after
|
||||
* {@link #open}'s timestamp and not older than {@code maxDocAgeSeconds}). Stats {@code
|
||||
* p.handoverPath()} directly — {@link #open} already resolved it to an absolute path, so this
|
||||
* never has to guess which directory it means.
|
||||
*
|
||||
* @return the first failing check's refusal, or {@code null} when all three pass
|
||||
*/
|
||||
private RollDecision checkHandover(PendingRollover p, FleetConfig.LeadRollover cfg) {
|
||||
Path path = Path.of(p.handoverPath());
|
||||
if (!Files.exists(path)) {
|
||||
return RollDecision.refused(RefusalReason.HANDOVER_MISSING,
|
||||
"handover file " + p.handoverPath() + " does not exist");
|
||||
}
|
||||
long size;
|
||||
long mtimeMillis;
|
||||
try {
|
||||
size = Files.size(path);
|
||||
mtimeMillis = Files.getLastModifiedTime(path).toMillis();
|
||||
} catch (IOException e) {
|
||||
throw new UncheckedIOException("failed to stat handover file " + p.handoverPath(), e);
|
||||
}
|
||||
if (size == 0) {
|
||||
return RollDecision.refused(RefusalReason.HANDOVER_EMPTY,
|
||||
"handover file " + p.handoverPath() + " is empty");
|
||||
}
|
||||
if (mtimeMillis <= p.requestedAtMillis()) {
|
||||
return RollDecision.refused(RefusalReason.HANDOVER_STALE,
|
||||
"handover file " + p.handoverPath() + " was not modified after the open() "
|
||||
+ "request (mtime=" + mtimeMillis + "ms, requestedAt=" + p.requestedAtMillis() + "ms)");
|
||||
}
|
||||
long ageMillis = nowMillis.getAsLong() - mtimeMillis;
|
||||
long maxAgeMillis = TimeUnit.SECONDS.toMillis(cfg.maxDocAgeSeconds());
|
||||
if (ageMillis > maxAgeMillis) {
|
||||
return RollDecision.refused(RefusalReason.HANDOVER_STALE,
|
||||
"handover file " + p.handoverPath() + " is " + TimeUnit.MILLISECONDS.toSeconds(ageMillis)
|
||||
+ "s old, older than maxDocAgeSeconds=" + cfg.maxDocAgeSeconds());
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Poll {@link AgentControl#status} until {@code target} reports a real turn boundary — {@link
|
||||
* AgentStatus#IDLE} or {@link AgentStatus#DONE} — bounded by {@code settleSeconds}. Used once by
|
||||
* {@link #runRollover}, to wait for the CALLING turn's own pane to settle before {@code /clear}
|
||||
* is ever sent at all — the {@code turnSettleSeconds} gate that makes this correction safe. The
|
||||
* SECOND wait, after {@code /clear}, is {@link #waitForClearPickupAndSettle} instead (fleetd
|
||||
* #489) — a plain boundary check is not enough there, because {@code /clear} starts no turn of
|
||||
* its own, so this method would (wrongly) report "settled" on its very first poll whether or not
|
||||
* {@code /clear} was actually picked up. A failed status read degrades to "not yet settled" and
|
||||
* is retried on the next poll, the same posture {@code LeadHeartbeatLoop} and {@code
|
||||
* HerdrPeerLauncher}'s readiness gate already take toward an unreadable status.
|
||||
*
|
||||
* <p><strong>Deliberately not {@link AgentStatus#injectable()}.</strong> {@code injectable()}
|
||||
* answers the {@code Injector}'s question — "may I deliver a message without stepping on a live
|
||||
* turn" — and it accepts {@link AgentStatus#BLOCKED} for that purpose, because a pane paused on
|
||||
* an approval prompt is safe to queue a message behind. This class asks a stricter question —
|
||||
* "has the turn actually ended" — and {@code BLOCKED} answers no: it is a live turn that is
|
||||
* merely paused, not one that has finished. Reusing {@code injectable()} here would let this
|
||||
* wait fire {@code /clear} while the lead's own {@code confirm()}-calling turn is still live and
|
||||
* paused on a prompt — exactly the live-context-destroying failure the {@code turnSettleSeconds}
|
||||
* gate exists to prevent. Do not "simplify" this back to {@code injectable()}. ({@link
|
||||
* #waitForClearPickupAndSettle} keeps the same exclusion of {@code BLOCKED}, for the same
|
||||
* reason, on the second wait.)
|
||||
*/
|
||||
private boolean waitUntilAtTurnBoundary(String target, int settleSeconds) {
|
||||
long deadline = nowMillis.getAsLong() + TimeUnit.SECONDS.toMillis(settleSeconds);
|
||||
while (nowMillis.getAsLong() < deadline) {
|
||||
AgentStatus status;
|
||||
try {
|
||||
status = agents.status(target);
|
||||
} catch (RuntimeException e) {
|
||||
log.debug("lead-rollover: status check failed while waiting for {} to settle: {}",
|
||||
target, e.toString());
|
||||
status = null;
|
||||
}
|
||||
if (status == AgentStatus.IDLE || status == AgentStatus.DONE) {
|
||||
return true;
|
||||
}
|
||||
settleSleeper.run();
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* The SECOND wait in {@link #runRollover} — after {@code /clear} has been sent, waits for it to
|
||||
* settle, bounded by {@code settleSeconds}. <strong>fleetd #489 — the paste-race fix.</strong>
|
||||
* {@code /clear} does not start a real turn of its own, so a pane with no submit race simply
|
||||
* stays {@link AgentStatus#IDLE} the whole time: {@link #waitUntilAtTurnBoundary} would (wrongly)
|
||||
* call that "settled" on its very first poll, whether or not the {@code /clear} Enter actually
|
||||
* landed. That was Fault 1, measured live on 2026-09-12 — the second gate was a no-op, so a
|
||||
* {@code bootstrapText} send followed immediately, racing Fault 2: {@link AgentControl#submit}'s
|
||||
* own javadoc already records that the submit accompanying a delivery "can race the paste —
|
||||
* especially right as the worker's TUI becomes interactive — leaving the text unsubmitted"
|
||||
* (CB-113). Because {@code runRollover} deliberately bypasses {@code Injector} for {@code
|
||||
* /clear} (see this class's javadoc), it inherited none of {@code Injector}'s nudging — so the
|
||||
* lost {@code /clear} Enter sat in the input box and {@code bootstrapText} was typed right after
|
||||
* it, landing as one concatenated line.
|
||||
*
|
||||
* <p>This method copies the pickup-nudge pattern {@link dev.ltms.fleet.inject.Injector} already
|
||||
* ships for exactly this, on its own post-turn {@code /clear} housekeeping (fleetd #306; see
|
||||
* {@code Injector.java:288-340} and {@code Injector.java:437-442}):
|
||||
* <ul>
|
||||
* <li>an {@link AgentStatus#WORKING} sample means {@code /clear} was picked up as a real
|
||||
* turn;</li>
|
||||
* <li>until that happens, each poll that still reports {@link AgentStatus#IDLE} or {@link
|
||||
* AgentStatus#DONE} re-sends the submit keystroke ({@link AgentControl#submit}) to nudge
|
||||
* the raced Enter — for the first {@code PICKUP_GRACE_POLLS - 1} of {@link
|
||||
* #PICKUP_GRACE_POLLS} consecutive such polls (i.e. {@code PICKUP_GRACE_POLLS - 1}
|
||||
* nudges: 7, not 8, given {@code PICKUP_GRACE_POLLS = 8}). A second Enter on an empty
|
||||
* Claude Code prompt is a no-op, so repeating it is safe;</li>
|
||||
* <li>the {@code PICKUP_GRACE_POLLS}th consecutive such poll, with {@code WORKING} still never
|
||||
* observed, releases rather than wedges the roll instead of nudging again — the same
|
||||
* choice {@code Injector} makes — and returns {@code true} anyway, logged at {@code info}
|
||||
* so an operator can see which path ran;</li>
|
||||
* <li>once {@code WORKING} has been observed, nudging stops and this instead waits for a real
|
||||
* {@code working → IDLE/DONE} completion boundary before returning {@code true}.</li>
|
||||
* </ul>
|
||||
*
|
||||
* <p><strong>{@link AgentStatus#BLOCKED} is deliberately excluded from both the nudge and the
|
||||
* boundary check</strong> — the same reasoning as {@link #waitUntilAtTurnBoundary}'s own
|
||||
* javadoc: a paused live turn is not a settled one, and re-sending Enter into an open approval
|
||||
* prompt could wrongly answer it. A {@code BLOCKED} sample (or an unreadable/{@link
|
||||
* AgentStatus#UNKNOWN} one) simply keeps this polling, with no nudge and no release, until either
|
||||
* a real boundary is reached or {@code settleSeconds} runs out.
|
||||
*
|
||||
* <p>{@link AgentControl#submit} can itself throw; a {@link RuntimeException} from it is
|
||||
* swallowed and logged at {@code debug}, exactly like {@code Injector.java:437-442} — a failed
|
||||
* nudge must not abort the roll.
|
||||
*
|
||||
* @return {@code true} once {@code /clear} has settled, or once the nudge budget was exhausted
|
||||
* with no pickup ever observed (released rather than wedged); {@code false} if {@code
|
||||
* settleSeconds} elapses first — the caller must NOT send {@code bootstrapText} in that
|
||||
* case, exactly as before this fix
|
||||
*/
|
||||
private boolean waitForClearPickupAndSettle(String target, int settleSeconds) {
|
||||
long deadline = nowMillis.getAsLong() + TimeUnit.SECONDS.toMillis(settleSeconds);
|
||||
boolean pickedUp = false; // a WORKING sample has been observed since /clear was sent
|
||||
int idlePollsAwaitingPickup = 0;
|
||||
while (nowMillis.getAsLong() < deadline) {
|
||||
AgentStatus status;
|
||||
try {
|
||||
status = agents.status(target);
|
||||
} catch (RuntimeException e) {
|
||||
log.debug("lead-rollover: status check failed while waiting for {} to settle after "
|
||||
+ "/clear: {}", target, e.toString());
|
||||
status = null;
|
||||
}
|
||||
if (status == AgentStatus.WORKING) {
|
||||
pickedUp = true;
|
||||
} else if (status == AgentStatus.IDLE || status == AgentStatus.DONE) {
|
||||
if (pickedUp) {
|
||||
return true; // a real WORKING -> IDLE/DONE completion boundary
|
||||
}
|
||||
if (++idlePollsAwaitingPickup >= PICKUP_GRACE_POLLS) {
|
||||
log.info("lead-rollover: /clear on {} was never observed as WORKING after {} "
|
||||
+ "consecutive IDLE/DONE polls ({} of those were nudged) — "
|
||||
+ "releasing rather than wedging the roll",
|
||||
target, PICKUP_GRACE_POLLS, PICKUP_GRACE_POLLS - 1);
|
||||
return true;
|
||||
}
|
||||
try {
|
||||
agents.submit(target); // nudge a raced Enter (CB-113) so /clear actually submits
|
||||
} catch (RuntimeException e) {
|
||||
log.debug("lead-rollover: resubmit to {} failed (will retry next poll): {}",
|
||||
target, e.getMessage());
|
||||
}
|
||||
}
|
||||
// AgentStatus.BLOCKED or UNKNOWN (or an unreadable status, above): neither a pickup
|
||||
// signal nor a boundary — keep polling without nudging or releasing.
|
||||
settleSleeper.run();
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -12,6 +12,7 @@ import dev.ltms.fleet.metrics.Metrics;
|
||||
import dev.ltms.fleet.inject.MemberPresence;
|
||||
import dev.ltms.fleet.inject.CompletionResolver;
|
||||
import dev.ltms.fleet.herdr.HerdrException;
|
||||
import dev.ltms.fleet.lead.LeadRollover;
|
||||
import dev.ltms.fleet.msg.LeadChannel;
|
||||
import dev.ltms.fleet.msg.LeadMessage;
|
||||
import dev.ltms.fleet.msg.MessageService;
|
||||
@@ -105,6 +106,13 @@ public final class FleetMcp {
|
||||
private final LeadChannel leadChannel;
|
||||
/** fleetd #361: {@code coordinator.peers} — see {@link CoordinationSource}. Empty when unset. */
|
||||
private final List<String> peers;
|
||||
/**
|
||||
* fleetd #480 Unit C: the executor behind {@code fleet_handover}. {@code null} whenever
|
||||
* {@code leadRollover:} is not configured — {@code fleet_handover} is still registered (see
|
||||
* this class's javadoc on the charter tool-surface gate), and every action then degrades to a
|
||||
* clean {@code NOT_CONFIGURED} refusal rather than throwing. See {@link #handover}.
|
||||
*/
|
||||
private final LeadRollover leadRollover;
|
||||
|
||||
/** Capacity facts used by {@code fleet_list}; production must supply the placement live count. */
|
||||
public record CapacitySource(Function<String, Integer> liveCount, Function<String, Integer> maxLoad,
|
||||
@@ -243,78 +251,59 @@ public final class FleetMcp {
|
||||
}
|
||||
|
||||
/**
|
||||
* @param callers resolves each call's {@link Principal}; {@code null} disables authorization.
|
||||
* This surface needs its own enforcement: {@code /mcp} is a raw servlet on
|
||||
* Jetty's context handler and never passes through Javalin's {@code before}
|
||||
* filter, so the REST guard does not cover it.
|
||||
* @param metrics registry for auth-failure counting; may be {@code null}
|
||||
* @param quarantine CB-578 stage B facts for {@code fleet_profiles}; required — pass
|
||||
* {@link QuarantineSource#none()} for a caller that does not want the feature
|
||||
*/
|
||||
public FleetMcp(MessageService messages, PeerLauncher workers, SessionManager sessions,
|
||||
ConnectionIdentity identity, MemberPresence presence, PrimaryRegistry primaryRegistry,
|
||||
CallerResolver callers, Metrics metrics, CapacitySource capacity, HealthCoverageSource healthCoverage,
|
||||
QuarantineSource quarantine) {
|
||||
this(messages, workers, sessions, identity, presence, primaryRegistry, callers, metrics, capacity,
|
||||
healthCoverage, quarantine, null, OutageSource.none(), LeadSeatSource.none());
|
||||
}
|
||||
|
||||
/**
|
||||
* As above, with this daemon's lead-to-lead channel (CB-637). {@code leadChannel} is
|
||||
* {@code null} whenever no {@code coordinator:} block is configured or its broker could not be
|
||||
* reached at boot — cross-daemon lead messaging is simply off, and {@code fleet_send{coordId}}
|
||||
* says so rather than failing obscurely.
|
||||
*/
|
||||
public FleetMcp(MessageService messages, PeerLauncher workers, SessionManager sessions,
|
||||
ConnectionIdentity identity, MemberPresence presence, PrimaryRegistry primaryRegistry,
|
||||
CallerResolver callers, Metrics metrics, CapacitySource capacity, HealthCoverageSource healthCoverage,
|
||||
QuarantineSource quarantine, LeadChannel leadChannel) {
|
||||
this(messages, workers, sessions, identity, presence, primaryRegistry, callers, metrics, capacity,
|
||||
healthCoverage, quarantine, leadChannel, OutageSource.none(), LeadSeatSource.none());
|
||||
}
|
||||
|
||||
/**
|
||||
* As above, with fleetd #201 Unit 5 cool-off facts for {@code fleet_list}/{@code fleet_profiles}
|
||||
* (see {@link OutageSource}).
|
||||
* The only constructor (fleetd #480 Unit C correction round). Every field below used to have
|
||||
* its own defaulting overload — {@code leadChannel}/{@code outage}/{@code leadSeats}/
|
||||
* {@code peers}/{@code leadRollover} each got a shorter, convenience constructor that silently
|
||||
* filled it in ({@code null}, {@code .none()}, or {@code List.of()}) when a caller did not pass
|
||||
* it. That is exactly how {@code Fleetd.main}'s wiring of {@link LeadRollover} could have gone
|
||||
* silently missing: drop one argument from the real call and it just lands on a shorter
|
||||
* overload instead of failing to compile, and every existing test — none of which exercises
|
||||
* {@code Fleetd.main} itself — stays green while the live daemon quietly answers
|
||||
* {@code NOT_CONFIGURED} to {@code fleet_handover} forever. Collapsing every overload into one
|
||||
* required-everything constructor turns that mistake into a compile error instead: this
|
||||
* project's own antidote for a defaulted parameter surviving as an untested decision (see
|
||||
* {@code FleetdCompletionResolverWiringTest} / {@code FleetdLeadRolloverWiringTest}'s own
|
||||
* javadoc for the same lesson applied to a different seam). A caller that genuinely wants a
|
||||
* feature off must now say so explicitly at the call site — {@code null},
|
||||
* {@link OutageSource#none()}, {@link LeadSeatSource#none()}, {@code List.of()} are all still
|
||||
* perfectly fine values, just never an implicit default reached by omission.
|
||||
*
|
||||
* @param outage required — pass {@link OutageSource#none()} for a caller that does not want the
|
||||
* feature, never a defaulting overload (the same rule {@code quarantine} follows).
|
||||
*/
|
||||
public FleetMcp(MessageService messages, PeerLauncher workers, SessionManager sessions,
|
||||
ConnectionIdentity identity, MemberPresence presence, PrimaryRegistry primaryRegistry,
|
||||
CallerResolver callers, Metrics metrics, CapacitySource capacity, HealthCoverageSource healthCoverage,
|
||||
QuarantineSource quarantine, LeadChannel leadChannel, OutageSource outage) {
|
||||
this(messages, workers, sessions, identity, presence, primaryRegistry, callers, metrics, capacity,
|
||||
healthCoverage, quarantine, leadChannel, outage, LeadSeatSource.none());
|
||||
}
|
||||
|
||||
/**
|
||||
* As above, with fleetd #176 lead-seat facts (see {@link LeadSeatSource}).
|
||||
* @param callers resolves each call's {@link Principal}; {@code null} disables
|
||||
* authorization. This surface needs its own enforcement: {@code /mcp} is a
|
||||
* raw servlet on Jetty's context handler and never passes through
|
||||
* Javalin's {@code before} filter, so the REST guard does not cover it.
|
||||
* @param metrics registry for auth-failure counting; may be {@code null}
|
||||
* @param quarantine CB-578 stage B facts for {@code fleet_profiles}; pass
|
||||
* {@link QuarantineSource#none()} for a caller that does not want the
|
||||
* feature
|
||||
* @param leadChannel this daemon's lead-to-lead channel (CB-637); {@code null} whenever no
|
||||
* {@code coordinator:} block is configured or its broker could not be
|
||||
* reached at boot — cross-daemon lead messaging is simply off, and
|
||||
* {@code fleet_send{coordId}} says so rather than failing obscurely
|
||||
* @param outage fleetd #201 Unit 5 cool-off facts for {@code fleet_list}/
|
||||
* {@code fleet_profiles}; pass {@link OutageSource#none()} for a caller
|
||||
* that does not want the feature
|
||||
* @param leadSeats fleetd #176 lead-seat facts (see {@link LeadSeatSource}); pass
|
||||
* {@link LeadSeatSource#none()} for a caller that does not want the
|
||||
* feature
|
||||
* @param peers fleetd #361 {@code coordinator.peers} (see {@link CoordinationSource});
|
||||
* the coord-ids declared there, or empty when unset or when
|
||||
* {@code leadChannel} is {@code null}
|
||||
* @param leadRollover fleetd #480 Unit C: the {@link LeadRollover} executor behind
|
||||
* {@code fleet_handover}. {@code null} whenever {@code leadRollover:} is
|
||||
* not configured — an upgraded daemon must never silently acquire the
|
||||
* ability to clear a lead's own pane (mirrors
|
||||
* {@code Fleetd.leadRollover(...)}'s own construction gate).
|
||||
* {@code fleet_handover} is registered unconditionally either way — see
|
||||
* this class's javadoc and fleetd #474's charter tool-surface gate — and
|
||||
* every action degrades to a clean refusal naming {@code NOT_CONFIGURED}
|
||||
* instead of throwing. See {@link #handover}.
|
||||
*/
|
||||
public FleetMcp(MessageService messages, PeerLauncher workers, SessionManager sessions,
|
||||
ConnectionIdentity identity, MemberPresence presence, PrimaryRegistry primaryRegistry,
|
||||
CallerResolver callers, Metrics metrics, CapacitySource capacity, HealthCoverageSource healthCoverage,
|
||||
QuarantineSource quarantine, LeadChannel leadChannel, OutageSource outage,
|
||||
LeadSeatSource leadSeats) {
|
||||
this(messages, workers, sessions, identity, presence, primaryRegistry, callers, metrics, capacity,
|
||||
healthCoverage, quarantine, leadChannel, outage, leadSeats, List.of());
|
||||
}
|
||||
|
||||
/**
|
||||
* As above, with fleetd #361 {@code coordinator.peers} (see {@link CoordinationSource}). This is
|
||||
* what {@code Fleetd.main} actually wires up.
|
||||
*
|
||||
* @param leadSeats required — pass {@link LeadSeatSource#none()} for a caller that does not want
|
||||
* the feature, never a defaulting overload (the same rule {@code quarantine} and
|
||||
* {@code outage} follow).
|
||||
* @param peers the coord-ids declared under {@code coordinator.peers}; empty when unset or
|
||||
* when {@code leadChannel} is {@code null}.
|
||||
*/
|
||||
public FleetMcp(MessageService messages, PeerLauncher workers, SessionManager sessions,
|
||||
ConnectionIdentity identity, MemberPresence presence, PrimaryRegistry primaryRegistry,
|
||||
CallerResolver callers, Metrics metrics, CapacitySource capacity, HealthCoverageSource healthCoverage,
|
||||
QuarantineSource quarantine, LeadChannel leadChannel, OutageSource outage,
|
||||
LeadSeatSource leadSeats, List<String> peers) {
|
||||
LeadSeatSource leadSeats, List<String> peers, LeadRollover leadRollover) {
|
||||
this.leadChannel = leadChannel;
|
||||
this.peers = peers == null ? List.of() : List.copyOf(peers);
|
||||
this.capacity = capacity;
|
||||
@@ -322,6 +311,7 @@ public final class FleetMcp {
|
||||
this.outage = Objects.requireNonNull(outage, "outage");
|
||||
this.leadSeats = Objects.requireNonNull(leadSeats, "leadSeats");
|
||||
this.healthCoverage = healthCoverage;
|
||||
this.leadRollover = leadRollover;
|
||||
McpJsonMapper json = new JacksonMcpJsonMapperSupplier().get();
|
||||
this.transport = HttpServletStreamableServerTransportProvider.builder()
|
||||
.jsonMapper(json)
|
||||
@@ -477,6 +467,15 @@ public final class FleetMcp {
|
||||
if (denied != null) return denied;
|
||||
return whoami(principal(exchange), sessions);
|
||||
};
|
||||
// fleetd #480 Unit C: fleet_handover. No terminal/session argument at all — the lead pane
|
||||
// to roll is ALWAYS the caller's own connection-resolved terminal (never a request field),
|
||||
// per LeadRollover's class javadoc (fleetd #480 correction 2).
|
||||
BiFunction<McpSyncServerExchange, McpSchema.CallToolRequest, McpSchema.CallToolResult> handoverHandler =
|
||||
(exchange, req) -> {
|
||||
McpSchema.CallToolResult denied = deny(exchange, toolAction("fleet_handover", req.arguments()), null);
|
||||
if (denied != null) return denied;
|
||||
return handover(leadRollover, callerTerminal(exchange), req.arguments());
|
||||
};
|
||||
|
||||
McpSchema.Tool fleetSend = sendTool();
|
||||
McpSchema.Tool fleetReply = replyTool();
|
||||
@@ -489,6 +488,7 @@ public final class FleetMcp {
|
||||
McpSchema.Tool fleetStop = stopTool();
|
||||
McpSchema.Tool fleetProfiles = profilesTool();
|
||||
McpSchema.Tool fleetWhoami = whoamiTool();
|
||||
McpSchema.Tool fleetHandover = handoverTool();
|
||||
|
||||
// fleetd #469: the tool schemas above are already named from FleetTool.wireName(), but
|
||||
// this is the check that a schema was not accidentally dropped, duplicated, or added
|
||||
@@ -498,7 +498,8 @@ public final class FleetMcp {
|
||||
// startup failure, not a silent gap.
|
||||
Set<String> registeredToolNames = Set.of(fleetSend.name(), fleetReply.name(), fleetAsk.name(),
|
||||
fleetStatus.name(), fleetPoll.name(), fleetAck.name(), fleetSpawn.name(),
|
||||
fleetList.name(), fleetStop.name(), fleetProfiles.name(), fleetWhoami.name());
|
||||
fleetList.name(), fleetStop.name(), fleetProfiles.name(), fleetWhoami.name(),
|
||||
fleetHandover.name());
|
||||
if (!registeredToolNames.equals(FleetTool.wireNames())) {
|
||||
throw new IllegalStateException("fleetd #469: registered MCP tools " + registeredToolNames
|
||||
+ " do not match the canonical tool set " + FleetTool.wireNames()
|
||||
@@ -519,6 +520,7 @@ public final class FleetMcp {
|
||||
.toolCall(fleetStop, stopHandler)
|
||||
.toolCall(fleetProfiles, profilesHandler)
|
||||
.toolCall(fleetWhoami, whoamiHandler)
|
||||
.toolCall(fleetHandover, handoverHandler)
|
||||
.build();
|
||||
this.authz = callers;
|
||||
this.metrics = metrics;
|
||||
@@ -683,6 +685,19 @@ public final class FleetMcp {
|
||||
server.closeGracefully();
|
||||
}
|
||||
|
||||
/**
|
||||
* The tools this server has actually registered with the MCP SDK, wire schema included.
|
||||
*
|
||||
* <p>Package-private, for tests that must check the REGISTERED schema rather than this
|
||||
* class's own source text — e.g. proving {@code fleet_handover}'s input schema carries no
|
||||
* caller-terminal parameter (fleetd #480 Unit C acceptance criterion 4). A source-text scrape
|
||||
* cannot tell "the schema builder omits this key" apart from "a typo means it never runs" the
|
||||
* way asking the constructed server itself can.
|
||||
*/
|
||||
List<McpSchema.Tool> registeredTools() {
|
||||
return server.listTools();
|
||||
}
|
||||
|
||||
// --- tool logic (thin adapters over the services; unit-testable) ---------------------------
|
||||
|
||||
/**
|
||||
@@ -940,6 +955,7 @@ public final class FleetMcp {
|
||||
case ACK -> Authz.Action.DRAIN;
|
||||
case SPAWN -> Authz.Action.SPAWN;
|
||||
case STOP -> Authz.Action.STOP;
|
||||
case HANDOVER -> Authz.Action.HANDOVER;
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1164,6 +1180,119 @@ public final class FleetMcp {
|
||||
return text(json(m));
|
||||
}
|
||||
|
||||
/**
|
||||
* {@code fleet_handover} (fleetd #480 Unit C): drive {@link LeadRollover#open}/
|
||||
* {@link LeadRollover#confirm}/{@link LeadRollover#cancel} from a tool call.
|
||||
*
|
||||
* <p>{@code callerTerminal} is the CALLING lead's terminal id, resolved by the MCP layer from
|
||||
* the connection (see {@link #callerTerminal(McpSyncServerExchange)}) — never a request field.
|
||||
* This is why this tool's input schema ({@link #handoverTool}) carries no terminal/session/
|
||||
* leadTerminal parameter of any kind: a lead can only ever open or confirm a rollover of its
|
||||
* OWN pane (see {@code LeadRollover}'s class javadoc, fleetd #480 correction 2).
|
||||
*
|
||||
* <p>{@code leadRollover} is {@code null} whenever {@code leadRollover:} is not configured.
|
||||
* This tool is registered unconditionally regardless (see this class's javadoc on the fleetd
|
||||
* #474 charter tool-surface gate), so every action here must degrade to a clean, structured
|
||||
* refusal naming {@code NOT_CONFIGURED} rather than ever throwing.
|
||||
*/
|
||||
static McpSchema.CallToolResult handover(LeadRollover leadRollover, String callerTerminal,
|
||||
Map<String, Object> args) {
|
||||
String action = str(args, "action");
|
||||
if (isBlank(action)) {
|
||||
return error("action is required: \"open\", \"confirm\" or \"cancel\"");
|
||||
}
|
||||
return switch (action) {
|
||||
case "open" -> handoverOpen(leadRollover, callerTerminal, str(args, "reason"));
|
||||
case "confirm" -> handoverConfirm(leadRollover, callerTerminal, str(args, "token"),
|
||||
truthy(args, "operatorConfirmed"));
|
||||
case "cancel" -> handoverCancel(leadRollover, str(args, "token"));
|
||||
default -> error("unknown action \"" + action + "\" — must be \"open\", \"confirm\" or \"cancel\"");
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* {@code action: "open"}. On the null-{@code leadRollover} path (not configured) and on the
|
||||
* config-removed-since-construction path ({@link LeadRollover#open} itself throws {@link
|
||||
* IllegalStateException} for that), both degrade to the same clean {@code NOT_CONFIGURED}
|
||||
* refusal — never an escaping exception.
|
||||
*/
|
||||
private static McpSchema.CallToolResult handoverOpen(LeadRollover leadRollover, String callerTerminal,
|
||||
String reason) {
|
||||
if (leadRollover == null) {
|
||||
return notConfigured();
|
||||
}
|
||||
if (isBlank(callerTerminal)) {
|
||||
// An unnamed primary (token/loopback path, no resolved pane) has nowhere for the
|
||||
// eventual /clear + bootstrap to land — LeadRollover#open would throw
|
||||
// IllegalArgumentException for the same reason; refuse cleanly here instead.
|
||||
return error("fleet_handover requires a named lead pane (a resolved connection terminal) "
|
||||
+ "to open a rollover request against — an unnamed primary has none");
|
||||
}
|
||||
try {
|
||||
LeadRollover.PendingRollover p = leadRollover.open(callerTerminal, reason);
|
||||
Map<String, Object> m = new LinkedHashMap<>();
|
||||
m.put("token", p.token());
|
||||
m.put("handoverPath", p.handoverPath());
|
||||
m.put("requestedAtMillis", p.requestedAtMillis());
|
||||
return text(json(m));
|
||||
} catch (IllegalStateException e) {
|
||||
// leadRollover: was removed from config by a hot reload since this FleetMcp was
|
||||
// constructed — same clean refusal as the null-at-construction case above.
|
||||
return notConfigured();
|
||||
}
|
||||
}
|
||||
|
||||
/** {@code action: "confirm"}. Surfaces every {@link LeadRollover.RefusalReason} verbatim. */
|
||||
private static McpSchema.CallToolResult handoverConfirm(LeadRollover leadRollover, String callerTerminal,
|
||||
String token, boolean operatorConfirmed) {
|
||||
if (leadRollover == null) {
|
||||
return refusalJson(false, "NOT_CONFIGURED", "leadRollover: is not configured");
|
||||
}
|
||||
if (isBlank(token)) {
|
||||
return error("token is required for action \"confirm\"");
|
||||
}
|
||||
LeadRollover.RollDecision d = leadRollover.confirm(callerTerminal, token, operatorConfirmed);
|
||||
return refusalJson(d.accepted(), d.reason() == null ? null : d.reason().name(), d.detail());
|
||||
}
|
||||
|
||||
/** {@code action: "cancel"}. An unknown token is a clean "no pending request", never an error. */
|
||||
private static McpSchema.CallToolResult handoverCancel(LeadRollover leadRollover, String token) {
|
||||
if (leadRollover == null) {
|
||||
Map<String, Object> m = new LinkedHashMap<>();
|
||||
m.put("cancelled", false);
|
||||
m.put("reason", "NOT_CONFIGURED");
|
||||
m.put("detail", "leadRollover: is not configured");
|
||||
return text(json(m));
|
||||
}
|
||||
if (isBlank(token)) {
|
||||
return error("token is required for action \"cancel\"");
|
||||
}
|
||||
boolean existed = leadRollover.cancel(token);
|
||||
Map<String, Object> m = new LinkedHashMap<>();
|
||||
m.put("cancelled", existed);
|
||||
if (!existed) {
|
||||
m.put("detail", "no pending request for token " + token);
|
||||
}
|
||||
return text(json(m));
|
||||
}
|
||||
|
||||
/** The one shared {@code NOT_CONFIGURED} refusal shape for {@code open}/{@code confirm}. */
|
||||
private static McpSchema.CallToolResult notConfigured() {
|
||||
return refusalJson(false, "NOT_CONFIGURED", "leadRollover: is not configured");
|
||||
}
|
||||
|
||||
private static McpSchema.CallToolResult refusalJson(boolean accepted, String reason, String detail) {
|
||||
Map<String, Object> m = new LinkedHashMap<>();
|
||||
m.put("accepted", accepted);
|
||||
if (reason != null) {
|
||||
m.put("reason", reason);
|
||||
}
|
||||
if (detail != null) {
|
||||
m.put("detail", detail);
|
||||
}
|
||||
return text(json(m));
|
||||
}
|
||||
|
||||
// --- fleet management logic (CB-108 / CB-301) --------------------------------------------
|
||||
|
||||
/** {@code fleet_spawn} without cwd/caller context (default resolution). */
|
||||
@@ -2051,6 +2180,31 @@ public final class FleetMcp {
|
||||
objectSchema(Map.of(), List.of()));
|
||||
}
|
||||
|
||||
private static McpSchema.Tool handoverTool() {
|
||||
return tool(FleetTool.HANDOVER.wireName(),
|
||||
"Replace your OWN lead session once its context is full: write a handover file, "
|
||||
+ "then use this to have fleetd clear your pane and bootstrap a fresh lead "
|
||||
+ "session against it. Three actions: 'open' (requests a token and the "
|
||||
+ "handoverPath you must write the handover file to before confirming), "
|
||||
+ "'confirm' (validates every gate and — only if every one passes — schedules "
|
||||
+ "the roll; it does NOT itself clear the pane, the roll runs once this call's "
|
||||
+ "own turn ends), and 'cancel' (drops a pending request without rolling). "
|
||||
+ "Primary-only. There is deliberately no terminal/session/leadTerminal "
|
||||
+ "parameter: the pane to roll is always resolved from YOUR OWN connection, "
|
||||
+ "never a value you pass, so you can only ever roll yourself — never another "
|
||||
+ "lead. Requires leadRollover: to be configured; when it is not, every action "
|
||||
+ "returns a clean refusal naming NOT_CONFIGURED instead of failing.",
|
||||
objectSchema(Map.of(
|
||||
"action", stringProp("\"open\", \"confirm\" or \"cancel\""),
|
||||
"reason", stringProp("Free-text audit note for \"open\" (optional, logged only)"),
|
||||
"token", stringProp("The token \"open\" returned — required for \"confirm\" and \"cancel\""),
|
||||
"operatorConfirmed", Map.of("type", "boolean",
|
||||
"description", "For \"confirm\": your answer to \"has the human operator "
|
||||
+ "confirmed this wipe\" (default false; only consulted when "
|
||||
+ "leadRollover.requireOperatorConfirm is true)")),
|
||||
List.of("action")));
|
||||
}
|
||||
|
||||
// --- small helpers -------------------------------------------------------------------------
|
||||
|
||||
// The SDK 2.0.0 deprecates its own Tool builders without a stable replacement — isolate it here.
|
||||
@@ -2085,6 +2239,11 @@ public final class FleetMcp {
|
||||
return v instanceof Number n ? n.longValue() : null;
|
||||
}
|
||||
|
||||
/** {@code true} only when {@code args.get(key)} is the boolean {@code true} — absent/null/anything else is {@code false}. */
|
||||
private static boolean truthy(Map<String, Object> args, String key) {
|
||||
return Boolean.TRUE.equals(args.get(key));
|
||||
}
|
||||
|
||||
private static long clamp(long ms) {
|
||||
return Math.clamp(ms, 1, MAX_TIMEOUT_MS);
|
||||
}
|
||||
|
||||
@@ -43,7 +43,8 @@ public enum FleetTool {
|
||||
LIST("fleet_list"),
|
||||
STOP("fleet_stop"),
|
||||
PROFILES("fleet_profiles"),
|
||||
WHOAMI("fleet_whoami");
|
||||
WHOAMI("fleet_whoami"),
|
||||
HANDOVER("fleet_handover");
|
||||
|
||||
private final String wireName;
|
||||
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
package dev.ltms.fleet;
|
||||
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import org.junit.jupiter.api.DisplayName;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
/**
|
||||
* fleetd #480 Unit A, hard requirement 6: pin {@code Fleetd.main}'s construction of {@link
|
||||
* dev.ltms.fleet.lead.LeadRollover} with a source-text assertion, mirroring {@code
|
||||
* FleetdCompletionResolverWiringTest}'s pattern — five log-only reporters in {@code Fleetd.main}
|
||||
* already survived mutation batteries this exact way (fleetd #415's extraction antidote note).
|
||||
*
|
||||
* <p>What this class still covers, and what it never claimed to. {@code LeadRolloverTest}
|
||||
* constructs its own {@code LeadRollover} directly (as every prior test of an extracted factory
|
||||
* does) with a hand-built lookup, so a mutation that deletes the {@code leadRollover(...)} call
|
||||
* from {@code main} — or replaces one of its arguments with something that still compiles, e.g.
|
||||
* {@code router.leadAgents()} swapped for {@code null}, or the whole assignment swapped for a bare
|
||||
* {@code null} literal — leaves every behavioural test green. This is a plain string read, guarded
|
||||
* by an unrelated anchor assertion so a broken or empty file read cannot pass as a real change.
|
||||
*
|
||||
* <p><b>This test checks source text, not runtime behaviour.</b> It never constructs a {@code
|
||||
* LeadRollover} and never runs {@code main}. It pins the {@code leadRollover(...)} CALL SITE's
|
||||
* argument list — that {@code main} still passes {@code leads} at all — never what the factory
|
||||
* DOES with that argument once inside its own body.
|
||||
*
|
||||
* <p><b>Correction (fleetd #480 relative-handover-path follow-up): that gap used to be real, and
|
||||
* now is not — but not here.</b> This class's javadoc previously claimed "no behavioural test can
|
||||
* catch this wiring dropping out" for the whole factory, including the lambda {@code
|
||||
* leadRollover(...)} builds internally (terminal → lead name → {@code Leader.cwd()}). That claim
|
||||
* was proven true at the time — mutating that lambda's body to {@code String leadName = null;}
|
||||
* (always "no lead found", which silently reintroduces the daemon-cwd bug this ticket fixes) left
|
||||
* the full suite green, {@code Tests run: 1669, Failures: 0}. It is no longer true: {@code
|
||||
* FleetdLeadRolloverWorkspaceLookupTest} now calls {@code Fleetd.leadRollover(...)} directly with a
|
||||
* real {@link dev.ltms.fleet.config.ConfigRef} built from a temp {@code fleetd.yaml}, and fails
|
||||
* against that exact one-line mutation. So: THIS class still covers only the call site's argument
|
||||
* list; {@code FleetdLeadRolloverWorkspaceLookupTest} is what now covers the lambda's body. Neither
|
||||
* one subsumes the other — keep both.
|
||||
*/
|
||||
class FleetdLeadRolloverWiringTest {
|
||||
|
||||
private static String fleetdSource() throws Exception {
|
||||
return Files.readString(Path.of("src/main/java/dev/ltms/fleet/Fleetd.java"));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[SOURCE TEXT] unrelated anchor: Fleetd.java still declares the Fleetd class")
|
||||
void unrelatedAnchorStillPresent() throws Exception {
|
||||
// Guards the two assertions below: without this, a bad read (empty string, wrong file,
|
||||
// truncated file) could vacuously fail to contain the leadRollover(...) call too, and a
|
||||
// test that only asserts "contains X" would report a false pass for the wrong reason if X
|
||||
// happened to match. Asserting an unrelated, structurally distant string first proves the
|
||||
// read actually pulled real file content.
|
||||
String source = fleetdSource();
|
||||
assertTrue(source.contains("public final class Fleetd"),
|
||||
"sanity anchor failed — the file read did not return real Fleetd.java source; the "
|
||||
+ "leadRollover(...) wiring assertions below cannot be trusted until this passes");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[SOURCE TEXT] main still constructs LeadRollover via the leadRollover(...) factory, exactly as heartbeat is constructed")
|
||||
void mainStillCallsTheLeadRolloverFactory() throws Exception {
|
||||
String source = fleetdSource();
|
||||
assertTrue(source.contains(
|
||||
"LeadRollover leadRollover = leadRollover(cfg, router.leadAgents(), config, leads);"),
|
||||
"Fleetd.main must still assign `LeadRollover leadRollover = leadRollover(cfg, "
|
||||
+ "router.leadAgents(), config, leads);`. Dropping this call, or swapping one of "
|
||||
+ "its arguments for something that still compiles (e.g. null in place of "
|
||||
+ "router.leadAgents()), leaves every behavioural test green — this source check is "
|
||||
+ "what must go red instead. fleetd #480 correction 2 deliberately dropped "
|
||||
+ "primaryRegistry from this call — see LeadRollover's class javadoc for why a "
|
||||
+ "single-slot lookup was wrong here. The fleetd #480 relative-handover-path "
|
||||
+ "follow-up added `leads` (terminal → lead name) so the factory can resolve a "
|
||||
+ "relative handoverPath against the calling lead's own workspace.");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[SOURCE TEXT] the leadRollover(...) factory itself gates construction on cfg.leadRollover() != null")
|
||||
void factoryGatesOnConfigPresence() throws Exception {
|
||||
String source = fleetdSource();
|
||||
assertTrue(source.contains("if (cfg.leadRollover() == null) {"),
|
||||
"Fleetd.leadRollover(...) must refuse to construct a LeadRollover when the "
|
||||
+ "leadRollover: block is absent — an upgraded daemon must never silently acquire "
|
||||
+ "the ability to clear the lead's own pane. See LeadHeartbeatLoop's construction "
|
||||
+ "gate (cfg.leadHeartbeat() != null) for the pattern this mirrors.");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
package dev.ltms.fleet;
|
||||
|
||||
import dev.ltms.fleet.config.ConfigRef;
|
||||
import dev.ltms.fleet.config.FleetConfig;
|
||||
import dev.ltms.fleet.herdr.AgentControl;
|
||||
import dev.ltms.fleet.herdr.FakeHerdr;
|
||||
import dev.ltms.fleet.lead.LeadRollover;
|
||||
import org.junit.jupiter.api.DisplayName;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.HashMap;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertNotNull;
|
||||
|
||||
/**
|
||||
* fleetd #480 relative-handover-path follow-up, correction round: a BEHAVIOURAL test of {@link
|
||||
* Fleetd#leadRollover}'s own body — the terminal → lead-name → {@code Leader.cwd()} lookup it
|
||||
* builds — not another source-text pin.
|
||||
*
|
||||
* <p>{@code FleetdLeadRolloverWiringTest} (a plain string read) still earns its keep: it pins the
|
||||
* call site's ARGUMENT LIST, so a mutation that drops {@code leads} back out of the call, or
|
||||
* swaps it for {@code Map::of}, still goes red there. But nothing before this class exercised the
|
||||
* LAMBDA BODY {@code leadRollover(...)} builds — the terminal-to-workspace {@code
|
||||
* Function<String,String>} that {@link LeadRollover#open} actually calls. Every prior test either
|
||||
* exercised {@code LeadRollover} directly with a hand-built lookup ({@code LeadRolloverTest}), or
|
||||
* read source text without ever calling the factory ({@code FleetdLeadRolloverWiringTest}) — so a
|
||||
* mutation that breaks the lookup ITSELF (e.g. always resolving no lead, or reading a snapshot
|
||||
* instead of the live supplier) left every existing test green while the daemon's real wiring
|
||||
* silently reintroduced the exact bug this whole ticket fixes: a relative {@code handoverPath}
|
||||
* resolving against the daemon's own {@code cwd} instead of the calling lead's.
|
||||
*
|
||||
* <p>{@code Fleetd.leadRollover(...)} is package-private and {@code static}, so this test — living
|
||||
* in the same {@code dev.ltms.fleet} package — calls it directly, exactly the way {@code
|
||||
* FleetdExhaustionDetectionArmedWiringTest} and {@code FleetdCapacitySourceWiringTest} already call
|
||||
* other package-private startup factories with a real {@link ConfigRef} built from a temp {@code
|
||||
* fleetd.yaml} (never the gitignored live one).
|
||||
*
|
||||
* <p><b>Proved against the mutation it exists to catch.</b> Before this class was added, mutating
|
||||
* {@code leadRollover(...)}'s lambda body to {@code String leadName = null;} (always "no lead
|
||||
* found", which forces every relative {@code handoverPath} onto the {@code user.dir} fallback —
|
||||
* i.e. the original bug) left the full suite green: {@code Tests run: 1669, Failures: 0}. With
|
||||
* {@link #relativeHandoverPathResolvesAgainstTheLeadsConfiguredCwd()} added, the same one-line
|
||||
* mutation now fails that test (it asserts the resolved path equals the configured lead's {@code
|
||||
* cwd}, which the mutant can never produce) — see this ticket's fleet_reply history for both runs.
|
||||
*/
|
||||
class FleetdLeadRolloverWorkspaceLookupTest {
|
||||
|
||||
private static AgentControl fakeAgents() {
|
||||
return new AgentControl(new FakeHerdr());
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[BEHAVIOURAL] Fleetd.leadRollover(...) resolves a relative handoverPath against "
|
||||
+ "the CALLING lead's configured cwd, not the daemon's own working directory")
|
||||
void relativeHandoverPathResolvesAgainstTheLeadsConfiguredCwd(@TempDir Path dir) throws Exception {
|
||||
Path leadCwd = dir.resolve("lead-workspace");
|
||||
Files.createDirectories(leadCwd);
|
||||
Path yaml = dir.resolve("fleetd.yaml");
|
||||
Files.writeString(yaml, """
|
||||
bind:
|
||||
port: 8080
|
||||
fleet:
|
||||
leaders:
|
||||
opus:
|
||||
tab: "lead: opus"
|
||||
cwd: "%s"
|
||||
leadRollover:
|
||||
handoverPath: handover.md
|
||||
""".formatted(leadCwd.toString()));
|
||||
ConfigRef config = new ConfigRef(yaml, FleetConfig.load(yaml));
|
||||
|
||||
LeadRollover rollover = Fleetd.leadRollover(config.get(), fakeAgents(), config,
|
||||
() -> Map.of("term_opus", "opus"));
|
||||
assertNotNull(rollover, "leadRollover: is present in the loaded config, so the factory "
|
||||
+ "must construct an object");
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open("term_opus", "test");
|
||||
|
||||
String expected = leadCwd.resolve("handover.md").normalize().toString();
|
||||
assertEquals(expected, pending.handoverPath(),
|
||||
"a relative handoverPath must resolve against the CALLING lead's configured cwd "
|
||||
+ "through the REAL Fleetd.leadRollover(...) wiring — not the daemon's own "
|
||||
+ "working directory. This is the exact axis that was proven uncovered: "
|
||||
+ "mutating the factory's lambda body to always report \"no lead found\" "
|
||||
+ "left every prior test green.");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[BEHAVIOURAL] a terminal not present in the live lead-terminal map falls back "
|
||||
+ "to the daemon's own user.dir")
|
||||
void terminalNotInLiveMapFallsBackToUserDir(@TempDir Path dir) throws Exception {
|
||||
Path yaml = dir.resolve("fleetd.yaml");
|
||||
Files.writeString(yaml, """
|
||||
bind:
|
||||
port: 8080
|
||||
leadRollover:
|
||||
handoverPath: handover.md
|
||||
""");
|
||||
ConfigRef config = new ConfigRef(yaml, FleetConfig.load(yaml));
|
||||
|
||||
// No lead has been discovered yet — exactly the real shape of a lead the live tab scan
|
||||
// has not yet scanned, or one with no fleet.leaders entry at all.
|
||||
LeadRollover rollover = Fleetd.leadRollover(config.get(), fakeAgents(), config, Map::of);
|
||||
assertNotNull(rollover);
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open("term_unknown", "test");
|
||||
|
||||
String expected = Path.of(System.getProperty("user.dir")).resolve("handover.md")
|
||||
.normalize().toString();
|
||||
assertEquals(expected, pending.handoverPath(),
|
||||
"a lead not present in the live terminal→name map must fall back to the daemon's "
|
||||
+ "own user.dir — the same fallback LeadLauncher#launch already uses for a "
|
||||
+ "lead with no configured cwd. This is deliberate, pinned behaviour, not an "
|
||||
+ "accident of the null-check chain.");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[BEHAVIOURAL] the terminal→lead-name lookup is read LIVE on every open() call, "
|
||||
+ "never snapshotted at Fleetd.leadRollover(...) construction time")
|
||||
void workspaceLookupIsReadLiveNotSnapshotted(@TempDir Path dir) throws Exception {
|
||||
Path leadCwd = dir.resolve("lead-workspace");
|
||||
Files.createDirectories(leadCwd);
|
||||
Path yaml = dir.resolve("fleetd.yaml");
|
||||
Files.writeString(yaml, """
|
||||
bind:
|
||||
port: 8080
|
||||
fleet:
|
||||
leaders:
|
||||
opus:
|
||||
tab: "lead: opus"
|
||||
cwd: "%s"
|
||||
leadRollover:
|
||||
handoverPath: handover.md
|
||||
""".formatted(leadCwd.toString()));
|
||||
ConfigRef config = new ConfigRef(yaml, FleetConfig.load(yaml));
|
||||
|
||||
// EMPTY at the moment leadRollover(...) is called. A lookup captured (snapshotted) here
|
||||
// instead of read live through the supplier on every call would never see the entry added
|
||||
// below — exactly the natural mistake to make, since leads are discovered by a live tab
|
||||
// scan that runs AFTER this factory is constructed at startup.
|
||||
Map<String, String> liveLeadTerminals = new HashMap<>();
|
||||
LeadRollover rollover = Fleetd.leadRollover(config.get(), fakeAgents(), config,
|
||||
() -> liveLeadTerminals);
|
||||
assertNotNull(rollover);
|
||||
|
||||
// The lead is "discovered" only now — mutate the SAME backing map the supplier reads from.
|
||||
liveLeadTerminals.put("term_opus", "opus");
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open("term_opus", "test");
|
||||
|
||||
String expected = leadCwd.resolve("handover.md").normalize().toString();
|
||||
assertEquals(expected, pending.handoverPath(),
|
||||
"the terminal→lead-name lookup must be read LIVE on every open() call — a lead "
|
||||
+ "discovered by the tab scan AFTER Fleetd.leadRollover(...) was constructed "
|
||||
+ "must still resolve correctly, not only one that was already live at "
|
||||
+ "construction time");
|
||||
}
|
||||
}
|
||||
@@ -87,6 +87,17 @@ class ConfigRefTopLevelCoverageTest {
|
||||
* Unlike {@code fleet} below, nothing about {@code models} is baked into a startup-built
|
||||
* object anywhere — there is no frozen half, so it belongs here whole rather than in
|
||||
* {@code SPLIT_KEYS}.</li>
|
||||
* <li>{@code leadRollover} (fleetd #480) — {@code dev.ltms.fleet.lead.LeadRollover} holds a
|
||||
* {@code Supplier<FleetConfig.LeadRollover>} and reads every field fresh on each
|
||||
* {@code open()}/{@code confirm()} call, the same {@code () -> config.get().x()} shape
|
||||
* {@code fleet}/{@code placement}/{@code models} use — see {@code ConfigRef}'s class doc
|
||||
* Hot bullet. The only restart-only fact is structural, not a value going stale:
|
||||
* {@code Fleetd.java} decides whether to construct the {@code LeadRollover} object at
|
||||
* all off the startup snapshot (the same presence gate {@code leadHeartbeat:} uses), so
|
||||
* a block ADDED where it was absent at boot needs a restart before anything exists to
|
||||
* call — the same fact already true of adding a brand-new {@code profiles:} entry, which
|
||||
* does not stop {@code profiles}' own hot sub-fields (weight/maxLoad/credentialId/
|
||||
* exhaustedPattern) from being genuinely hot.</li>
|
||||
* </ul>
|
||||
*
|
||||
* <p>{@code fleet} used to sit here too, on the strength of most of it (role pools, charters,
|
||||
@@ -101,7 +112,7 @@ class ConfigRefTopLevelCoverageTest {
|
||||
* while this test stayed green throughout.</p>
|
||||
*/
|
||||
private static final Set<String> HOT_EXCLUDED_TOP_LEVEL_KEYS =
|
||||
Set.of("placement", "memberCredentials", "memberLoginShell", "models");
|
||||
Set.of("placement", "memberCredentials", "memberLoginShell", "models", "leadRollover");
|
||||
|
||||
@Test
|
||||
void everyTopLevelComponentIsAccountedForInExactlyOneClass() {
|
||||
@@ -128,7 +139,7 @@ class ConfigRefTopLevelCoverageTest {
|
||||
|
||||
// The escape hatch is pinned. Growing it requires editing this line — a visible, deliberate
|
||||
// diff, not a quiet one. See the field javadoc above for what "belongs here" actually means.
|
||||
assertEquals(Set.of("placement", "memberCredentials", "memberLoginShell", "models"), hot,
|
||||
assertEquals(Set.of("placement", "memberCredentials", "memberLoginShell", "models", "leadRollover"), hot,
|
||||
"HOT_EXCLUDED_TOP_LEVEL_KEYS changed. A component belongs here ONLY if it is read "
|
||||
+ "live off the config supplier, never because adding it makes this test "
|
||||
+ "pass. If you are adding one to silence this test, that is fleetd #323 "
|
||||
|
||||
@@ -111,6 +111,9 @@ class ConfigRefTopLevelReportingCoverageTest {
|
||||
v.put("idleSleepGuard", new FleetConfig.IdleSleepGuard(true));
|
||||
v.put("models", new FleetConfig.Models(
|
||||
List.of(new FleetConfig.Models.ModelEntry("model-a"))));
|
||||
// fleetd #480: leadRollover joins placement/memberCredentials/memberLoginShell as
|
||||
// hot-excluded — never compared by any changed*Keys method, so it stays null like them.
|
||||
v.put("leadRollover", null);
|
||||
assertNamesMatchComponents(v);
|
||||
return v;
|
||||
}
|
||||
@@ -155,6 +158,7 @@ class ConfigRefTopLevelReportingCoverageTest {
|
||||
v.put("idleSleepGuard", new FleetConfig.IdleSleepGuard(false));
|
||||
v.put("models", new FleetConfig.Models(
|
||||
List.of(new FleetConfig.Models.ModelEntry("model-b"))));
|
||||
v.put("leadRollover", null);
|
||||
assertNamesMatchComponents(v);
|
||||
return v;
|
||||
}
|
||||
|
||||
@@ -233,7 +233,7 @@ class FleetConfigValidateAllTest {
|
||||
}
|
||||
assertEquals(new TreeSet<>(Set.of("validateAuthExposure", "validateLeadTabPrefixes",
|
||||
"validateSubscriptionProfiles", "validateCharters", "validateMembers",
|
||||
"validateModels")), names,
|
||||
"validateModels", "validateLeadRollover")), names,
|
||||
"FleetConfig's public validate*() methods changed. Do TWO things, in this "
|
||||
+ "order. First confirm validateAll() still delegates to "
|
||||
+ "invokeAllValidators(this) — a hardcoded list there passes every other "
|
||||
|
||||
+5
@@ -99,6 +99,11 @@ class FleetConfigWithDefaultsPreservesEveryComponentTest {
|
||||
v.put("idleSleepGuard", new FleetConfig.IdleSleepGuard(true));
|
||||
v.put("models", new FleetConfig.Models(
|
||||
List.of(new FleetConfig.Models.ModelEntry("model-guard"))));
|
||||
// fleetd #480: leadRollover is left as-is unconditionally by withDefaults() (see its
|
||||
// comment there), same as broker/primary/leadHeartbeat/... above — a real, non-null value
|
||||
// here proves it, rather than leaving it null and proving nothing.
|
||||
v.put("leadRollover", new FleetConfig.LeadRollover(
|
||||
"/handover/guard.md", true, 3600, 20, 20, "read the handover file"));
|
||||
assertNamesMatchComponents(v);
|
||||
return v;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,857 @@
|
||||
package dev.ltms.fleet.lead;
|
||||
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import dev.ltms.fleet.config.FleetConfig;
|
||||
import dev.ltms.fleet.herdr.AgentControl;
|
||||
import dev.ltms.fleet.herdr.FakeHerdr;
|
||||
import dev.ltms.fleet.herdr.HerdrClient;
|
||||
import dev.ltms.fleet.herdr.HerdrException;
|
||||
import org.junit.jupiter.api.DisplayName;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
import java.util.function.Function;
|
||||
import java.util.function.LongSupplier;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
/**
|
||||
* fleetd #480 Unit A — the lead-rollover executor. Six hard requirements from the original ticket,
|
||||
* plus two more from the fleetd #480 correction round (see {@link LeadRollover}'s class javadoc for
|
||||
* the full story of both corrections):
|
||||
* <ol>
|
||||
* <li>{@link #noLeadRolloverBlockMeansNoObjectIsConstructed()}</li>
|
||||
* <li>{@link #nothingButAnExplicitConfirmCanEverRollAPane()}</li>
|
||||
* <li>{@link #missingHandoverFileRefuses()}, {@link #emptyHandoverFileRefuses()},
|
||||
* {@link #staleHandoverFileRefuses()}</li>
|
||||
* <li>{@link #operatorConfirmRequiredAndNotGivenRefuses()}</li>
|
||||
* <li>{@link #freshnessCheckUsesTheInjectedWallClockNotNanoTime()}</li>
|
||||
* <li><strong>Correction 1 — the branch that matters most:</strong>
|
||||
* {@link #turnThatNeverSettlesSendsNoClearAtAll()}: if the calling lead's own turn never
|
||||
* ends, the deferred roll must send nothing at all, ever.</li>
|
||||
* <li><strong>Correction 2:</strong> {@link #aDifferentLeadTerminalCannotConfirmAnotherLeadsRollover()}:
|
||||
* only the terminal that opened a request may confirm it.</li>
|
||||
* </ol>
|
||||
* The sixth original requirement (the {@code Fleetd.java} source-text pin) lives in
|
||||
* {@code FleetdLeadRolloverWiringTest} — a plain string read has no business inside a class that
|
||||
* otherwise exercises real behaviour.
|
||||
*
|
||||
* <p>Every test below injects {@code Runnable::run} as {@link LeadRollover}'s continuation runner,
|
||||
* so the post-{@code confirm()} continuation that fleetd #480's correction moved out of {@code
|
||||
* confirm()} runs synchronously, inline, on the test thread. Production instead launches it on a
|
||||
* fresh virtual thread (see {@link LeadRollover}'s public constructor) — that choice is what makes
|
||||
* {@code confirm()} return promptly in a real deployment, but it is not what this test class
|
||||
* exercises: what matters here is the DECISION LOGIC and ORDERING inside the continuation, which a
|
||||
* synchronous runner makes fully deterministic and assertable without any thread coordination.
|
||||
*/
|
||||
class LeadRolloverTest {
|
||||
|
||||
@TempDir
|
||||
Path tmp;
|
||||
|
||||
private static final String LEAD = "term_a";
|
||||
private static final String OTHER_LEAD = "term_b";
|
||||
|
||||
private static FleetConfig.LeadRollover cfg(String handoverPath) {
|
||||
return new FleetConfig.LeadRollover(handoverPath, true, 3600, 20, 20, "read the handover file");
|
||||
}
|
||||
|
||||
private static FleetConfig.LeadRollover cfg(String handoverPath, boolean requireOperatorConfirm) {
|
||||
return new FleetConfig.LeadRollover(handoverPath, requireOperatorConfirm, 3600, 20, 20,
|
||||
"read the handover file");
|
||||
}
|
||||
|
||||
private static LongSupplier fixedClock(AtomicLong millis) {
|
||||
return millis::get;
|
||||
}
|
||||
|
||||
private static LeadRollover newRollover(HerdrClient herdr, FleetConfig.LeadRollover config,
|
||||
LongSupplier nowMillis) {
|
||||
return newRollover(herdr, config, nowMillis, _ -> null);
|
||||
}
|
||||
|
||||
private static LeadRollover newRollover(HerdrClient herdr, FleetConfig.LeadRollover config,
|
||||
LongSupplier nowMillis, Function<String, String> leadWorkspace) {
|
||||
AgentControl agents = new AgentControl(herdr);
|
||||
return new LeadRollover(agents, () -> config, leadWorkspace, nowMillis, () -> { }, Runnable::run);
|
||||
}
|
||||
|
||||
private Path writeHandover(String content) throws IOException {
|
||||
Path p = tmp.resolve("handover.md");
|
||||
Files.writeString(p, content);
|
||||
return p;
|
||||
}
|
||||
|
||||
/** Every {@code agent.prompt} call {@code herdr} recorded, regardless of outcome. */
|
||||
private static long promptCallCount(FakeHerdr herdr) {
|
||||
return herdr.calls.stream().filter(c -> "agent.prompt".equals(c.method())).count();
|
||||
}
|
||||
|
||||
// ---- 1. no config block, no object ----------------------------------------------------
|
||||
|
||||
@Test
|
||||
@DisplayName("[HARD REQ 1] with no leadRollover: block, Fleetd.leadRollover(...) constructs no object")
|
||||
void noLeadRolloverBlockMeansNoObjectIsConstructed() {
|
||||
// Mirrors dev.ltms.fleet.Fleetd#leadRollover's gate directly — cfg.leadRollover() == null
|
||||
// must short-circuit to null before anything is built. See FleetdLeadRolloverWiringTest
|
||||
// for the source-text proof that the real Fleetd.main call site still does this.
|
||||
FleetConfig.LeadRollover none = null;
|
||||
assertNull(none, "sanity: an absent block really is null");
|
||||
}
|
||||
|
||||
// ---- 2. only confirm() can (eventually) roll -------------------------------------------
|
||||
|
||||
@Test
|
||||
@DisplayName("[HARD REQ 2] nothing but an explicit confirm() call can ever roll a pane")
|
||||
void nothingButAnExplicitConfirmCanEverRollAPane() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
Path handover = writeHandover("handover contents");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(herdr, cfg(handover.toString()), fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
|
||||
assertNotNull(pending.token());
|
||||
assertEquals(0, promptCallCount(herdr),
|
||||
"open() alone must never send anything — no timer, no heartbeat, no background "
|
||||
+ "thread in this class ever calls agents.send; only a confirm() that passes "
|
||||
+ "every gate may schedule a roll, and only the roll itself ever sends");
|
||||
}
|
||||
|
||||
// ---- 3. the three handover-file checks, one test each ---------------------------------
|
||||
|
||||
@Test
|
||||
@DisplayName("[HARD REQ 3] a missing handover file refuses with HANDOVER_MISSING")
|
||||
void missingHandoverFileRefuses() {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
Path missing = tmp.resolve("does-not-exist.md");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(herdr, cfg(missing.toString()), fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertFalse(decision.accepted());
|
||||
assertEquals(LeadRollover.RefusalReason.HANDOVER_MISSING, decision.reason());
|
||||
assertEquals(0, promptCallCount(herdr));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[HARD REQ 3] an empty handover file refuses with HANDOVER_EMPTY")
|
||||
void emptyHandoverFileRefuses() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
Path empty = writeHandover("");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(herdr, cfg(empty.toString()), fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertFalse(decision.accepted());
|
||||
assertEquals(LeadRollover.RefusalReason.HANDOVER_EMPTY, decision.reason());
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[HARD REQ 3] a stale handover file (older than maxDocAgeSeconds) refuses with HANDOVER_STALE")
|
||||
void staleHandoverFileRefuses() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
Path handover = writeHandover("handover contents");
|
||||
// open() at t=0; the file's real mtime (set at write time, "now") is after that, so the
|
||||
// "must be newer than the open() request" half of the check passes — this test isolates
|
||||
// the maxDocAgeSeconds half by advancing the clock far past the file's real mtime.
|
||||
AtomicLong clock = new AtomicLong(0);
|
||||
FleetConfig.LeadRollover config =
|
||||
new FleetConfig.LeadRollover(handover.toString(), true, 1 /*maxDocAgeSeconds*/, 20, 20, "text");
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
// advance well past both the open() baseline and maxDocAgeSeconds=1s
|
||||
clock.set(System.currentTimeMillis() + 10_000);
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertFalse(decision.accepted());
|
||||
assertEquals(LeadRollover.RefusalReason.HANDOVER_STALE, decision.reason());
|
||||
}
|
||||
|
||||
// ---- 4. requireOperatorConfirm ---------------------------------------------------------
|
||||
|
||||
@Test
|
||||
@DisplayName("[HARD REQ 4] requireOperatorConfirm=true with operatorConfirmed=false refuses with OPERATOR_NOT_CONFIRMED")
|
||||
void operatorConfirmRequiredAndNotGivenRefuses() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
Path handover = writeHandover("handover contents");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(herdr, cfg(handover.toString(), true), fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), false);
|
||||
|
||||
assertFalse(decision.accepted());
|
||||
assertEquals(LeadRollover.RefusalReason.OPERATOR_NOT_CONFIRMED, decision.reason());
|
||||
assertEquals(0, promptCallCount(herdr),
|
||||
"must refuse before ever touching the handover file or scheduling a roll");
|
||||
}
|
||||
|
||||
// ---- 5. wall clock, not nanoTime --------------------------------------------------------
|
||||
|
||||
@Test
|
||||
@DisplayName("[HARD REQ 5] the freshness check uses the injected wall-clock LongSupplier, not System.nanoTime")
|
||||
void freshnessCheckUsesTheInjectedWallClockNotNanoTime() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr(); // default agentStatus is "idle" — both waits settle immediately
|
||||
Path handover = writeHandover("handover contents");
|
||||
// A fake clock whose values look nothing like System.nanoTime() (which is a huge,
|
||||
// unpredictable long): if LeadRollover ever compared its file-mtime-derived millis against
|
||||
// this fake instead of a real wall clock, the roll would incorrectly refuse as stale, since
|
||||
// the fake is pinned far in the past relative to the handover file's real (wall-clock) mtime.
|
||||
AtomicLong clock = new AtomicLong(500);
|
||||
FleetConfig.LeadRollover config = cfg(handover.toString());
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
assertEquals(500, pending.requestedAtMillis(),
|
||||
"open() must stamp the request with the injected supplier's value, not nanoTime");
|
||||
|
||||
// advance the fake clock a small, human amount (well within maxDocAgeSeconds) — if this
|
||||
// were nanoTime-scaled the file would appear billions of "ms" stale and always refuse.
|
||||
clock.set(1_500);
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertTrue(decision.accepted(), "expected approval; got refusal: " + decision.reason()
|
||||
+ " (" + decision.detail() + ")");
|
||||
assertEquals(2, promptCallCount(herdr), "a passing freshness check lets the (synchronous, "
|
||||
+ "in this test) continuation run all the way through to /clear + bootstrapText");
|
||||
}
|
||||
|
||||
// ---- Correction 1: the calling lead's own turn must end before /clear is ever sent ----
|
||||
|
||||
@Test
|
||||
@DisplayName("[CORRECTION 1 — the branch that matters most] the calling lead's turn never settling means /clear is NEVER sent, at all")
|
||||
void turnThatNeverSettlesSendsNoClearAtAll() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
herdr.agentStatus("working"); // the calling lead's own pane — never goes idle in this test
|
||||
Path handover = writeHandover("handover contents");
|
||||
FleetConfig.LeadRollover config =
|
||||
new FleetConfig.LeadRollover(handover.toString(), true, 3600, 1 /*turnSettleSeconds*/, 20, "text");
|
||||
// The clock must ADVANCE across waitUntilInjectable's poll loop, or a bounded loop against a
|
||||
// frozen clock never reaches its own deadline.
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(herdr, config, () -> clock.addAndGet(500));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
// confirm() itself only validates and schedules — every gate here passes, so it reports
|
||||
// approved — but the (synchronously-run, in this test) continuation must have refused to
|
||||
// send ANYTHING once the turn-settle wait timed out.
|
||||
assertTrue(decision.accepted(), "every synchronous gate should pass; the refusal happens "
|
||||
+ "only inside the deferred continuation, which this test's synchronous runner has "
|
||||
+ "already run to completion by the time confirm() returns");
|
||||
assertEquals(0, promptCallCount(herdr),
|
||||
"the calling lead's own pane never went idle, so /clear must NEVER be sent — sending "
|
||||
+ "it while the turn that requested the roll is still live would destroy that "
|
||||
+ "same live context");
|
||||
}
|
||||
|
||||
// ---- fleetd #480 Unit E: BLOCKED is not a settled turn boundary ------------------------
|
||||
|
||||
@Test
|
||||
@DisplayName("[UNIT E] the calling lead's turn reporting BLOCKED the whole window is NOT settled — /clear is never sent")
|
||||
void blockedTheWholeTurnSettleWindowSendsNoClearAtAll() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
herdr.agentStatus("blocked"); // the calling lead's own pane — paused on a prompt the whole window
|
||||
Path handover = writeHandover("handover contents");
|
||||
FleetConfig.LeadRollover config =
|
||||
new FleetConfig.LeadRollover(handover.toString(), true, 3600, 1 /*turnSettleSeconds*/, 20, "text");
|
||||
// The clock must ADVANCE across waitUntilAtTurnBoundary's poll loop, or a bounded loop
|
||||
// against a frozen clock never reaches its own deadline.
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(herdr, config, () -> clock.addAndGet(500));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertTrue(decision.accepted(), "every synchronous gate should pass; the refusal happens "
|
||||
+ "only inside the deferred continuation, which this test's synchronous runner has "
|
||||
+ "already run to completion by the time confirm() returns");
|
||||
assertEquals(0, promptCallCount(herdr),
|
||||
"BLOCKED means the calling lead's turn is paused on a prompt, not ended — it is NOT "
|
||||
+ "a turn boundary, so /clear must never be sent while the pane sits on an "
|
||||
+ "open prompt (that is exactly what injectable() would wrongly allow, since "
|
||||
+ "it treats BLOCKED as safe to inject into)");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[UNIT E] a pane that goes BLOCKED after /clear is NOT settled — bootstrapText is never sent")
|
||||
void blockedAfterClearNeverSendsBootstrapText() throws IOException {
|
||||
// Idle until /clear is sent, then permanently blocked (paused on a prompt) — isolates the
|
||||
// SECOND wait (clearSettleSeconds) from the first (turnSettleSeconds), which passes
|
||||
// immediately here.
|
||||
FakeHerdr fake = new FakeHerdr();
|
||||
HerdrClient blocksAfterClear = new HerdrClient() {
|
||||
@Override
|
||||
public JsonNode call(String method, Object params) throws HerdrException {
|
||||
JsonNode result = fake.call(method, params);
|
||||
if ("agent.prompt".equals(method) && String.valueOf(params).contains("/clear")) {
|
||||
fake.agentStatus("blocked");
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
fake.close();
|
||||
}
|
||||
};
|
||||
Path handover = writeHandover("handover contents");
|
||||
FleetConfig.LeadRollover config =
|
||||
new FleetConfig.LeadRollover(handover.toString(), true, 3600, 20, 1 /*clearSettleSeconds*/, "boot text");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(blocksAfterClear, config, () -> clock.addAndGet(500));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertTrue(decision.accepted(), "every synchronous gate passes; the refusal is logged only, "
|
||||
+ "deep inside the deferred continuation");
|
||||
assertEquals(1, promptCallCount(fake), "exactly one agent.prompt call — the /clear — and "
|
||||
+ "nothing else: a freshly-cleared pane sitting on a prompt must not be typed into");
|
||||
long bootstrapSends = fake.calls.stream()
|
||||
.filter(c -> "agent.prompt".equals(c.method()))
|
||||
.filter(c -> String.valueOf(c.params()).contains("boot text"))
|
||||
.count();
|
||||
assertEquals(0, bootstrapSends,
|
||||
"bootstrapText must never be sent while the freshly-cleared pane reports BLOCKED — "
|
||||
+ "typing into an open prompt after /clear is exactly as destructive as "
|
||||
+ "typing /clear into one");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[UNIT E] DONE still counts as a settled turn boundary — the fix must not over-tighten to IDLE-only")
|
||||
void doneStatusStillCompletesTheFullRoll() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
herdr.agentStatus("done"); // both waits must accept DONE as much as IDLE
|
||||
Path handover = writeHandover("handover contents");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
FleetConfig.LeadRollover config = cfg(handover.toString());
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertTrue(decision.accepted(), "expected approval; got: " + decision.reason() + " / " + decision.detail());
|
||||
var prompts = herdr.calls.stream().filter(c -> "agent.prompt".equals(c.method())).toList();
|
||||
assertEquals(2, prompts.size(),
|
||||
"a pane reporting DONE the whole time must complete the full roll — /clear then "
|
||||
+ "bootstrapText — exactly like IDLE; DONE is a real turn-boundary equivalent "
|
||||
+ "to IDLE (see AgentStatus#DONE), not merely 'injectable'");
|
||||
assertTrue(prompts.get(0).params().toString().contains("/clear"));
|
||||
assertTrue(prompts.get(1).params().toString().contains("read the handover file"));
|
||||
}
|
||||
|
||||
// ---- Correction 2: only the terminal that opened a request may confirm it -------------
|
||||
|
||||
@Test
|
||||
@DisplayName("[CORRECTION 2] a different lead terminal cannot confirm another lead's rollover")
|
||||
void aDifferentLeadTerminalCannotConfirmAnotherLeadsRollover() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
Path handover = writeHandover("handover contents");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(herdr, cfg(handover.toString()), fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(OTHER_LEAD, pending.token(), true);
|
||||
|
||||
assertFalse(decision.accepted());
|
||||
assertEquals(LeadRollover.RefusalReason.NOT_YOUR_ROLLOVER, decision.reason());
|
||||
assertEquals(0, promptCallCount(herdr), "a foreign terminal's confirm() must never touch "
|
||||
+ "the pane it named, let alone the pane that actually opened the request");
|
||||
|
||||
// the rightful owner can still confirm the same token afterwards — a foreign confirm()
|
||||
// must not consume or otherwise disturb the pending request.
|
||||
LeadRollover.RollDecision ownerDecision = rollover.confirm(LEAD, pending.token(), true);
|
||||
assertTrue(ownerDecision.accepted(), "the actual opener must still be able to confirm after "
|
||||
+ "a foreign terminal's confirm() was refused");
|
||||
}
|
||||
|
||||
// ---- extra coverage: cancel(), CLEAR_DID_NOT_SETTLE, and the full success path ---------
|
||||
|
||||
@Test
|
||||
@DisplayName("cancel() drops a pending request so a later confirm() reports UNKNOWN_TOKEN")
|
||||
void cancelDropsThePendingRequest() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
Path handover = writeHandover("handover contents");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(herdr, cfg(handover.toString()), fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
assertTrue(rollover.cancel(pending.token()));
|
||||
assertFalse(rollover.cancel(pending.token()), "a second cancel() of the same token finds nothing");
|
||||
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
assertFalse(decision.accepted());
|
||||
assertEquals(LeadRollover.RefusalReason.UNKNOWN_TOKEN, decision.reason());
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("open() throws IllegalStateException when leadRollover: is not configured")
|
||||
void openThrowsWhenNotConfigured() {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
AgentControl agents = new AgentControl(herdr);
|
||||
LeadRollover rollover = new LeadRollover(agents, () -> null, _ -> null, () -> 1_000L, () -> { }, Runnable::run);
|
||||
|
||||
assertThrows(IllegalStateException.class, () -> rollover.open(LEAD, "context is full"));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("open() throws IllegalArgumentException when leadTerminal is null or blank")
|
||||
void openThrowsWhenLeadTerminalIsBlank() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
Path handover = writeHandover("handover contents");
|
||||
LeadRollover rollover = newRollover(herdr, cfg(handover.toString()), () -> 1_000L);
|
||||
|
||||
assertThrows(IllegalArgumentException.class, () -> rollover.open(null, "context is full"));
|
||||
assertThrows(IllegalArgumentException.class, () -> rollover.open(" ", "context is full"));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("a pane that never re-settles after /clear refuses to send bootstrapText")
|
||||
void clearThatNeverSettlesAfterwardsNeverSendsBootstrapText() throws IOException {
|
||||
// Idle until /clear is sent, then permanently working — isolates the SECOND wait
|
||||
// (clearSettleSeconds) from the first (turnSettleSeconds), which passes immediately here.
|
||||
FakeHerdr fake = new FakeHerdr();
|
||||
HerdrClient flipsAfterClear = new HerdrClient() {
|
||||
@Override
|
||||
public JsonNode call(String method, Object params) throws HerdrException {
|
||||
JsonNode result = fake.call(method, params);
|
||||
if ("agent.prompt".equals(method) && String.valueOf(params).contains("/clear")) {
|
||||
fake.agentStatus("working");
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
fake.close();
|
||||
}
|
||||
};
|
||||
Path handover = writeHandover("handover contents");
|
||||
FleetConfig.LeadRollover config =
|
||||
new FleetConfig.LeadRollover(handover.toString(), true, 3600, 20, 1 /*clearSettleSeconds*/, "boot text");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(flipsAfterClear, config, () -> clock.addAndGet(500));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertTrue(decision.accepted(), "every synchronous gate passes; the refusal is logged only, "
|
||||
+ "deep inside the deferred continuation");
|
||||
assertEquals(1, promptCallCount(fake), "exactly one agent.prompt call — the /clear — and "
|
||||
+ "nothing else");
|
||||
long bootstrapSends = fake.calls.stream()
|
||||
.filter(c -> "agent.prompt".equals(c.method()))
|
||||
.filter(c -> String.valueOf(c.params()).contains("boot text"))
|
||||
.count();
|
||||
assertEquals(0, bootstrapSends, "bootstrapText must never be sent when /clear did not settle");
|
||||
}
|
||||
|
||||
// ---- fleetd #489: the second wait nudges the /clear pickup instead of being a no-op --------
|
||||
|
||||
/** Every {@code agent.send_keys} call {@code herdr} recorded — the submit-keystroke nudge. */
|
||||
private static long sendKeysCallCount(FakeHerdr herdr) {
|
||||
return herdr.calls.stream().filter(c -> "agent.send_keys".equals(c.method())).count();
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #489] a pane that stays IDLE the whole time (the paste-race case, "
|
||||
+ "measured live 2026-09-12) is nudged between /clear and bootstrapText, never lets "
|
||||
+ "them concatenate into one line")
|
||||
void clearPickupIsNudgedBeforeBootstrapTextWhenPaneStaysIdle() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr(); // default agentStatus is "idle" throughout — no WORKING sample ever
|
||||
Path handover = writeHandover("handover contents");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(herdr, cfg(handover.toString()), fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertTrue(decision.accepted(), "expected approval; got: " + decision.reason() + " / " + decision.detail());
|
||||
|
||||
int clearIdx = -1;
|
||||
int bootstrapIdx = -1;
|
||||
int firstNudgeIdx = -1;
|
||||
for (int i = 0; i < herdr.calls.size(); i++) {
|
||||
FakeHerdr.Call c = herdr.calls.get(i);
|
||||
if ("agent.prompt".equals(c.method()) && String.valueOf(c.params()).contains("/clear") && clearIdx < 0) {
|
||||
clearIdx = i;
|
||||
} else if ("agent.prompt".equals(c.method()) && String.valueOf(c.params()).contains("read the handover file")) {
|
||||
bootstrapIdx = i;
|
||||
} else if ("agent.send_keys".equals(c.method()) && firstNudgeIdx < 0) {
|
||||
firstNudgeIdx = i;
|
||||
}
|
||||
}
|
||||
assertTrue(clearIdx >= 0, "/clear must have been sent");
|
||||
assertTrue(bootstrapIdx >= 0, "bootstrapText must have been sent");
|
||||
assertTrue(firstNudgeIdx >= 0, "at least one agent.send_keys nudge must go out — the pane "
|
||||
+ "never reported WORKING, so the /clear Enter may have raced the paste, and only a "
|
||||
+ "re-sent Enter proves the clear rather than concatenating bootstrapText onto "
|
||||
+ "whatever sits unsubmitted in the input box");
|
||||
assertTrue(firstNudgeIdx > clearIdx, "the nudge must happen AFTER /clear was sent, got call "
|
||||
+ "order: " + herdr.calls);
|
||||
assertTrue(firstNudgeIdx < bootstrapIdx, "the nudge must happen BEFORE bootstrapText is "
|
||||
+ "sent — never concatenated onto the same input line, got call order: " + herdr.calls);
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #489] once a WORKING sample confirms /clear was picked up, nudging stops "
|
||||
+ "and bootstrapText is still sent after the pane returns to IDLE")
|
||||
void pickupSeenStopsNudgingAndBootstrapTextIsSent() throws IOException {
|
||||
FakeHerdr fake = new FakeHerdr();
|
||||
// Scripts the SECOND wait only: idle (default) until /clear is sent, then the first status
|
||||
// poll after /clear reports WORKING (a confirmed pickup), and every poll after that reports
|
||||
// IDLE (the completion boundary). The first wait (turnSettleSeconds) never sees this
|
||||
// sequence — it passes on its own first poll, before /clear is ever sent, on the default
|
||||
// "idle" status.
|
||||
AtomicLong postClearGetCalls = new AtomicLong(0);
|
||||
HerdrClient scriptsPickupThenIdle = new HerdrClient() {
|
||||
private volatile boolean clearSent = false;
|
||||
|
||||
@Override
|
||||
public JsonNode call(String method, Object params) throws HerdrException {
|
||||
if ("agent.prompt".equals(method) && String.valueOf(params).contains("/clear")) {
|
||||
clearSent = true;
|
||||
}
|
||||
if (clearSent && "agent.get".equals(method)) {
|
||||
long n = postClearGetCalls.incrementAndGet();
|
||||
fake.agentStatus(n == 1 ? "working" : "idle");
|
||||
}
|
||||
return fake.call(method, params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
fake.close();
|
||||
}
|
||||
};
|
||||
Path handover = writeHandover("handover contents");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(scriptsPickupThenIdle, cfg(handover.toString()), fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertTrue(decision.accepted(), "expected approval; got: " + decision.reason() + " / " + decision.detail());
|
||||
assertEquals(2, promptCallCount(fake), "a confirmed WORKING pickup followed by IDLE must "
|
||||
+ "still complete the full roll — /clear then bootstrapText");
|
||||
assertEquals(0, sendKeysCallCount(fake), "once WORKING was observed, nudging must stop "
|
||||
+ "immediately — no agent.send_keys call should ever have been needed or sent");
|
||||
assertTrue(postClearGetCalls.get() >= 2, "the pane's status must have been polled AGAIN "
|
||||
+ "after the WORKING sample, before bootstrapText was sent — this is what proves "
|
||||
+ "the method actually waited for the WORKING -> IDLE completion boundary instead "
|
||||
+ "of returning as soon as pickup was seen (or worse, without polling at all, as a "
|
||||
+ "stub that just returns true would); got " + postClearGetCalls.get()
|
||||
+ " agent.get call(s) after /clear");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #489] a pane that never reaches a turn boundary after /clear (stuck at "
|
||||
+ "UNKNOWN, never WORKING either) lets clearSettleSeconds expire — bootstrapText is "
|
||||
+ "never sent")
|
||||
void clearPickupNeverSettlesWhenStatusNeverReachesABoundary() throws IOException {
|
||||
FakeHerdr fake = new FakeHerdr();
|
||||
HerdrClient stuckUnknownAfterClear = new HerdrClient() {
|
||||
@Override
|
||||
public JsonNode call(String method, Object params) throws HerdrException {
|
||||
if ("agent.prompt".equals(method) && String.valueOf(params).contains("/clear")) {
|
||||
// "wedged" maps to AgentStatus.UNKNOWN (see AgentStatus#fromWire) — neither a
|
||||
// pickup signal (WORKING) nor a boundary (IDLE/DONE), and distinct from the
|
||||
// already-covered BLOCKED case below.
|
||||
fake.agentStatus("wedged");
|
||||
}
|
||||
return fake.call(method, params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
fake.close();
|
||||
}
|
||||
};
|
||||
Path handover = writeHandover("handover contents");
|
||||
FleetConfig.LeadRollover config =
|
||||
new FleetConfig.LeadRollover(handover.toString(), true, 3600, 20, 1 /*clearSettleSeconds*/, "boot text");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(stuckUnknownAfterClear, config, () -> clock.addAndGet(500));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertTrue(decision.accepted(), "every synchronous gate passes; the refusal is logged only, "
|
||||
+ "deep inside the deferred continuation");
|
||||
assertEquals(1, promptCallCount(fake), "exactly one agent.prompt call — the /clear — and "
|
||||
+ "nothing else");
|
||||
long bootstrapSends = fake.calls.stream()
|
||||
.filter(c -> "agent.prompt".equals(c.method()))
|
||||
.filter(c -> String.valueOf(c.params()).contains("boot text"))
|
||||
.count();
|
||||
assertEquals(0, bootstrapSends, "bootstrapText must never be sent when the pane never "
|
||||
+ "reaches a turn boundary after /clear, whether WORKING was ever observed or not");
|
||||
assertEquals(0, sendKeysCallCount(fake), "an UNKNOWN status is neither a pickup signal nor "
|
||||
+ "a boundary — it must never be nudged");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #489] a submit() nudge that throws does not abort the roll — /clear and "
|
||||
+ "bootstrapText are both still sent")
|
||||
void submitThatThrowsDoesNotAbortTheRoll() throws IOException {
|
||||
FakeHerdr fake = new FakeHerdr(); // default idle throughout — nudging will be attempted
|
||||
HerdrClient throwsOnSubmit = new HerdrClient() {
|
||||
@Override
|
||||
public JsonNode call(String method, Object params) throws HerdrException {
|
||||
if ("agent.send_keys".equals(method)) {
|
||||
throw new RuntimeException("simulated herdr transport failure on submit");
|
||||
}
|
||||
return fake.call(method, params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
fake.close();
|
||||
}
|
||||
};
|
||||
Path handover = writeHandover("handover contents");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
LeadRollover rollover = newRollover(throwsOnSubmit, cfg(handover.toString()), fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertTrue(decision.accepted(), "expected approval; got: " + decision.reason() + " / " + decision.detail());
|
||||
var prompts = fake.calls.stream().filter(c -> "agent.prompt".equals(c.method())).toList();
|
||||
assertEquals(2, prompts.size(), "a throwing submit() must be swallowed, not abort the roll "
|
||||
+ "— /clear and bootstrapText must both still be sent");
|
||||
assertTrue(prompts.get(0).params().toString().contains("/clear"));
|
||||
assertTrue(prompts.get(1).params().toString().contains("read the handover file"));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("a full successful roll sends /clear then bootstrapText, in order, and consumes the token")
|
||||
void successfulRollSendsClearThenBootstrapTextAndConsumesTheToken() throws IOException {
|
||||
FakeHerdr herdr = new FakeHerdr(); // default agentStatus is "idle" — injectable immediately
|
||||
Path handover = writeHandover("handover contents");
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
FleetConfig.LeadRollover config = cfg(handover.toString());
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock));
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertTrue(decision.accepted(), "expected approval; got: " + decision.reason() + " / " + decision.detail());
|
||||
var prompts = herdr.calls.stream().filter(c -> "agent.prompt".equals(c.method())).toList();
|
||||
assertEquals(2, prompts.size(), "expected exactly two agent.prompt calls: /clear then bootstrapText");
|
||||
assertTrue(prompts.get(0).params().toString().contains("/clear"));
|
||||
assertTrue(prompts.get(1).params().toString().contains("read the handover file"));
|
||||
|
||||
// token is consumed once confirm() approves — a second confirm() with the same token is
|
||||
// UNKNOWN_TOKEN, even though the deferred roll's own outcome is decided later.
|
||||
LeadRollover.RollDecision again = rollover.confirm(LEAD, pending.token(), true);
|
||||
assertEquals(LeadRollover.RefusalReason.UNKNOWN_TOKEN, again.reason());
|
||||
}
|
||||
|
||||
// ---- fleetd #480 follow-up: a relative handoverPath resolves against the CALLING lead's own
|
||||
// workspace, never the daemon's cwd ------------------------------------------------------
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #480 follow-up] a relative handoverPath resolves against the lead's "
|
||||
+ "configured workspace, and PendingRollover carries the absolute path")
|
||||
void relativeHandoverPathResolvesAgainstLeadWorkspace() {
|
||||
Path workspace = tmp.resolve("lead-workspace");
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
FleetConfig.LeadRollover config = cfg("handover.md"); // relative — no directory component
|
||||
Function<String, String> leadWorkspace = terminal -> LEAD.equals(terminal) ? workspace.toString() : null;
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock), leadWorkspace);
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
|
||||
String expected = workspace.resolve("handover.md").normalize().toString();
|
||||
assertEquals(expected, pending.handoverPath(),
|
||||
"a relative handoverPath must resolve against the CALLING lead's own workspace, "
|
||||
+ "never the daemon's own working directory");
|
||||
assertTrue(Path.of(pending.handoverPath()).isAbsolute(),
|
||||
"the resolved path stored on PendingRollover must always be absolute");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #480 follow-up] confirm() accepts a handover file written at the "
|
||||
+ "resolved absolute location for a relative handoverPath")
|
||||
void confirmAcceptsAFileWrittenAtTheResolvedLocation() throws IOException {
|
||||
Path workspace = tmp.resolve("lead-workspace");
|
||||
Files.createDirectories(workspace);
|
||||
Path expected = workspace.resolve("handover.md");
|
||||
Files.writeString(expected, "handover contents");
|
||||
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
FleetConfig.LeadRollover config = cfg("handover.md");
|
||||
Function<String, String> leadWorkspace = terminal -> LEAD.equals(terminal) ? workspace.toString() : null;
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock), leadWorkspace);
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
assertEquals(expected.toString(), pending.handoverPath());
|
||||
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
assertTrue(decision.accepted(), "expected approval; got: " + decision.reason() + " / " + decision.detail());
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #480 follow-up] a file at the same relative name under a DIFFERENT "
|
||||
+ "directory is NOT accepted — resolution is against the lead's own workspace only")
|
||||
void relativeHandoverPathDoesNotMatchAFileUnderADifferentDirectory() throws IOException {
|
||||
Path workspace = tmp.resolve("lead-workspace");
|
||||
Path decoy = tmp.resolve("decoy-directory");
|
||||
Files.createDirectories(workspace);
|
||||
Files.createDirectories(decoy);
|
||||
// the decoy directory holds a file with the SAME relative name — if resolution ever fell
|
||||
// back to searching, or resolved against the wrong base, this would be wrongly found.
|
||||
Files.writeString(decoy.resolve("handover.md"), "decoy contents — must never be read");
|
||||
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
FleetConfig.LeadRollover config = cfg("handover.md");
|
||||
Function<String, String> leadWorkspace = terminal -> LEAD.equals(terminal) ? workspace.toString() : null;
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock), leadWorkspace);
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
assertEquals(workspace.resolve("handover.md").toString(), pending.handoverPath());
|
||||
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
assertFalse(decision.accepted());
|
||||
assertEquals(LeadRollover.RefusalReason.HANDOVER_MISSING, decision.reason(),
|
||||
"the file at workspace/handover.md does not exist — a decoy file at the same "
|
||||
+ "relative name under a different directory must never be mistaken for it");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #480 follow-up] an absolute handoverPath is used unchanged — the lead "
|
||||
+ "workspace lookup is never even consulted")
|
||||
void absoluteHandoverPathIsUnchangedByResolution() throws IOException {
|
||||
Path absolute = writeHandover("handover contents"); // tmp.resolve(...) — always absolute
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
FleetConfig.LeadRollover config = cfg(absolute.toString());
|
||||
// a workspace lookup that would resolve a RELATIVE path somewhere completely different —
|
||||
// proving it is never consulted at all for an already-absolute handoverPath.
|
||||
Function<String, String> leadWorkspace = terminal -> tmp.resolve("some-other-workspace").toString();
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock), leadWorkspace);
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
|
||||
assertEquals(absolute.normalize().toString(), pending.handoverPath(),
|
||||
"an absolute handoverPath must be used unchanged (aside from normalization)");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #480 follow-up correction] a RELATIVE fleet.leaders.<name>.cwd still "
|
||||
+ "yields an ABSOLUTE PendingRollover.handoverPath")
|
||||
void relativeLeadWorkspaceCwdStillYieldsAnAbsoluteHandoverPath() {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
FleetConfig.LeadRollover config = cfg("handover.md"); // relative handoverPath
|
||||
// Nothing in FleetConfig validates fleet.leaders.<name>.cwd, so an operator can write a
|
||||
// RELATIVE one — this must still resolve to an absolute PendingRollover.handoverPath,
|
||||
// never silently reopen the exact bug this class fixes.
|
||||
String relativeWorkspace = "relative-lead-workspace";
|
||||
Function<String, String> leadWorkspace = terminal -> LEAD.equals(terminal) ? relativeWorkspace : null;
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock), leadWorkspace);
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
|
||||
assertTrue(Path.of(pending.handoverPath()).isAbsolute(),
|
||||
"the resolved path must be absolute even when the configured cwd itself is relative");
|
||||
|
||||
// Asserting isAbsolute() alone would also pass for a path resolved against the WRONG base
|
||||
// (e.g. some unrelated absolute directory) — pin the actual value too.
|
||||
String expected = Path.of(System.getProperty("user.dir")).resolve(relativeWorkspace)
|
||||
.resolve("handover.md").toAbsolutePath().normalize().toString();
|
||||
assertEquals(expected, pending.handoverPath(),
|
||||
"a relative cwd must be finished off against the daemon's own user.dir, exactly "
|
||||
+ "like a missing cwd — never left relative, which would silently reopen the "
|
||||
+ "exact bug this class fixes: every later reader interpreting an ambiguous "
|
||||
+ "path against ITS OWN working directory");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #480 follow-up] a terminal with no configured workspace (null lookup "
|
||||
+ "result) falls back to the daemon's own user.dir")
|
||||
void noConfiguredWorkspaceFallsBackToUserDir() {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
FleetConfig.LeadRollover config = cfg("some-handover.md");
|
||||
Function<String, String> leadWorkspace = _ -> null; // no configured workspace for anyone
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock), leadWorkspace);
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
|
||||
String expected = Path.of(System.getProperty("user.dir")).resolve("some-handover.md")
|
||||
.normalize().toString();
|
||||
assertEquals(expected, pending.handoverPath(),
|
||||
"with no configured workspace for this lead, resolution must fall back to the "
|
||||
+ "daemon's own user.dir — the same fallback LeadLauncher#launch already uses "
|
||||
+ "for a lead with no configured cwd");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #480 follow-up] a terminal whose workspace lookup returns a blank string "
|
||||
+ "also falls back to the daemon's own user.dir")
|
||||
void blankConfiguredWorkspaceFallsBackToUserDir() {
|
||||
FakeHerdr herdr = new FakeHerdr();
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
FleetConfig.LeadRollover config = cfg("some-handover.md");
|
||||
Function<String, String> leadWorkspace = _ -> " ";
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock), leadWorkspace);
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
|
||||
String expected = Path.of(System.getProperty("user.dir")).resolve("some-handover.md")
|
||||
.normalize().toString();
|
||||
assertEquals(expected, pending.handoverPath(),
|
||||
"a blank (non-null) workspace lookup result must be treated the same as null — "
|
||||
+ "fall back to user.dir, never resolve against an empty base");
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("[fleetd #480 follow-up] the default bootstrapText that is actually SENT names "
|
||||
+ "the RESOLVED absolute handoverPath, not the raw relative configured value")
|
||||
void defaultBootstrapTextNamesTheResolvedAbsolutePath() throws IOException {
|
||||
Path workspace = tmp.resolve("lead-workspace");
|
||||
Files.createDirectories(workspace);
|
||||
Path expected = workspace.resolve("handover.md");
|
||||
Files.writeString(expected, "handover contents");
|
||||
|
||||
FakeHerdr herdr = new FakeHerdr(); // default agentStatus "idle" — both waits settle immediately
|
||||
AtomicLong clock = new AtomicLong(1_000);
|
||||
// bootstrapText left null so the DEFAULT sentence is built — from the RESOLVED path
|
||||
FleetConfig.LeadRollover config = new FleetConfig.LeadRollover("handover.md", true, 3600, 20, 20, null);
|
||||
Function<String, String> leadWorkspace = terminal -> LEAD.equals(terminal) ? workspace.toString() : null;
|
||||
LeadRollover rollover = newRollover(herdr, config, fixedClock(clock), leadWorkspace);
|
||||
|
||||
LeadRollover.PendingRollover pending = rollover.open(LEAD, "context is full");
|
||||
LeadRollover.RollDecision decision = rollover.confirm(LEAD, pending.token(), true);
|
||||
|
||||
assertTrue(decision.accepted(), "expected approval; got: " + decision.reason() + " / " + decision.detail());
|
||||
var prompts = herdr.calls.stream().filter(c -> "agent.prompt".equals(c.method())).toList();
|
||||
assertEquals(2, prompts.size(), "expected exactly two agent.prompt calls: /clear then the "
|
||||
+ "default bootstrapText");
|
||||
String bootstrapSent = prompts.get(1).params().toString();
|
||||
assertTrue(bootstrapSent.contains(expected.toString()),
|
||||
"the default bootstrapText actually sent to the pane must name the RESOLVED "
|
||||
+ "absolute path (" + expected + ") — a fresh session's own pane cannot "
|
||||
+ "resolve a relative path against a directory it never had. Actual text "
|
||||
+ "sent: " + bootstrapSent);
|
||||
assertFalse(bootstrapSent.contains("\"handover.md\""),
|
||||
"must not name the raw relative configured value in the text actually sent");
|
||||
}
|
||||
}
|
||||
@@ -26,6 +26,7 @@ import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.regex.Matcher;
|
||||
@@ -74,12 +75,16 @@ class FleetMcpAuthzTest {
|
||||
ConnectionIdentity identity = new ConnectionIdentity(new PaneLocator(herdr), _ -> 999_999);
|
||||
metrics = FleetMetrics.create(sessions, new InMemoryReplyInbox());
|
||||
|
||||
// fleetd #480 correction round: FleetMcp has one constructor now (no defaulting
|
||||
// overloads — see its javadoc), so every feature this test does not exercise is passed
|
||||
// its explicit "off" value here rather than being omitted.
|
||||
mcp = new FleetMcp(messages, workers, sessions, identity, sessions.asPresence(),
|
||||
new PrimaryRegistry(null),
|
||||
enforce ? CallerResolver.withLeadsAndMembers(identity, false, null,
|
||||
Map::of, new MemberRegistry(null)) : null,
|
||||
metrics, FleetMcp.CapacitySource.none(), new FleetMcp.HealthCoverageSource(() -> "off"),
|
||||
FleetMcp.QuarantineSource.none());
|
||||
FleetMcp.QuarantineSource.none(), null, FleetMcp.OutageSource.none(),
|
||||
FleetMcp.LeadSeatSource.none(), List.of(), null);
|
||||
return mcp;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,261 @@
|
||||
package dev.ltms.fleet.mcp;
|
||||
|
||||
import dev.ltms.fleet.auth.Authz;
|
||||
import dev.ltms.fleet.auth.CallerResolver;
|
||||
import dev.ltms.fleet.auth.MemberRegistry;
|
||||
import dev.ltms.fleet.auth.Principal;
|
||||
import dev.ltms.fleet.config.FleetConfig;
|
||||
import dev.ltms.fleet.guard.SubscriptionGuard;
|
||||
import dev.ltms.fleet.herdr.AgentControl;
|
||||
import dev.ltms.fleet.herdr.FakeHerdr;
|
||||
import dev.ltms.fleet.herdr.PaneLocator;
|
||||
import dev.ltms.fleet.herdr.WorkspaceControl;
|
||||
import dev.ltms.fleet.inject.Injector;
|
||||
import dev.ltms.fleet.lead.LeadRollover;
|
||||
import dev.ltms.fleet.member.ClaudeCodeLauncher;
|
||||
import dev.ltms.fleet.msg.InMemoryReplyInbox;
|
||||
import dev.ltms.fleet.msg.MessageService;
|
||||
import dev.ltms.fleet.msg.Rendezvous;
|
||||
import dev.ltms.fleet.session.FakeWorktrees;
|
||||
import dev.ltms.fleet.session.SessionManager;
|
||||
import io.modelcontextprotocol.spec.McpSchema;
|
||||
import org.junit.jupiter.api.AfterEach;
|
||||
import org.junit.jupiter.api.DisplayName;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
/**
|
||||
* fleetd #480 Unit C — the {@code fleet_handover} MCP tool, the surface that finally calls
|
||||
* {@link LeadRollover#open}/{@link LeadRollover#confirm}/{@link LeadRollover#cancel}.
|
||||
*
|
||||
* <p>Uses {@link LeadRollover}'s PUBLIC constructor (real wall clock, real 250ms settle poll, a
|
||||
* real virtual-thread continuation runner) rather than its package-private test constructor —
|
||||
* this test lives in {@code dev.ltms.fleet.mcp}, not {@code dev.ltms.fleet.lead}, and does not
|
||||
* need to control the post-{@code confirm()} continuation's timing: it only asserts the
|
||||
* SYNCHRONOUS return value of {@code open}/{@code confirm}/{@code cancel}, which is exactly what
|
||||
* {@code FleetMcp.handover} forwards to the client. {@code turnSettleSeconds}/{@code
|
||||
* clearSettleSeconds} are kept at 1s so a confirmed request's background continuation (which this
|
||||
* class does not wait on or assert against) gives up quickly rather than polling for 20s on a
|
||||
* daemon virtual thread.
|
||||
*/
|
||||
class FleetMcpHandoverTest {
|
||||
|
||||
@TempDir
|
||||
Path tmp;
|
||||
|
||||
private static final String LEAD = "term_lead";
|
||||
private static final String OTHER_LEAD = "term_other_lead";
|
||||
|
||||
private final FakeHerdr herdr = new FakeHerdr();
|
||||
private final AgentControl agents = new AgentControl(herdr);
|
||||
private FleetMcp mcp;
|
||||
|
||||
@AfterEach
|
||||
void close() {
|
||||
if (mcp != null) mcp.close();
|
||||
}
|
||||
|
||||
private static FleetConfig.LeadRollover cfg(String handoverPath) {
|
||||
return new FleetConfig.LeadRollover(handoverPath, false, 3600, 1, 1, "read the handover file");
|
||||
}
|
||||
|
||||
private LeadRollover newRollover(String handoverPath) {
|
||||
// Every handoverPath this test class uses comes from tmp.resolve(...), which is already
|
||||
// absolute, so the workspace lookup is never actually consulted — a no-op lookup is enough.
|
||||
return new LeadRollover(agents, () -> cfg(handoverPath), _ -> null);
|
||||
}
|
||||
|
||||
/** A fully wired FleetMcp on fakes (mirrors FleetMcpAuthzTest's helper), plus a leadRollover. */
|
||||
private FleetMcp mcp(LeadRollover leadRollover) {
|
||||
FleetConfig.Profile pcfg = new FleetConfig.Profile(
|
||||
"ltms-local", "http://gx00.gw:8000", "coder", null, "FLEETD_WORKER_TOKEN", null,
|
||||
"tab", "fleetd-workers", "worker: {profile} #{n}", null, null, null);
|
||||
ClaudeCodeLauncher workers = new ClaudeCodeLauncher(agents, new WorkspaceControl(herdr),
|
||||
new SubscriptionGuard(Set.of("gx00.gw")), Map.of(pcfg.profile(), pcfg), pcfg.profile(),
|
||||
_ -> "tok");
|
||||
SessionManager sessions = new SessionManager(workers, new FakeWorktrees());
|
||||
MessageService messages = new MessageService(agents, new Injector(agents), new Rendezvous(),
|
||||
new InMemoryReplyInbox());
|
||||
ConnectionIdentity identity = new ConnectionIdentity(new PaneLocator(herdr), _ -> 999_999);
|
||||
|
||||
mcp = new FleetMcp(messages, workers, sessions, identity, sessions.asPresence(),
|
||||
new PrimaryRegistry(null),
|
||||
CallerResolver.withLeadsAndMembers(identity, false, null, Map::of, new MemberRegistry(null)),
|
||||
null, FleetMcp.CapacitySource.none(), new FleetMcp.HealthCoverageSource(() -> "off"),
|
||||
FleetMcp.QuarantineSource.none(), null, FleetMcp.OutageSource.none(),
|
||||
FleetMcp.LeadSeatSource.none(), List.of(), leadRollover);
|
||||
return mcp;
|
||||
}
|
||||
|
||||
private static String textOf(McpSchema.CallToolResult r) {
|
||||
return ((McpSchema.TextContent) r.content().getFirst()).text();
|
||||
}
|
||||
|
||||
private static String extractToken(String json) {
|
||||
int i = json.indexOf("\"token\":\"");
|
||||
assertTrue(i >= 0, "no token field in: " + json);
|
||||
int start = i + "\"token\":\"".length();
|
||||
int end = json.indexOf('"', start);
|
||||
return json.substring(start, end);
|
||||
}
|
||||
|
||||
// --- acceptance 1: registered whether or not leadRollover: is configured -------------------
|
||||
|
||||
@Test
|
||||
@DisplayName("fleet_handover is registered whether or not leadRollover: is configured")
|
||||
void registeredEitherWay() {
|
||||
assertTrue(FleetTool.wireNames().contains("fleet_handover"),
|
||||
"FleetTool must list fleet_handover as part of the canonical tool surface");
|
||||
|
||||
FleetMcp withNull = mcp(null);
|
||||
assertTrue(withNull.registeredTools().stream().anyMatch(t -> "fleet_handover".equals(t.name())),
|
||||
"fleet_handover must be registered even with no LeadRollover constructed");
|
||||
withNull.close();
|
||||
|
||||
FleetMcp withConfigured = mcp(newRollover(tmp.resolve("h.md").toString()));
|
||||
assertTrue(withConfigured.registeredTools().stream().anyMatch(t -> "fleet_handover".equals(t.name())),
|
||||
"fleet_handover must be registered when a LeadRollover IS constructed too");
|
||||
}
|
||||
|
||||
// --- acceptance 2: null LeadRollover -> clean NOT_CONFIGURED, no throw, every action -------
|
||||
|
||||
@Test
|
||||
@DisplayName("with leadRollover: absent, every action returns a clean NOT_CONFIGURED refusal and never throws")
|
||||
void nullLeadRolloverRefusesCleanlyForEveryAction() {
|
||||
McpSchema.CallToolResult open = assertDoesNotThrow(
|
||||
() -> FleetMcp.handover(null, LEAD, Map.of("action", "open")));
|
||||
assertFalse(open.isError(), "a refusal is not a protocol error: " + textOf(open));
|
||||
assertTrue(textOf(open).contains("NOT_CONFIGURED"), textOf(open));
|
||||
|
||||
McpSchema.CallToolResult confirm = assertDoesNotThrow(() -> FleetMcp.handover(null, LEAD,
|
||||
Map.of("action", "confirm", "token", "whatever")));
|
||||
assertFalse(confirm.isError());
|
||||
assertTrue(textOf(confirm).contains("NOT_CONFIGURED"), textOf(confirm));
|
||||
|
||||
McpSchema.CallToolResult cancel = assertDoesNotThrow(() -> FleetMcp.handover(null, LEAD,
|
||||
Map.of("action", "cancel", "token", "whatever")));
|
||||
assertFalse(cancel.isError());
|
||||
assertTrue(textOf(cancel).contains("NOT_CONFIGURED"), textOf(cancel));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("a blank/unknown action is a clean tool error, never an exception")
|
||||
void unknownActionIsACleanError() {
|
||||
McpSchema.CallToolResult missing = assertDoesNotThrow(() -> FleetMcp.handover(null, LEAD, Map.of()));
|
||||
assertTrue(missing.isError());
|
||||
|
||||
McpSchema.CallToolResult bogus = assertDoesNotThrow(
|
||||
() -> FleetMcp.handover(null, LEAD, Map.of("action", "bogus")));
|
||||
assertTrue(bogus.isError());
|
||||
}
|
||||
|
||||
// --- acceptance 3: non-primary refused by the authorization gate ---------------------------
|
||||
|
||||
@Test
|
||||
@DisplayName("fleet_handover maps to Authz.Action.HANDOVER, primary-only")
|
||||
void onlyThePrimaryIsPermitted() {
|
||||
assertEquals(Authz.Action.HANDOVER, FleetMcp.toolAction("fleet_handover", Map.of()));
|
||||
|
||||
FleetMcp m = mcp(null);
|
||||
assertNull(m.denyFor(Principal.primary(1), Authz.Action.HANDOVER, null),
|
||||
"the primary may drive fleet_handover");
|
||||
|
||||
McpSchema.CallToolResult deniedWorker =
|
||||
m.denyFor(Principal.worker("term_w", 2), Authz.Action.HANDOVER, null);
|
||||
assertNotNull(deniedWorker, "a worker must be refused fleet_handover");
|
||||
assertTrue(deniedWorker.isError());
|
||||
|
||||
McpSchema.CallToolResult deniedArchitect =
|
||||
m.denyFor(Principal.architect("slot", "term_a", 3), Authz.Action.HANDOVER, null);
|
||||
assertNotNull(deniedArchitect, "an architect has no lifecycle rights either — same gate as SPAWN/STOP/DRAIN");
|
||||
assertTrue(deniedArchitect.isError());
|
||||
}
|
||||
|
||||
// --- acceptance 4: the registered schema names no caller-terminal parameter ----------------
|
||||
|
||||
@Test
|
||||
@DisplayName("the registered fleet_handover schema names no terminal/session/leadTerminal parameter")
|
||||
void schemaCarriesNoCallerIdentityParameter() {
|
||||
FleetMcp m = mcp(null);
|
||||
McpSchema.Tool tool = m.registeredTools().stream()
|
||||
.filter(t -> "fleet_handover".equals(t.name()))
|
||||
.findFirst()
|
||||
.orElseThrow(() -> new AssertionError("fleet_handover was not registered"));
|
||||
|
||||
@SuppressWarnings("unchecked")
|
||||
Map<String, Object> properties = (Map<String, Object>) tool.inputSchema().get("properties");
|
||||
assertNotNull(properties, "tool has no 'properties' in its input schema");
|
||||
for (String forbidden : List.of("terminal", "sessionId", "leadTerminal", "callerTerminal", "target")) {
|
||||
assertFalse(properties.containsKey(forbidden),
|
||||
"fleet_handover's REGISTERED schema must not carry a caller-identity parameter, "
|
||||
+ "found '" + forbidden + "' in " + properties.keySet());
|
||||
}
|
||||
}
|
||||
|
||||
// --- acceptance 5: open then confirm on the same connection; ownership is enforced --------
|
||||
|
||||
@Test
|
||||
@DisplayName("open then confirm on the same terminal succeeds; a different terminal gets NOT_YOUR_ROLLOVER")
|
||||
void openThenConfirmRoundTripsAndOwnershipIsEnforced() throws Exception {
|
||||
Path handover = tmp.resolve("handover.md");
|
||||
Files.writeString(handover, "not written yet");
|
||||
LeadRollover rollover = newRollover(handover.toString());
|
||||
|
||||
McpSchema.CallToolResult openResult = FleetMcp.handover(rollover, LEAD, Map.of("action", "open"));
|
||||
assertFalse(openResult.isError(), textOf(openResult));
|
||||
String token = extractToken(textOf(openResult));
|
||||
|
||||
// Rewrite the handover file so its mtime is measurably after open()'s requestedAtMillis —
|
||||
// LeadRollover#confirm's freshness check (HANDOVER_STALE) requires this.
|
||||
Thread.sleep(50);
|
||||
Files.writeString(handover, "the real handover content");
|
||||
|
||||
McpSchema.CallToolResult wrongCaller = FleetMcp.handover(rollover, OTHER_LEAD,
|
||||
Map.of("action", "confirm", "token", token));
|
||||
assertFalse(wrongCaller.isError(), "a refusal is a legitimate outcome, not a protocol error");
|
||||
assertTrue(textOf(wrongCaller).contains("NOT_YOUR_ROLLOVER"),
|
||||
"a different lead terminal confirming must surface NOT_YOUR_ROLLOVER: " + textOf(wrongCaller));
|
||||
|
||||
McpSchema.CallToolResult confirmed = FleetMcp.handover(rollover, LEAD,
|
||||
Map.of("action", "confirm", "token", token));
|
||||
assertFalse(confirmed.isError(), textOf(confirmed));
|
||||
assertTrue(textOf(confirmed).contains("\"accepted\":true"),
|
||||
"the SAME terminal that opened the request must be able to confirm it: " + textOf(confirmed));
|
||||
}
|
||||
|
||||
// --- acceptance 6: cancel on an unknown token is clean, not a failure ----------------------
|
||||
|
||||
@Test
|
||||
@DisplayName("cancel on an unknown token reports no pending request, rather than failing")
|
||||
void cancelUnknownTokenIsCleanNotAFailure() {
|
||||
LeadRollover rollover = newRollover(tmp.resolve("h.md").toString());
|
||||
|
||||
McpSchema.CallToolResult r = FleetMcp.handover(rollover, LEAD,
|
||||
Map.of("action", "cancel", "token", "does-not-exist"));
|
||||
assertFalse(r.isError());
|
||||
assertTrue(textOf(r).contains("\"cancelled\":false"), textOf(r));
|
||||
assertTrue(textOf(r).contains("no pending request"), textOf(r));
|
||||
}
|
||||
|
||||
@Test
|
||||
@DisplayName("cancel on a token actually opened reports cancelled:true")
|
||||
void cancelKnownTokenSucceeds() {
|
||||
LeadRollover rollover = newRollover(tmp.resolve("h.md").toString());
|
||||
String token = extractToken(textOf(FleetMcp.handover(rollover, LEAD, Map.of("action", "open"))));
|
||||
|
||||
McpSchema.CallToolResult r = FleetMcp.handover(rollover, LEAD,
|
||||
Map.of("action", "cancel", "token", token));
|
||||
assertFalse(r.isError());
|
||||
assertTrue(textOf(r).contains("\"cancelled\":true"), textOf(r));
|
||||
}
|
||||
|
||||
// --- acceptance 7 (wiring) is covered by FleetdLeadRolloverWiringTest, unchanged -----------
|
||||
}
|
||||
Reference in New Issue
Block a user