fleetd #425 rework: resolve acquireWithWorktree via real placement routing #433
Reference in New Issue
Block a user
Delete Branch "worker/425-rework-placement-resolve-c58ba1-9"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
fleetd #425 rework
Follows up on PR #430 (commit
e1d7dde), which got two of three things right and regressed the third. This PR cherry-pickse1d7ddeonto current main (already has #429/#428), keeps the two good fixes, and replaces the bad one.Kept from
e1d7ddeCompositePeerLauncher.defaultProfile()delegates todefaultProfileFor(MemberRole.DEV)—fleet_profiles'"default"is live, not frozen at construction.PeerLauncher.defaultProfileFor(MemberRole)default method.FleetProfilesLiveDefaultTest, and the live-default additions toCompositePeerLauncherTest.The regression, and the fix
acquireWithWorktreepre-resolved a profile vialauncher.defaultProfileFor(memberRole)— the role pool's first entry, blind to quarantine/cool-off/model-off. That name then went tolauncher.spawnas an explicit profile, which takesCompositePeerLauncher.spawn's throwing branch (enforceNotQuarantined/enforceMaxLoad/enforceModelEnabled) instead of the routing branch a blank profile gets. A quarantined or model-off pool-first profile turned a routine unqualified worktree spawn into a hardPlacementException— undermining fleetd #429's "the fleet keeps working when a model is turned off" guarantee for every worktree spawn.Fix: added
PeerLauncher.routedProfileFor(MemberRole)— the profile an unqualified spawn of that role would actually route to right now, using the same candidate list, the samequarantined/coolingOff/modelOfffiltering, and the samePlacementPolicyspawn()itself consults.CompositePeerLauncherimplements it by extractingspawn()'s context-building into a shared privateplacementContextFor(role, unreachable), sospawn()androutedProfileFor()can never disagree.acquireWithWorktreenow resolves once throughroutedProfileForand reuses that name forrepoRoot,parityOverlay, and the spawn.An explicit profile named by the caller is untouched — still hits the throwing branch, correct for an operator override.
Trade-off (named explicitly, per the ticket's request)
An unqualified worktree spawn still loses
CompositePeerLauncher's cross-candidate retry on a livePeerUnreachableException— a transport failure at spawn time that placement cannot see in advance. It does not lose quarantine/cool-off/model-off routing —routedProfileForalready resolves around those before spawn ever runs. Accepted: a worktree provisioned for the wrong backend is worse than a spawn that fails cleanly and can be retried by the caller.Tests
CompositePeerLauncherTest:routedProfileForSkipsAQuarantinedPoolFirstProfileUnderFixedPolicy,routedProfileForSkipsAModelOffPoolFirstProfileUnderFixedPolicy— both underPlacementPolicies.fixed()(the default policy; the previous round's tests all usedweighted()and never exercisedFixedPlacementPolicy's own inline filter).SessionManagerTest:acquireWithWorktreeRoutesAroundAQuarantinedPoolFirstProfile— provesrepoRoot/parityOverlay/spawn all agree on the routed profile (quarantine skips the pool's first entry), not just the pool-reordered-by-reload case the existing #425 tests already cover.acquireWithWorktreeProvisionsTheOverlayForTheProfileActuallySpawned,acquireWithWorktreeForANonDevRoleUsesThatRolesPoolNotTheDevPool) unmodified — they pass unchanged becauseroutedProfileForagrees withdefaultProfileForwhenever nothing is quarantined/cooling-off/model-off.Build
mvn -f fleetd/pom.xml clean install, unpiped, full output read:Tests run: 1543, Failures: 0, Errors: 0, Skipped: 0—BUILD SUCCESS.Mutation proof
Three mutations, one per filter site added/moved, each killed exactly the expected test(s), reverted after. A control run (unmutated) passed 140/140 on the three affected classes. Full table in the PR description below / reported to the lead.
Round 2 — closing the maxLoad gap the lead found
The lead measured a real regression in the round-1 fix above: an at-cap pool-first profile survives
routedProfileFor(which only filters quarantine/cool-off/model-off), and that name is then fed tolauncher.spawn(SpawnRequest)as an explicit profile — which takes the throwing branch, includingenforceMaxLoad.FixedPlacementPolicy(the default) deliberately never evaluatesmaxLoadduringautomatic selection, so a plain unqualified spawn with no worktree never hits this. The worktree path
now could, purely because its route to the spawn passed through an explicit profile name:
Same #425 shape as before, moved to a different filter instead of closed.
Mechanism chosen:
place(role)+spawn(req, decision)Took the lead's candidate mechanism directly, since it removes the second code path instead of moving
it again:
PeerLauncher.place(MemberRole)— resolves aPlacementDecision(an opaque record wrapping thechosen profile name) using the exact same candidate list, filtering, and
PlacementPolicya blankspawn's routing branch already uses. It does not spawn anything.
PeerLauncher.spawn(SpawnRequest, PlacementDecision)— spawns by honoring that decision through therouting branch (
CompositePeerLauncherroutes straight to the delegate, noenforce*re-check),never the throwing branch.
routedProfileFor(role)is kept as a convenience delegating toplace(role).profile()— the two round-1 tests that call it directly still pass unchanged.SessionManager.acquireWithWorktreenow keeps thePlacementDecisionfromplace()for anunqualified request, and hands it straight to
spawn(req, decision)instead of re-resolving throughan explicit profile name. An explicitly-named profile is untouched: it still goes through
spawn(req)and the throwing branch, exactly as before this rework — an operator naming one profilestill gets every
enforce*check.This makes a resolve-then-spawn caller (the worktree path) and a blank-profile
spawn(req)caller (theno-worktree path) go through the identical routing code for the identical decision, so they can never
disagree about which conditions apply —
maxLoadincluded.maxLoaditself is not touched eitherway: it stays exactly as unenforced for an unqualified spawn as it is on
maintoday. Whether thatshould change is fleetd #435, not this ticket.
The one accepted, unchanged cost from round 1:
spawn(req, decision)commits to the one profileplace()already chose, so an unqualified worktree spawn still does not getCompositePeerLauncher'scross-candidate retry on a live
PeerUnreachableException(a transport failure at spawn time placementcannot see in advance). That trade was already accepted in round 1 and is not widened here.
Also corrected the
SessionManager.acquireWithWorktreecomment's false claim that round 1 "losesnothing else" beside that retry —
maxLoadwas lost too, as a new hard failure, not a retry. Thecomment now names
maxLoadexplicitly, and explains the round-2 mechanism.Tests
Kept all four round-1 tests unmodified — they still pass, since
routedProfileFornow just delegatesto
place(role).profile()and behaves identically.Added one new test,
SessionManagerTest#unqualifiedAcquireAgreesWithAndWithoutAWorktreeWhenTheOnlyProfileIsAtMaxLoad,implementing the lead's probe: one dev profile
a,maxLoad: 1,liveCountpinned at 1,PlacementPolicies.fixed(), unqualified spawn, run once with a worktree and once without. It assertsthe pair agrees — same profile spawned, or the same exception type + message — never a hardcoded
"spawns" or "throws", so it stays correct however #435 eventually resolves whether
maxLoadshouldgate an unqualified spawn.
Build
mvn -f <abs>/fleetd/pom.xml clean install, unpiped, full output read:Tests run: 1544, Failures: 0, Errors: 0, Skipped: 0—BUILD SUCCESS.Mutation proof (two, one per side of the new seam)
SessionManager.java:663— mutatedhandle = unqualifiedProfile ? launcher.spawn(spawnReq, decision) : launcher.spawn(spawnReq);back to the round-1 shape,
handle = launcher.spawn(spawnReq);(always explicit). New probe testfailed exactly as expected:
expected: <spawned:a> but was: <threw:...PlacementException:worker profile 'a' is at maxLoad: 1 live >= 1 cap...>— this is the exact bug the lead found, reproducedlive in this tree. Reverted; control run passed again (
Tests run: 1, Failures: 0).CompositePeerLauncher.java:742(mirror, the other side of the same seam) — addedenforceMaxLoad(decision.profile());as the first line ofspawn(SpawnRequest, PlacementDecision).Same probe test failed the same way (with-worktree now throws, without-worktree still spawns).
Reverted; control run passed again.
Before/after probe, run in this tree
Before (round-1 commit
b066eb1, per the lead's own measurement — not re-run here since that commitis superseded, but reproduced live via mutation #1 above, same exception text):
After (this round, commit on this branch — from the passing probe test):
Out of scope, not fixed (per the ticket's instruction)
One line, spotted but not investigated further to confirm it actually diverges:
SessionManager. acquire's non-worktree branch andacquireWithWorktreeeach build their ownSpawnRequestinlineinstead of sharing one construction helper — structurally the same "one intent, more than one code
path" shape as this ticket, but I have not checked whether the two ever disagree on a check the way
maxLoaddid here.Round 3 — main merged fleetd #435, rewrote the prose it made stale
fleetd #435 (merged to
mainas5d422f8while round 2 was in review) madeFixedPlacementPolicyevaluate
maxLoadduring automatic selection, the same wayweighted/round-robinalready did.Round 2's justification for
place()/spawn(req, decision)rested partly onfixedNOT doingthat, so several comments now described behaviour that no longer exists.
Unit 1 — merge
git merge origin/main(merge commit84034b3, on top of5d422f8). No conflicts —git merge-treeand the actual merge agreed.CompositePeerLauncherTest.java(edited by both sides)auto-merged cleanly.
Build after merge, before any prose edit:
mvn -f <abs>/fleetd/pom.xml compile— 0 errors.mvn -f <abs>/fleetd/pom.xml test-compile— 0 errors.Compile errors from the merge: 0.
Unit 2 — the stale prose
Re-grepped myself (
grep -n "fixed.*maxLoad\|maxLoad.*fixed\|FixedPlacementPolicy" …across thefour named files) rather than trusting the lead's line numbers, since the merge moved them. Found
and rewrote six sites (five named, plus one the lead's grep pattern didn't catch — the
place()javadoc in
CompositePeerLauncher.javahad its own independent copy of the same stale claim):CompositePeerLauncher.java—place()'s own javadoc (not one of the five named, found byre-grepping myself).
CompositePeerLauncher.java—spawn(SpawnRequest, PlacementDecision)'s javadoc (the lead'sline 728, pre-merge).
PeerLauncher.java—routedProfileFor's javadoc (the lead's line 198, pre-merge).PeerLauncher.java—spawn(SpawnRequest, PlacementDecision)'s javadoc.PlacementDecision.java— both paragraphs (the lead's lines 20 and 29, pre-merge).SessionManager.java—acquireWithWorktree's long comment (the lead's line 608, pre-merge).Each was rewritten, not just trimmed, along the lines the lead laid out:
place()) fallsthrough an excluded candidate to the next one; the explicit-profile branch refuses on the same
condition — correct, because an operator who names a profile should get a refusal, not a silent
substitution onto a different backend.
resolving a name through the routing side and then re-entering the refusing side with it.
fixedignoresmaxLoad, and that aPlacementDecisioncouldtherefore name an at-cap profile that would die at
enforceMaxLoadone call later. After #435,place()cannot return an at-cap candidate, so that specific failure is gone.spawn(req, decision): it never re-evaluates a conditionplace()already decided, and it closes the window between that decision and the spawn in whichthe underlying state (another spawn landing on the same profile, a config reload) could
otherwise move — not a failure #435 already prevents.
Re-grepped again after editing (
deliberately never evaluates|deliberately would|ignores maxLoad|does not evaluate maxLoad|never evaluates.*maxLoad|maxLoad.*never evaluat) across allfour files: 0 matches. No stale claim remains.
Unit 3 — re-measured the sibling paths
Same probe as round 2 (
SessionManagerTest#unqualifiedAcquireAgreesWithAndWithoutAWorktreeWhenTheOnlyProfileIsAtMaxLoad,unchanged): one profile
a,maxLoad: 1,liveCountpinned at 1,PlacementPolicies.fixed(),unqualified spawn, with a worktree and without.
Outcome, verbatim, in this tree after the merge:
They agree — the observable asymmetry this PR was filed to fix is now closed by #435, not by
anything in this branch. Both paths fail at the same place, for the same reason, before either one
ever reaches a spawn call:
launcher.place(memberRole)(called beforeacquireWithWorktree'stryblock) throws the identicalPlacementExceptionthat the no-worktree path's blanklauncher.spawn(req)throws, because both now go throughFixedPlacementPolicy.select()'s newat-cap refusal. What remains is the structural argument in Unit 2, not an observable failure.
I additionally tried to re-run round 2's mutation proof unchanged (mutate
SessionManager.java'sspawn call back to always-explicit) to see if it still had teeth against this exact scenario. It
did not: with only one profile configured,
launcher.place(memberRole)now throws before themutated line is ever reached, so the mutation was inert — a mutation that never applies looks
exactly like one that passes. To actually re-prove
spawn(req, decision)still buys something, Iwrote a throwaway test (not part of this PR — written, run, and deleted, never committed) with a
liveCountsupplier that returns 0 on its first call (whatplace()reads) and 1 on every callafter (simulating a second spawn landing on the profile in the gap before this one commits — a
real race between two separate calls in production).
place()approveda;spawn(req, decision)still spawned onawithout re-readingliveCount. MutatingCompositePeerLauncher.java:769(spawn(SpawnRequest req, PlacementDecision decision)'s firstline) to add
enforceMaxLoad(decision.profile());made it throwPlacementException: worker profile 'a' is at maxLoad: 1 live >= 1 cap; refusing spawn — no fallback to another profileinstead. Reverted; control passed again. This is the throwaway proofbehind the "closes the window" claim in Unit 2 — it is not part of the committed test suite.
Build (full, after the merge and all prose edits, run myself, unpiped, full output read)
mvn -f /Users/dai.ha/LTMS/.bridged-worktrees/0f8770-9/fleetd/pom.xml clean installTests run: 1564, Failures: 0, Errors: 0, Skipped: 0—BUILD SUCCESS. Compile errors: 0.(1564 vs. round 2's 1544 — the +20 came from main's #435 merge, not from this branch.)
Other sites with this shape (not fixed, per the ticket's instruction)
Swept every file under
src/main/java/dev/ltms/fleet/referencingPlacementPolicy/placement policy/FixedPlacementPolicy/select(for a comment that justifies itself by what a placementpolicy does or does not evaluate. Found none beyond the six already rewritten above — the only
other files with the same wording pattern (
ConfigRef.java,PlacementPolicyUtil.java,RoundRobinPlacementPolicy.java, etc.) use "never"/"deliberately" for unrelated, still-accuratethings (deferred config keys,
nullmaxLoad meaning unlimited).No behaviour change beyond the merge itself:
place()/PlacementDecision/spawn(req, decision)are untouched, andFixedPlacementPolicy.java/PlacementPolicyUtil.javawere taken wholesale from main, never edited on this branch.
Round 4 — mutation-pinning test for the dropped PlacementDecision
The lead mutated a line I had not touched in prior rounds —
SessionManager.java:677:That drops the
PlacementDecisionentirely for the unqualified case. 186 tests, 0 failures — itsurvived, because every
SessionManagertest in this file usesPlacementPolicies.fixed(), andfixed()answersselect()the same way on every call.weighted()/round-robin()do not:WeightedRoundRobinPolicymutates acurrentscore map on every call, andRoundRobinPlacementPolicyadvances anAtomicInteger indexon every call — two consecutiveselect()calls on the SAME policy instance disagree by design. So with the decision dropped:place()picks profile A (the worktree'srepoRoot/parity overlay get built for A), then theblank-profile
launcher.spawn(spawnReq)runs placement a second time and can pick B — the memberspawns on B inside a worktree provisioned for A. Both live hosts run
placement: weighted, sothis is the configured case.
New test (no production change)
SessionManagerTest#acquireWithWorktreeSpawnsOnTheSameProfileItProvisionedTheWorktreeForUnderARotatingPolicyTwo profiles (
a,b) in aLinkedHashMap(definition order fixed),PlacementPolicies.roundRobin()instead of
fixed(). Round-robin is deterministic AND stateful: the firstselect()call (fromplace()) lands on index 0 ("a"); a secondselect()call on the SAME policy instance (onlyreached if the decision is dropped) lands on index 1 ("b"). The assertion checks AGREEMENT, not a
hardcoded profile name: whichever profile the worktree's parity overlay was recorded for
(
FakeWorktrees.lastOverlay().requested()) must equals.profile() + ".mcp.json"— the profilethe member actually spawned on.
FakeWorktreesalready recorded what it was asked to provision(
OverlayCall.requested()); no test-infrastructure change was needed.Mutation proof (both outcomes, verbatim)
Applied the exact mutation above to
SessionManager.java:677, ran only the new test:RED, and it failed with exactly the predicted mismatch — overlay built for "a" (the first
select(), fromplace()), member spawned on "b" (the secondselect(), from the blank-profilespawn) — confirming the mutated line is actually reached, not short-circuited.
Reverted
SessionManager.java:677to the exact original line (git diff --staton that file showsno diff after reverting). Ran the same test again:
GREEN.
Full build (after revert, run myself, unpiped, full output read)
mvn -f /Users/dai.ha/LTMS/.bridged-worktrees/0f8770-9/fleetd/pom.xml clean installTests run: 1572, Failures: 0, Errors: 0, Skipped: 0—BUILD SUCCESS. Compile errors (grep -c "^\[ERROR\]"): 0. (1572 vs. round 3's 1564 — the +1 is this round's new test; the other +7 camefrom main commits already merged into this branch before this round started — fleetd #421/#434/#436/#438
— not from anything I authored this round.)
Other sites with this shape (report only, not fixed — per the ticket's instruction)
Grepped every caller of
launcher.place(/routedProfileFor(/defaultProfileFor(insrc/main/java/dev/ltms/fleet:SessionManager.acquireWithWorktreeis the only production callerof
place(), and it now carries thePlacementDecisionthrough correctly — no other callerresolves a placement decision and then re-derives it. Broader sweep completed (report only,
nothing changed): every other stateful sequence generator in
src/main/java/dev/ltms/fleet(
RoundRobinPlacementPolicy.index,WeightedRoundRobinPolicy.current,HerdrPeerLauncher.labelSeq/nextLabelSeq,GitWorktrees.seq(worktree-path nonce),SessionManager.nonceSeq(branch nonce),MessageService.ticketSeq,Rendezvous.askSeq,BackendOutagePolicy.incidentSequence,UnixSocketHerdrClient.ids) is consulted from exactlyONE call site for its one logical action — none of them has a second, independent caller that
re-derives the same value a first caller already resolved and used. This shape is specific to
placement:
place()and the blank-profile branch ofspawn()are two separate public entrypoints into the SAME stateful
select(), which is what let a caller resolve once and (if thedecision is dropped) re-derive via the other entry point. No other stateful resolver in this
codebase currently has two such entry points.
Scope discipline
Only
src/test/java/dev/ltms/fleet/session/SessionManagerTest.javawas changed and committed.SessionManager.java:677was mutated and reverted for the proof above, andgit diff --statonit shows no diff — confirmed no production change shipped this round.
Not merging yet. The two-paths split moved one filter over instead of closing.
The design is right and most of this PR is right:
routedProfileForreading the sameplacementContextFor+ oneselectis the correct single source, and killing the blinddefaultProfileForresolver was the fix #430 needed. Quarantine, cool-off and model-off are genuinely closed, and the four new tests prove it.But
maxLoadis not, and the PR's own comment says it is. This sentence inSessionManageris false:maxLoadis lost too — and not as a retry. As a new hard failure.Why
spawn's blank-profile branch never callsenforceMaxLoad. Capacity reaches it only through the placement candidates, andFixedPlacementPolicy— the default policy — deliberately ignores capacity ("capacity gating for automatic placement is deliberately out of scope forfixed"). So a capped profile survivesselect,routedProfileForreturns its name, and the spawn — now explicit — hitsenforceMaxLoadatCompositePeerLauncher.java:394and throws.Measured
One probe, run in both trees, one variable: a single dev profile
awithmaxLoad: 1,liveCountfixed at 1 (exactly at cap),PlacementPolicies.fixed(), unqualified spawn with a worktree.Then the same profile at the same cap, in the rework tree alone, with and without a worktree:
That second pair is the finding. One intent — "spawn a dev, let the fleet choose the profile" — now gets two different answers in the same build, decided by whether a worktree was asked for. That is the same shape #425 is about, one filter over.
Why this matters in real operation
fleet_spawn{worktree:true, ticket}with noprofileis the standard delegation call,fixedis the default policy, and profiles do sit at cap —sonnetis atfree: 0, maxLoad: 3, live: 3on this daemon as I write this. Under this PR that call starts refusing whenever the pool's first profile is full, where today it spawns.I am not asking you to decide whether an unqualified spawn should respect
maxLoad. It probably should. But it must not start doing so as a side effect of a provisioning fix, on one of the two paths only. That is a separate ticket and I will file it.What the next round has to hold
Goal.
repoRoot,parityOverlayand the spawn all name one profile — keep that, it is this PR's whole point.Invariant to add. For every placement policy and every profile state, an unqualified
acquirewith a worktree and an unqualifiedacquirewithout one either both succeed on the same profile, or both fail with the same exception. No condition may become newly fatal on the worktree path alone.Candidate mechanism, not a requirement. Have the launcher hand out a placement decision and then honour it: something like
place(role)returning an opaque decision thatspawn(req, decision)accepts, skipping exactly the checks placement already applied. That removes the second path rather than moving it. If you see a better way to get one path, take it and say why.Test. Both halves of the probe above, as one test class: pool-first profile at
maxLoad, unqualified spawn, with a worktree and without, asserting the two agree. Please also keep the four tests already here.The retry. Losing
CompositePeerLauncher's cross-candidate retry on a livePeerUnreachableExceptionfor a worktree spawn is a real cost and I accept it — a worktree provisioned for the wrong backend is worse. Keep that part of the comment; just correct the claim that nothing else is lost.One thing to fix in the prose either way
The long
SessionManagercomment is good and I want it kept, but the "loses nothing else" sentence has to namemaxLoad. A comment that lists its own exceptions and gets the list wrong is worse than no comment — a later session reads it as the audit.Retracting one paragraph of my rejection: the live-daemon claim was wrong
My rejection of this PR stands. The same-tree asymmetry table is unaffected, and round 2 (task-15) proceeds unchanged. But one paragraph of my reasoning was false, and it was the paragraph that made the problem sound urgent. I wrote:
The first clause is right. The second is wrong, and it invalidates the rest of the sentence.
This daemon does not run
fixed. It sets the policy explicitly:I checked the other host too, because I had made the same assumption about both:
weightedroutes throughPlacementPolicyUtil.available(), which skips an at-cap candidate:Callers of that helper, measured just now —
RoundRobinPlacementPolicy.java:17andWeightedRoundRobinPolicy.java:21. NotFixedPlacementPolicy.So
sonnetsitting atfree: 0, maxLoad: 3, live: 3was a true reading of an irrelevant number. On aweighteddaemon a full profile is already skipped, and the unqualified delegation spawn lands on the next candidate. The behaviour change I predicted for the live fleet cannot happen here, because the branch I predicted it in is not the branch this daemon takes.Why I got it wrong. My first pass used
grep '^placement:\s*$'. That anchor needs the value on the next line, so a same-lineplacement: weightedmatches nothing. I read the empty result as "no key set" and went tofromName(""), which really does returnfixed():Every step after the grep was sound. The grep was the defect. A pattern that finds nothing and a file that says nothing produce the same output, and I did not run a control that would tell them apart.
What this changes for round 2
Nothing in the requirements. The asymmetry is a real defect on its own terms: within one tree,
with worktree → THREWandwithout worktree → SPAWNEDfor the same request. That measurement never depended on which policy the live host runs.What changes is the justification you should write in the PR body. Do not carry my "the live fleet starts refusing" argument forward — it is false and a reviewer will find it. The honest case is:
enforceMaxLoad; the blank branch never does.placement: weighted. Masked is not fixed.Point 3 is worth stating out loud rather than leaving out. A reviewer who checks the live config will find the masking anyway, and a PR that names its own limited blast radius reads as more careful than one that oversells it.
Round 3 verified, and one gap found. Round 4 delegated — test only, no production change.
I re-ran everything rather than reading the report. Merged current main (
12cff28) into the branch myself: no conflicts, 1571 tests green, 0 compile errors. The prose claim checks out — 0 remaining sites sayingfixedignoresmaxLoad, across all four files.Round 3's own corrections to me are accepted. There were six stale sites, not the five I listed; my grep pattern missed a second copy in
CompositePeerLauncher.place()'s javadoc. Handing a worker a count as fact was the error, and the worker re-deriving it was right.Mutation battery — control 186 green
I mutated two lines the implementer did not.
M2 —
place()returns the pool-first default (the original #425 defect): KILLED.Failed 4 tests, two of them at the caller:
SessionManagerTest.acquireWithWorktreeRoutesAroundAQuarantinedPoolFirstProfile,SessionManagerTest.unqualifiedAcquireAgreesWithAndWithoutAWorktreeWhenTheOnlyProfileIsAtMaxLoad, plus twoCompositePeerLauncherTestroutedProfileFortests. So "placement, not a blind pool-first read" is genuinely pinned.M1 — the
SessionManagercall site drops the decision: SURVIVED.186 tests, 0 failures. Nothing objects to the whole mechanism being removed at its only production call site.
This is not an equivalent mutant
I checked before calling it a gap, because "the mutation changed nothing observable" is the honest alternative explanation. It does change behaviour, and the reason is that
select()mutates state:Two consecutive calls return different profiles by design. So under the mutation:
place()picks A;repoRootand the parity overlay are provisioned for A.spawn(spawnReq)runs placement a second time and picks B.That is precisely the invariant this branch's own comment claims to hold, at
SessionManager.java:643: "the overlay/repoRoot above and the spawn here must name the same profile … that agreement is the invariant this rework exists to hold."And it is the configured case, not a corner — both live hosts run
placement: weighted(Macfleetd.yaml:210, fleet01fleetd.yaml:172).A second consequence, not part of the fix: two
select()calls for one spawn also consume two steps of the rotation, skewing distribution away from the configured ratios.Why the suite could not see it
Every
SessionManagertest usesfixed(). Underfixed, twoselect()calls agree, so dropping the decision is invisible. The fixture's policy choice is what hides the defect — the tests are not weak about the assertion, they are weak about the setup.This is the same shape as the lesson this repo keeps relearning: a test on the seam does not prove the caller. Round 2's mutation proved
spawn(req, decision)behaves; round 3's race test called the launcher directly. Neither droveacquireWithWorktree.Round 4 — what I asked for
One test, no production change. It must use a rotating policy, and it must assert the agreement between the provisioned profile and the spawned profile rather than a hardcoded name. I told the worker that a green run is not evidence here — the buggy code is green too — so it must apply my M1 mutation, show the new test red, revert, show it green, and paste both.
The production code in this branch is, as far as I can measure, correct. It is the pin that is missing, and the branch is not mergeable without it: an unpinned invariant that only breaks under the policy both live fleets run is exactly the thing that regresses on the next refactor. This is the third round on #425 for the same reason each time — the mechanism was right and the proof was aimed one layer too low.