Merge #464: a test that charter text names only registered tools (fleetd #464)
CI / contract (push) Successful in 50s
CI / build (push) Successful in 1m56s

CharterToolSurfaceTest extracts every fleet_* / bridge_* token from configured
launch charters and every tool("fleet_...") FleetMcp registers, then asserts the
first set is a subset of the second.

Verified on the merge commit. Its three acceptance criteria are met:

- Catches the ticket's own example. Fixture charter naming bridge_send, the tool
  CB-634 renamed away: KILLED.
- Fails loudly with no charter text. Fixture stripped of every tool name: KILLED
  by its named.isEmpty() guard, not a silent pass.
- Fails loudly with no registered tools. The tool("...") scrape broken so it
  matches nothing: KILLED by its registered.isEmpty() guard.
- And one cell of my own: the server stops registering fleet_reply, which the
  fixture names. KILLED. This is what proves the 'registered' half reads real
  production source and is not a second fixture.

The scrape finds 11 registered tools: ack, ask, list, poll, profiles, reply,
send, spawn, status, stop, whoami. An independent count of every "fleet_x"
literal in FleetMcp.java is also 11.

WHAT THIS DOES NOT CLOSE, and it is the ticket's actual gap. The charter half is
a @TempDir fixture the test writes itself, so no charter text anyone writes can
make this test fail. Measured: the test mentions fleetd.yaml 0 times, and the
commit changes 0 production files -- FleetConfig.validateCharters() still never
reads charter text (0 lines of its body mention a tool name). So this pins the
comparison logic and acts as a rename tripwire for the two tools the fixture
names. It does not check the live config. That needs a production-side check and
is filed as a follow-up.

That residue is my ticket's fault, not the worker's: the three criteria I wrote
are exactly the three it met.

A note on my own battery, because it nearly published four false kills. The
first run showed rc=1 on all four mutation cells and I would have read that as
four kills. It was zsh: unquoted parameters are NOT word-split, so
'mvn -B $scope test' passed '-Dtest=X -DfailIfNoTests=false' as ONE argument and
surefire ran zero tests. The tell was a missing 'Tests run:' line. The rerun
proves the harness first -- selector alone must report 'Tests run: 1' -- and
every cell now prints its surefire summary count so a void cell cannot pass for
a kill.
This commit is contained in:
Dai Ha
2026-09-10 19:10:56 +07:00
@@ -0,0 +1,72 @@
package dev.ltms.fleet.mcp;
import dev.ltms.fleet.config.FleetConfig;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.LinkedHashSet;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import static org.junit.jupiter.api.Assertions.assertTrue;
/** fleetd #464: launch charters must not name MCP tools the server does not register. */
class CharterToolSurfaceTest {
private static final Path MCP_SOURCE = Path.of("src/main/java/dev/ltms/fleet/mcp/FleetMcp.java");
private static Set<String> matches(String text, String regex) {
Matcher m = Pattern.compile(regex).matcher(text);
Set<String> found = new LinkedHashSet<>();
while (m.find()) {
found.add(m.group(1));
}
return found;
}
/** Every {@code fleet_*} or legacy {@code bridge_*} token in the configured launch charters. */
private static Set<String> toolsNamedIn(FleetConfig config) {
return matches(String.join("\n", config.fleet().charters().values()),
"(fleet_[a-z_]+|bridge_[a-z_]+)");
}
/** Every tool {@link FleetMcp} registers, read from its {@code tool("…")} calls. */
private static Set<String> toolsTheServerRegisters() throws Exception {
return matches(Files.readString(MCP_SOURCE), "tool\\(\\\"(fleet_[a-z_]+)\\\"");
}
@Test
@DisplayName("[SOURCE TEXT] every tool named in a configured charter is registered by the server")
void configuredChartersNameOnlyRegisteredTools(@TempDir Path dir) throws Exception {
Path configFile = dir.resolve("charters.yaml");
Files.writeString(configFile, """
fleet:
charters:
dev: |
Send the final handoff through fleet_reply.
reviewer: |
Use fleet_ask only for the lead's decision.
""");
FleetConfig config = FleetConfig.load(configFile);
Set<String> named = toolsNamedIn(config);
Set<String> registered = toolsTheServerRegisters();
assertTrue(!named.isEmpty(),
"the charter fixture named no fleet_* or bridge_* tool. This test would check nothing; "
+ "add charter text that names a tool before changing the extraction.");
assertTrue(!registered.isEmpty(),
"the FleetMcp registration scrape found no tools. This test would check nothing; "
+ "repair the tool(\"…\") extraction before changing the assertion.");
Set<String> unknown = new LinkedHashSet<>(named);
unknown.removeAll(registered);
assertTrue(unknown.isEmpty(),
"configured charter text names " + unknown + ", but FleetMcp does not register it. "
+ "Checked " + named + " against " + registered + ". Fix the charter text or "
+ "register the tool; do NOT weaken this test.");
}
}