Compare commits

..

13 Commits

Author SHA1 Message Date
Dai Ha 136bec8e28 Merge PR #660: fleetd #637 — scale the lead context HIGH threshold with the effective auto-compact window
CI / shell-tests (push) Failing after 6s
CI / contract (push) Successful in 1m4s
CI / build (push) Failing after 2m35s
2026-10-03 16:27:55 +02:00
Dai Ha ae94d511d7 fleetd #637: pin the fleet_list window wiring and fold the threshold into the gauge's cache key
CI / shell-tests (pull_request) Failing after 9s
CI / contract (pull_request) Successful in 1m1s
CI / build (pull_request) Failing after 2m22s
Fleetd.leadConfigDirSource built its window argument with nothing calling the
factory itself to prove it, so a mutation to a no-op lookup left the whole
suite green. Add a wiring test that calls the factory directly, modeled on
FleetdLeadConfigDirSourceWiringTest's shape for the configDir half of the
same factory.

LeadContextGauge's result cache keyed only on (configDir, sessionId), but the
cached Reading's state now depends on the caller's resolved effective window.
Fold the derived threshold into the cache key so two reads of the same
session with different windows inside the TTL each report against their own
window.
2026-10-03 16:18:57 +02:00
Dai Ha 436b026696 fleetd #637: scale LeadContextGauge's HIGH threshold with the effective auto-compact window
HIGH_THRESHOLD_TOKENS was a hardcoded 200_000, while the event it warns about
(auto-compaction) is configured per profile via autoCompactWindow and can legally go
as low as 100_000 — making HIGH unreachable before a compaction on such a profile.

LeadContextGauge.read now takes an optional effective window and fires HIGH at 2/3 of
it, falling back to the fixed 200_000 when no window is resolvable (unresolved callers,
including the pre-existing 3-arg read(), keep today's behaviour exactly).

FleetConfig.Profile.effectiveAutoCompactWindow() resolves that window the way a
launched Claude Code session actually reads it: env.CLAUDE_CODE_AUTO_COMPACT_WINDOW
wins over the autoCompactWindow launch flag when both are set.

Wired into both real consumers: fleet_list's context row (FleetMcp.LeadConfigDirSource,
widened with a back-compat constructor so no unrelated call site changes) and the lead
heartbeat's context-high nudge (Fleetd.leadContextLookup/leadContextSource, widened the
same way).
2026-10-03 16:11:41 +02:00
Dai Ha 905fa3a454 Merge PR #658: fleetd #656 — regression tests for both redact() leaks
CI / shell-tests (push) Failing after 12s
CI / contract (push) Successful in 1m28s
CI / build (push) Failing after 1m54s
2026-10-03 16:10:38 +02:00
Dai Ha 011ee80067 fleetd #656: add regression tests for the two cases #639's redact() fix covers
CI / shell-tests (pull_request) Failing after 9s
CI / contract (pull_request) Successful in 1m25s
CI / build (pull_request) Failing after 1m52s
Criterion 19 covers a block-scalar body whose key line falls outside
diff -u's default 3-line context (an 8-line body with only the 6th
line changed). Criterion 20 covers a blank line inside the value,
which used to reset the old indentation-anchored mask.

Both are RED against the pre-#639 redact() (git show 28ea0de) and
GREEN against the current one; each asserts both the secret's
absence and a non-secret control line's presence.
2026-10-03 16:02:50 +02:00
Dai Ha 3fab743152 Merge PR #655: fleetd #639 — mask a masked key's value by file line number, not by indentation anchor
CI / shell-tests (push) Failing after 7s
CI / contract (push) Successful in 1m4s
CI / build (push) Failing after 2m4s
2026-10-03 15:50:21 +02:00
Dai Ha 52eb9c2277 Merge PR #653: fleetd #641 — warn when --set reformats the whole fleetd.yaml 2026-10-03 15:47:30 +02:00
Dai Ha 6794fd8200 Merge PR #654: fleetd #642 — scope the herdr-control source-text guard and add vacuity anchors 2026-10-03 15:47:30 +02:00
Dai Ha 9b5c1cdcff Merge PR #652: fleetd #650 — scope FleetdAssemblyFleetAppTest's dead-end javadoc to loopback-trust 2026-10-03 15:46:42 +02:00
Dai Ha 3b69e0103b fleetd #639: redact block scalars by line number
CI / shell-tests (pull_request) Failing after 6s
CI / build (pull_request) Failing after 1m53s
CI / contract (pull_request) Successful in 1m21s
2026-10-03 15:46:20 +02:00
Dai Ha c97b1bba5a fleetd #641: warn on --set reformat churn
CI / shell-tests (pull_request) Failing after 7s
CI / contract (pull_request) Successful in 50s
CI / build (pull_request) Failing after 2m30s
2026-10-03 15:45:14 +02:00
Dai Ha a42253f597 fleetd #642: widen FleetdHerdrControlConstructionTest to cover FleetdAssembly.java
CI / shell-tests (pull_request) Failing after 6s
CI / contract (pull_request) Successful in 1m22s
CI / build (pull_request) Failing after 1m53s
Add a positive anchor per watched file (Fleetd.java and FleetdAssembly.java),
matching FleetdConfigRefWiringTest's [SOURCE TEXT] style, so a broken read
fails loudly instead of passing the negative check vacuously. Fix dangling
javadoc @link references in FleetdConfigRefWiringTest to the *AssemblyTest
names those classes were renamed to.
2026-10-03 15:45:14 +02:00
Dai Ha e69eafcc9f fleetd #650: scope the READ-unreachable javadoc to loopback-trust
CI / shell-tests (pull_request) Failing after 9s
CI / build (pull_request) Failing after 2m25s
CI / contract (pull_request) Successful in 2m34s
FleetdAssemblyFleetAppTest's class javadoc stated that /sessions'
Authz.Action.READ gate is always refused, and that READ always needs
Caller.resolved(). That is true only under auth.mode: loopback-trust,
the mode this test runs under because it configures no auth: block.
Under auth.mode: token, CallerResolver.resolve() returns before ever
consulting Caller.resolved()/scanComplete(), so a valid bearer token
resolves to PRIMARY with no pid lookup on that path. Names the two
tests that already exercise that path against a real assembly.
2026-10-03 15:40:52 +02:00
8 changed files with 453 additions and 46 deletions
@@ -68,8 +68,10 @@ import java.util.function.LongSupplier;
* (never the whole 52 MB a long-lived transcript reaches on the host this was measured on), and
* {@link #DEFAULT_CACHE_TTL_MILLIS} bounds how often that bounded read actually happens — a burst
* of {@code fleet_list} calls inside one TTL window reads the file once. One instance's cache is
* keyed by {@code (configDir, sessionId)}, so it is safe to share across every lead a single
* {@code fleet_list} call reports on.
* keyed by {@code (configDir, sessionId, highThreshold)}, so it is safe to share across every lead
* a single {@code fleet_list} call reports on, and a call that resolves a different effective
* window for the same lead never reads back a state computed against the other window's
* threshold.
*/
public final class LeadContextGauge {
@@ -192,13 +194,14 @@ public final class LeadContextGauge {
String base = (configDir == null || configDir.isBlank())
? System.getProperty("user.home") + "/.claude"
: configDir;
String cacheKey = base + '\u0000' + sessionId;
long highThreshold = highThreshold(effectiveWindowTokens);
String cacheKey = base + '\u0000' + sessionId + '\u0000' + highThreshold;
long now = clock.getAsLong();
CacheEntry cached = cache.get(cacheKey);
if (cached != null && now - cached.readAtMillis() < ttlMillis) {
return cached.reading();
}
Reading fresh = readUncached(base, sessionId, highThreshold(effectiveWindowTokens));
Reading fresh = readUncached(base, sessionId, highThreshold);
cache.put(cacheKey, new CacheEntry(fresh, now));
return fresh;
}
@@ -58,20 +58,31 @@ import static org.junit.jupiter.api.Assertions.assertEquals;
* {@code HttpClient} — no accessor needed for this half.
*
* <p><strong>{@code GET /sessions} could not be driven the same way</strong>, so this class does
* not pin the merge half of the deleted test's javadoc. {@code /sessions} requires
* {@code Authz.Action.READ}, which — through the REAL assembly's real {@code
* CallerResolver}/{@code ConnectionIdentity} (built with a hardcoded {@code
* new LsofPeerPidLookup()}) — needs {@code Caller.resolved()}, i.e. a real positive pid from
* {@code lsof}. {@code LsofPeerPidLookup} excludes its own pid (see its javadoc), and a JUnit
* test's HTTP client and the daemon under test share one JVM pid, so the resolved pid is always
* {@code -1} and every such request is refused as {@code ANONYMOUS} (fleetd #317's fail-closed
* rule) before the route handler — and its {@code memberHerdr} merge — is ever reached. Verified
* directly: driving {@code GET /sessions} here returns {@code 401 unauthenticated}, not the
* merged body. {@code FleetAppTwoDaemonTest} avoids this because it builds {@code FleetApp} with
* {@code callers: null}, which is not what the real assembly passes. The {@code /healthz} pin
* below is what this class relies on for CB-185's {@code FleetApp} half; {@code
* FleetAppTwoDaemonTest} remains the full behavioural proof that {@code FleetApp} itself merges
* {@code /sessions} correctly once handed two clients.
* not pin the merge half of the deleted test's javadoc. This class configures no {@code auth:}
* block, so it runs under the default {@code loopback-trust} mode ({@code FleetConfig}). Under
* that mode, {@code /sessions} requires {@code Authz.Action.READ}, which — through the REAL
* assembly's real {@code CallerResolver}/{@code ConnectionIdentity} (built with a hardcoded
* {@code new LsofPeerPidLookup()}) — needs {@code Caller.resolved()}, i.e. a real positive pid
* from {@code lsof}. {@code LsofPeerPidLookup} excludes its own pid (see its javadoc), and a
* JUnit test's HTTP client and the daemon under test share one JVM pid, so the resolved pid is
* always {@code -1} and every such request is refused as {@code ANONYMOUS} (fleetd #317's
* fail-closed rule) before the route handler — and its {@code memberHerdr} merge — is ever
* reached. Verified directly: driving {@code GET /sessions} here returns {@code 401
* unauthenticated}, not the merged body. {@code FleetAppTwoDaemonTest} avoids this because it
* builds {@code FleetApp} with {@code callers: null}, which is not what the real assembly
* passes. The {@code /healthz} pin below is what this class relies on for CB-185's {@code
* FleetApp} half; {@code FleetAppTwoDaemonTest} remains the full behavioural proof that
* {@code FleetApp} itself merges {@code /sessions} correctly once handed two clients.
*
* <p><strong>This refusal is {@code loopback-trust}-specific, not a property of {@code
* CallerResolver} in general.</strong> Under {@code auth.mode: token}, {@code
* CallerResolver#resolve} returns before ever consulting {@code Caller.resolved()} or {@code
* Caller.scanComplete()}: a request carrying a valid bearer token in its {@code Authorization}
* header resolves to {@code Role#PRIMARY} with no pid lookup at all, so the same-JVM-pid
* exclusion above never comes into play. {@code FleetdQuarantineOutageDualWindowAssemblyTest}
* and {@code FleetdListReportingSourcesAssemblyTest} both drive {@code Authz.Action.READ} this
* way, over a real {@code McpSyncClient}/{@code HttpClient} against a real {@code
* FleetdAssembly#assembleAndStart}, and both get the real response rather than a refusal.
*
* <p><strong>fleetd #629 follow-up.</strong> The fix below (see {@link TwoHerdrResourcePorts})
* makes {@link #healthzGoesRedWhenTheLeadDaemonIsDownEvenThoughTheMemberIsUp}'s fake {@code
@@ -24,8 +24,8 @@ import static org.junit.jupiter.api.Assertions.assertTrue;
* {@code ConfigRefTest} and {@code FleetdConfigRefCharterToolSurfaceWiringTest} case — because
* neither of those tests constructs its {@code ConfigRef} through {@code main}; both build their own
* instance directly, wired with the check by hand. That silent regression is exactly the shape
* {@link FleetdBackendQuarantineWiringTest}, {@link FleetdLeadSeatWiringTest} and {@link
* FleetdCompletionResolverWiringTest} already guard against for their own constructor arguments —
* {@link FleetdBackendQuarantineAssemblyTest}, {@link FleetdLeadSeatAssemblyTest} and {@link
* FleetdCompletionResolverAssemblyTest} already guard against for their own constructor arguments —
* this class is the same class of gap for fleetd #474's {@code extraValidation} argument, following
* their approach.
*
@@ -2,16 +2,67 @@ package dev.ltms.fleet;
import java.nio.file.Files;
import java.nio.file.Path;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* {@code AgentControl} caches {@code paneByTerminal}, so {@code HerdrRouter} must be its only
* production factory — a second instance means a second cache; the same reasoning applies to
* {@code WorkspaceControl}. {@code HerdrRouter}'s constructor is the one place both are built.
*
* <p><b>This test checks source text, not runtime behaviour.</b> It never constructs a {@code
* HerdrRouter} and never runs {@code FleetdAssembly.assembleAndStart} — a green result proves only
* that neither watched file's text contains {@code new AgentControl(} or {@code new
* WorkspaceControl(}. It does not prove the instances {@code HerdrRouter} does build are the ones
* actually wired through the rest of the daemon, and it does not cover a bypass written into a
* production file other than the two this test reads.
*/
class FleetdHerdrControlConstructionTest {
private static String source(String relativePath) throws Exception {
return Files.readString(Path.of(relativePath));
}
@Test
@DisplayName("[SOURCE TEXT] Fleetd.java never constructs AgentControl or WorkspaceControl directly")
void fleetdDelegatesStatefulControlsToTheRouter() throws Exception {
// AgentControl caches paneByTerminal, so the router must be its only production factory.
String source = Files.readString(Path.of("src/main/java/dev/ltms/fleet/Fleetd.java"));
assertFalse(source.contains("new AgentControl("));
assertFalse(source.contains("new WorkspaceControl("));
String source = source("src/main/java/dev/ltms/fleet/Fleetd.java");
// A broken read (wrong working directory, wrong path, a file that came back empty) would
// make the assertFalse checks below pass vacuously — a "clean" negative check that actually
// checked nothing. Guard against that first, with an anchor that has nothing to do with
// this mutation, so a bad read fails loudly here instead of silently proving nothing below.
assertTrue(source.contains("public final class Fleetd"),
"the read of Fleetd.java did not come back containing its own class declaration — "
+ "the assertFalse checks below would pass vacuously on a broken read; fix the "
+ "read before trusting this test.");
assertFalse(source.contains("new AgentControl("),
"Fleetd.java must not construct AgentControl directly — HerdrRouter is its only "
+ "production factory");
assertFalse(source.contains("new WorkspaceControl("),
"Fleetd.java must not construct WorkspaceControl directly — HerdrRouter is its only "
+ "production factory");
}
@Test
@DisplayName("[SOURCE TEXT] FleetdAssembly.java never constructs AgentControl or WorkspaceControl directly")
void fleetdAssemblyDelegatesStatefulControlsToTheRouter() throws Exception {
String source = source("src/main/java/dev/ltms/fleet/FleetdAssembly.java");
assertTrue(source.contains("final class FleetdAssembly"),
"the read of FleetdAssembly.java did not come back containing its own class "
+ "declaration — the assertFalse checks below would pass vacuously on a broken "
+ "read; fix the read before trusting this test.");
assertFalse(source.contains("new AgentControl("),
"FleetdAssembly.java must not construct AgentControl directly — HerdrRouter is its "
+ "only production factory");
assertFalse(source.contains("new WorkspaceControl("),
"FleetdAssembly.java must not construct WorkspaceControl directly — HerdrRouter is "
+ "its only production factory");
}
}
@@ -0,0 +1,63 @@
package dev.ltms.fleet;
import dev.ltms.fleet.config.FleetConfig;
import dev.ltms.fleet.mcp.FleetMcp;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
/**
* Pins {@link Fleetd#leadConfigDirSource}'s own wiring of the window lookup into the returned
* {@link FleetMcp.LeadConfigDirSource}, not only the detached {@link Fleetd#leadContextWindowLookup}
* factory it delegates to. Calls the producer directly, with real {@link FleetConfig.Profile}/
* {@link FleetConfig.Leader} fixtures, and asserts on {@code windowFor()} — the companion of
* {@link FleetdLeadConfigDirSourceWiringTest}, which pins the same factory's {@code configDirFor()}.
*/
class FleetdLeadConfigDirSourceWindowWiringTest {
private static FleetConfig.Profile profileWithWindow(String name, Integer autoCompactWindow) {
return new FleetConfig.Profile(name, null, "claude-sonnet-5", null, null, null,
"tab", "fleet", "w #{n}", null, null, null, null, null, null, null,
null, null, true, null, null, null, null, null, autoCompactWindow, null);
}
private static FleetConfig.Leader leadOnProfile(String profile) {
return new FleetConfig.Leader(profile, "lead: primary", 1, "lead:", 10, "claude", "claude-sonnet-5");
}
@Test
@DisplayName("the returned source resolves the lead's REAL configured effective window, not a hardcoded null")
void resolvesTheRealConfiguredWindow() {
Map<String, FleetConfig.Profile> profiles = Map.of("opus", profileWithWindow("opus", 250_000));
Map<String, FleetConfig.Leader> leaders = Map.of("primary", leadOnProfile("opus"));
FleetMcp.LeadConfigDirSource source = Fleetd.leadConfigDirSource(() -> profiles, leaders);
assertEquals(250_000L, source.windowFor().apply("primary"),
"windowFor must delegate to the real leadContextWindowLookup, not a stub that always "
+ "returns null");
}
@Test
@DisplayName("a lead on a profile with no window configured still resolves to null, not a crash")
void leadWithNoWindowConfiguredResolvesToNull() {
Map<String, FleetConfig.Profile> profiles = Map.of("opus", profileWithWindow("opus", null));
Map<String, FleetConfig.Leader> leaders = Map.of("primary", leadOnProfile("opus"));
FleetMcp.LeadConfigDirSource source = Fleetd.leadConfigDirSource(() -> profiles, leaders);
assertNull(source.windowFor().apply("primary"));
}
@Test
@DisplayName("an unrecognised lead name resolves to null, not a thrown exception")
void unrecognisedLeadNameResolvesToNull() {
FleetMcp.LeadConfigDirSource source = Fleetd.leadConfigDirSource(Map::of, Map.of());
assertNull(source.windowFor().apply("ghost-lead"));
}
}
@@ -88,4 +88,23 @@ class LeadContextGaugeHighThresholdTest {
legacyGauge.read(legacyConfigDir, SESSION_ID, "claude").state(),
"the 3-arg read() (no window argument at all) must behave exactly like passing a null window");
}
@Test
@DisplayName("a second read with a different window, within the TTL, reports against its own window, not the first call's cached state")
void aSecondReadWithADifferentWindowWithinTheTtlReportsAgainstItsOwnWindow(@TempDir Path tmp) throws IOException {
String configDir = writeTranscript(tmp, SESSION_ID, 90_000);
long[] now = {0L};
LeadContextGauge gauge = new LeadContextGauge(() -> now[0], 5_000);
LeadContextGauge.Reading first = gauge.read(configDir, SESSION_ID, "claude", 100_000L);
assertEquals(LeadContextGauge.State.HIGH, first.state(),
"90,000 tokens against a 100,000 window is HIGH");
now[0] += 1_000; // stays inside the 5,000ms TTL — the cache key must still vary with the window
LeadContextGauge.Reading second = gauge.read(configDir, SESSION_ID, "claude", 1_000_000L);
assertEquals(LeadContextGauge.State.OK, second.state(),
"90,000 tokens against a 1,000,000 window must report OK regardless of the previous call's "
+ "window, even while that call's cache entry is still within its TTL");
}
}
+139 -22
View File
@@ -2,6 +2,10 @@
#
# The one auditable way to edit the live fleetd.yaml.
#
# `--set` uses yq and rewrites the whole YAML document in yq's output style. Use `--from` for a
# candidate whose comment alignment or other formatting carries meaning: it copies that file
# verbatim while keeping this script's backup, parse check, atomic install, and verdict read-back.
#
# fleetd ticket #635 — why this exists at all: fleetd.yaml is gitignored and holds the live
# fleet's settings. A bad raw edit reaches a daemon that is already serving, so a direct `Edit`
# on it is refused by policy. This script is the allow-listed alternative, and it is not just
@@ -46,6 +50,11 @@
# scripts/config-edit.sh --dry-run --set <yq-path>=<value>
# scripts/config-edit.sh --restore
#
# `--set` rewrites the whole file in yq's output style, not only the requested keys. The script
# warns before installation when the candidate changes more lines than its number of --set pairs.
# Use `--from <candidate.yaml>` when comment alignment or other formatting is meaningful: --from
# copies the candidate verbatim, with no yq round-trip.
#
# `--set .a.b=` (an empty value — a forgotten typo) is REFUSED, not accepted as "clear the
# field": a null value falls back to its default rather than erroring, which is silent, not
# safe. To clear a key on purpose, write a literal null: `--set .a.b=null`. Every other value
@@ -154,18 +163,10 @@ done
# story: a YAML block scalar (`|`, `|-`, `>`, `>-`, ...) puts the VALUE on the lines that follow
# the key, each indented deeper than it. The key-name match above only ever sees the key line
# itself, so those continuation lines used to flow straight through unredacted while the key line
# right above them printed a reassuring "<redacted>" — an incomplete redactor that looks complete
# is worse than one that visibly does nothing, because it stops a reviewer from looking further.
# The fix is structural, not another name to match: once a key line is masked, every following
# line indented STRICTLY DEEPER than that key is masked too, by indentation alone, until the
# indentation returns to the key's own level or shallower. This needs no knowledge of the key's
# name, so it covers a block scalar under any masked key — but ONLY while that key's own line is
# itself inside the hunk being printed. `diff -u` prints just three lines of context, so a block
# scalar's body often reaches this function with its key line left out; there is then nothing to
# anchor to, `masked` is never set, and the body prints in full. A blank line inside a block
# scalar loses the anchor the same way, because a blank diff line measures as indent 0. Both are
# measured and filed as fleetd #639 — do not read this paragraph as a guarantee that a masked
# key's value can never be printed.
# right above them printed a reassuring "<redacted>". The redactor maps masked continuation lines
# from each complete file before it reads the diff. It then masks a printed line when that file
# line is inside a masked key's value. This covers block-scalar bodies even when the key line is
# outside the printed hunk, and it keeps blank lines inside the value masked.
#
# `redact` is always fed `diff -u` output, and every line of a unified diff starts with exactly
# one of ' ', '+', '-' (the three body markers; '@'/'-'/'+' for the three header-line kinds too).
@@ -174,37 +175,102 @@ done
# column shallower than it really is, and either wrongly escapes a continuation mask or wrongly
# ends one early. Tabs are out of scope: YAML forbids them for indentation, and this is a bounded
# fix, not a YAML parser.
map_masked_lines() {
local file="$1" side="$2" line content indent lead key line_number=0
local masked=0 masked_indent=0
case "$side" in
old) OLD_MASKED_LINES=() ;;
new) NEW_MASKED_LINES=() ;;
*) die "internal error: unknown redaction map side $side" ;;
esac
while IFS= read -r line || [ -n "$line" ]; do
line_number=$((line_number + 1))
content="$line"
indent=0
while [ "${content:$indent:1}" = " " ]; do indent=$((indent + 1)); done
if [ "$masked" = 1 ]; then
if [ -z "${content// /}" ] || [ "$indent" -gt "$masked_indent" ]; then
case "$side" in
old) OLD_MASKED_LINES[$line_number]=1 ;;
new) NEW_MASKED_LINES[$line_number]=1 ;;
esac
continue
fi
masked=0
fi
if [[ "$content" =~ ^([[:space:]]*)([A-Za-z0-9_.-]+:) ]]; then
lead="${BASH_REMATCH[1]}"
key="${BASH_REMATCH[2]}"
if [[ "$key" =~ (TOKEN|SECRET|PASSWORD|PASSWD|PASSPHRASE|CREDENTIAL|URI|_KEY) ]]; then
masked=1
masked_indent="$indent"
fi
fi
done < "$file"
}
redact() {
local line prefix content indent lead key
local masked=0 masked_indent=0 saved_nocasematch=0
local old_file="$1" new_file="$2"
local line prefix content indent lead key old_line=0 new_line=0 in_hunk=0
local old_masked new_masked saved_nocasematch=0
shopt -q nocasematch && saved_nocasematch=1
shopt -s nocasematch
sed -E 's#://[^@]*@#://<redacted>@#g' | while IFS= read -r line || [ -n "$line" ]; do
map_masked_lines "$old_file" old
map_masked_lines "$new_file" new
while IFS= read -r line || [ -n "$line" ]; do
if [[ "$line" =~ ^@@\ -([0-9]+)(,([0-9]+))?\ \+([0-9]+)(,([0-9]+))?\ @@ ]]; then
old_line="${BASH_REMATCH[1]}"
new_line="${BASH_REMATCH[4]}"
in_hunk=1
printf '%s\n' "$line"
continue
fi
case "$line" in
[\ +-]*) prefix="${line:0:1}"; content="${line:1}" ;;
*) prefix=""; content="$line" ;;
esac
old_masked=0
new_masked=0
if [ "$in_hunk" = 1 ]; then
case "$prefix" in
' ')
[ "${OLD_MASKED_LINES[$old_line]:-}" = 1 ] && old_masked=1
[ "${NEW_MASKED_LINES[$new_line]:-}" = 1 ] && new_masked=1
old_line=$((old_line + 1)); new_line=$((new_line + 1)) ;;
-)
[ "${OLD_MASKED_LINES[$old_line]:-}" = 1 ] && old_masked=1
old_line=$((old_line + 1)) ;;
+)
[ "${NEW_MASKED_LINES[$new_line]:-}" = 1 ] && new_masked=1
new_line=$((new_line + 1)) ;;
esac
fi
indent=0
while [ "${content:$indent:1}" = " " ]; do indent=$((indent + 1)); done
if [ "$masked" = 1 ] && [ "$indent" -gt "$masked_indent" ]; then
if [ "$old_masked" = 1 ] || [ "$new_masked" = 1 ]; then
printf '%s%*s<redacted>\n' "$prefix" "$indent" ""
continue
fi
masked=0
if [[ "$content" =~ ^([[:space:]]*)([A-Za-z0-9_.-]+:) ]]; then
lead="${BASH_REMATCH[1]}"
key="${BASH_REMATCH[2]}"
if [[ "$key" =~ (TOKEN|SECRET|PASSWORD|PASSWD|PASSPHRASE|CREDENTIAL|URI|_KEY) ]]; then
printf '%s%s%s <redacted>\n' "$prefix" "$lead" "$key"
masked=1
masked_indent="$indent"
continue
fi
fi
printf '%s\n' "$line"
printf '%s\n' "$line" | sed -E 's#://[^@]*@#://<redacted>@#g'
done
[ "$saved_nocasematch" = 1 ] || shopt -u nocasematch
}
@@ -463,6 +529,50 @@ parse_check() {
yq eval '.' "$1" >/dev/null 2>&1
}
# Count logical changed lines in a unified diff. A replacement counts once, while an added or
# deleted line also counts once. One changed `--set` value normally produces one changed line.
changed_line_count() {
local before="$1" after="$2" line count=0 old_count=0 new_count=0
while IFS= read -r line || [ -n "$line" ]; do
case "$line" in
---\ *|+++\ *|@@\ *)
if [ "$old_count" -gt "$new_count" ]; then
count=$((count + old_count))
else
count=$((count + new_count))
fi
old_count=0
new_count=0
;;
-*) old_count=$((old_count + 1)) ;;
+*) new_count=$((new_count + 1)) ;;
*)
if [ "$old_count" -gt "$new_count" ]; then
count=$((count + old_count))
else
count=$((count + new_count))
fi
old_count=0
new_count=0
;;
esac
done < <(diff -u "$before" "$after" || true)
if [ "$old_count" -gt "$new_count" ]; then
count=$((count + old_count))
else
count=$((count + new_count))
fi
printf '%s' "$count"
}
warn_set_reformat() {
local before="$1" after="$2" changed
changed="$(changed_line_count "$before" "$after")"
if [ "$changed" -gt "${#SETS[@]}" ]; then
warn "--set changed $changed candidate lines for ${#SETS[@]} pair(s); yq reformatted the whole file. Use --from for meaningful comment alignment or formatting."
fi
}
install_candidate() {
local cand="$1" live="$2"
mv -f "$cand" "$live"
@@ -624,10 +734,14 @@ run_edit() {
fi
ok "candidate parses"
if [ "$MODE" = "set" ]; then
warn_set_reformat "$backup" "$cand"
fi
apply_mode "$cand" "$orig_mode"
say "change (redacted)"
diff -u "$backup" "$cand" | redact || true
diff -u "$backup" "$cand" | redact "$backup" "$cand" || true
say "install"
install_candidate "$cand" "$CONFIG" \
@@ -655,8 +769,11 @@ dry_run_diff() {
rm -f "$cand"; CAND=""
die "candidate does not parse as valid YAML — this was a --dry-run, nothing would have been installed either"
fi
if [ "$MODE" = "set" ]; then
warn_set_reformat "$CONFIG" "$cand"
fi
say "dry run — diff (redacted), nothing installed"
diff -u "$CONFIG" "$cand" | redact || true
diff -u "$CONFIG" "$cand" | redact "$CONFIG" "$cand" || true
rm -f "$cand"; CAND=""
return 0
}
+143
View File
@@ -474,6 +474,90 @@ test_passphrase_key_is_redacted() {
assert_not_contains "FAKELEAK-PASSPHRASE" "$RUN_OUTPUT" "passphrase case: the passphrase VALUE must never leak"
}
# ------------------- acceptance criterion 19: the key line falls outside the printed hunk
# fleetd #656 — criteria 15a/15b both put the edit right next to the key line, so the key line is
# always inside diff -u's default 3-line context. Neither covers the actual case #639 fixed: an
# 8-line block-scalar body with only its SIXTH line changed, so the printed hunk (3 lines of
# context on each side of the change) covers body lines 3-8 and never includes the "token:" key
# line at all. The old, line-by-line redact() only ever masks after it has SEEN the key line go
# past; with the key line outside the hunk it never sets its mask, and the whole body — the
# changed line included — passes through raw. The control key sits right after the body, inside
# the same hunk, so the positive control below proves the fix is not simply printing nothing.
new_fixture_hunk_without_key_line() {
local dir
dir="$(mktemp -d "$TMP/fixture.XXXXXX")"
cat > "$dir/fleetd.yaml" <<'YAML'
bind:
host: 127.0.0.1
port: 19999
broker:
uri: amqp://user:hunter2@host/vhost
auth:
token: |
SECRET-LINE-1
SECRET-LINE-2
SECRET-LINE-3
SECRET-LINE-4
SECRET-LINE-5
SECRET-LINE-6
SECRET-LINE-7
SECRET-LINE-8
control: CTRL-MUST-APPEAR
profiles:
sonnet:
weight: 3
maxLoad: 5
YAML
: > "$dir/fleetd.out"
printf '%s' "$dir"
}
test_key_line_outside_hunk_is_still_redacted() {
local dir
dir="$(new_fixture_hunk_without_key_line)"
sed 's/SECRET-LINE-6$/SECRET-LINE-6-CHANGED/' "$dir/fleetd.yaml" > "$dir/candidate.yaml"
start_run "$dir" 5 --from "$dir/candidate.yaml"
sleep 1
printf 'config reloaded\n' >> "$dir/fleetd.out"
collect_run "$dir"
assert_equals 0 "$RUN_RC" "hunk-without-key-line case reload exit code"
# Positive control FIRST: without this, a diff that printed nothing at all would pass the
# negative assertion right below identically to a correctly redacted one.
assert_contains "CTRL-MUST-APPEAR" "$RUN_OUTPUT" "hunk-without-key-line case: the non-secret control line must still print unmasked"
assert_not_contains "SECRET-LINE-6-CHANGED" "$RUN_OUTPUT" "hunk-without-key-line case: the changed body line must never leak, even with the key line outside the printed hunk"
}
# ------------------------------- acceptance criterion 20: a blank line inside the value
# fleetd #656 — the old, line-by-line redact() reset its mask on any line whose indentation was
# not STRICTLY greater than the key's, and a wholly blank line has indentation 0, so it reset the
# mask exactly like the "control:" line that legitimately ends the block scalar. Everything after
# the blank line then printed raw. The current fix tracks masked lines by FILE line number instead
# of by indentation seen so far, so a blank line inside the value stays masked.
new_fixture_blank_line_in_value() {
local dir
dir="$(mktemp -d "$TMP/fixture.XXXXXX")"
printf 'bind:\n host: 127.0.0.1\n port: 19999\nbroker:\n uri: amqp://user:hunter2@host/vhost\nauth:\n token: |\n LEAK-BEFORE-BLANK\n\n LEAK-AFTER-BLANK\n control: CTRL-MUST-APPEAR\nprofiles:\n sonnet:\n weight: 3\n maxLoad: 5\n' > "$dir/fleetd.yaml"
: > "$dir/fleetd.out"
printf '%s' "$dir"
}
test_blank_line_inside_value_is_still_redacted() {
local dir
dir="$(new_fixture_blank_line_in_value)"
sed 's/LEAK-AFTER-BLANK$/LEAK-AFTER-BLANK-CHANGED/' "$dir/fleetd.yaml" > "$dir/candidate.yaml"
start_run "$dir" 5 --from "$dir/candidate.yaml"
sleep 1
printf 'config reloaded\n' >> "$dir/fleetd.out"
collect_run "$dir"
assert_equals 0 "$RUN_RC" "blank-line-in-value case reload exit code"
assert_contains "CTRL-MUST-APPEAR" "$RUN_OUTPUT" "blank-line-in-value case: the non-secret control line must still print unmasked"
assert_not_contains "LEAK-AFTER-BLANK-CHANGED" "$RUN_OUTPUT" "blank-line-in-value case: the line after the blank must never leak"
}
# ----------------------------------- acceptance criterion 16: a failing --set must not echo value
# fleetd #635 follow-up (ticket comment 17673, defect 8) — apply_set_pairs used to echo the FULL
# "$kv" (path=value, exactly as typed) in its yq-failure messages, so a broken --set with a
@@ -545,6 +629,57 @@ test_refusal_shape_from_parse_failure_wording_is_recognised() {
assert_equals 4 "$RUN_RC" "the parse-failure refusal shape must also exit 4, not be read as silence"
}
# --set runs yq over the whole candidate. It warns when that changes more lines than the requested
# pairs, but a simple file with only the intended changed line must stay quiet.
new_fixture_reformat_sensitive() {
local dir
dir="$(mktemp -d "$TMP/fixture.XXXXXX")"
cat > "$dir/fleetd.yaml" <<'YAML'
# A section comment that documents the next block.
bind:
host: 127.0.0.1 # Keep this aligned with the port note.
port: 19999 # A fixture port.
# These comments use their placement as documentation.
profiles:
sonnet:
weight: 3
bootstrapText: >-
First line.
Second line.
YAML
: > "$dir/fleetd.out"
printf '%s' "$dir"
}
test_set_warns_when_yq_reformats_extra_lines() {
local dir
dir="$(new_fixture_reformat_sensitive)"
start_run "$dir" 5 --set '.profiles.sonnet.weight=4'
sleep 1
printf 'config reloaded\n' >> "$dir/fleetd.out"
collect_run "$dir"
assert_equals 0 "$RUN_RC" "reformat warning case reload exit code"
assert_contains "yq reformatted the whole file" "$RUN_OUTPUT" \
"a --set that changes extra candidate lines must warn before installation"
}
test_set_stays_quiet_without_formatting_churn() {
local dir
dir="$(new_fixture)"
start_run "$dir" 5 --set '.profiles.sonnet.weight=4'
sleep 1
printf 'config reloaded\n' >> "$dir/fleetd.out"
collect_run "$dir"
assert_equals 0 "$RUN_RC" "no-reformat warning case reload exit code"
assert_not_contains "yq reformatted the whole file" "$RUN_OUTPUT" \
"a --set that changes only its requested candidate line must not warn"
}
echo "== acceptance criterion 1: refusal restores byte for byte =="
test_refusal_restores_byte_for_byte
echo "== acceptance criterion 2: clean reload keeps the edit =="
@@ -573,6 +708,10 @@ echo "== acceptance criterion 15a: a block scalar's continuation lines are redac
test_block_scalar_continuation_is_redacted
echo "== acceptance criterion 15b: a passphrase key is also recognised =="
test_passphrase_key_is_redacted
echo "== acceptance criterion 19: the key line falls outside the printed hunk =="
test_key_line_outside_hunk_is_still_redacted
echo "== acceptance criterion 20: a blank line inside the value =="
test_blank_line_inside_value_is_still_redacted
echo "== acceptance criterion 16: a failing --set must not echo its value =="
test_failing_set_does_not_echo_its_value
echo "== extra: dry-run never installs, and redacts =="
@@ -581,5 +720,9 @@ echo "== extra: --check is read-only and always exits 0 =="
test_check_is_read_only_and_exits_zero
echo "== extra: the parse-failure refusal shape is also recognised =="
test_refusal_shape_from_parse_failure_wording_is_recognised
echo "== acceptance criterion 17: --set warns about yq formatting churn =="
test_set_warns_when_yq_reformats_extra_lines
echo "== acceptance criterion 18: --set stays quiet without formatting churn =="
test_set_stays_quiet_without_formatting_churn
printf 'PASS: config-edit acceptance criteria\n'