#339: the start-of-line check must skip terminal chrome
CI / build (push) Failing after 1m30s
CI / contract (push) Successful in 1m54s

#339 stopped a member's own prose about an error from recording a credential
outage, by requiring the pattern at the start of its matched line. A bare
lookingAt also rejected a genuine error line rendered as

    | 503 Service Unavailable: upstream credential rejected

The send still failed, but the outage was never recorded. That is the false
negative #339's own invariant 3 named as worse than the false positive it set
out to fix: an unrecorded outage leaves the fleet spawning into a dead
credential.

Measured with a throwaway probe on the raw-scrape path, whose own comment says
to expect leading chrome there: kind=FAILED, sinkNotified=0.

startsWithBackendError now skips a leading run of non-letter, non-digit
characters before the check. That keeps #339's intent: prose still does not
match, because there the pattern sits after words rather than after chrome.
The worker's own prose test still passes.

Mutation: restoring the bare lookingAt fails the new test.
This commit is contained in:
Dai Ha
2026-09-04 16:13:19 +07:00
parent d11d1d157c
commit 6a814176f0
2 changed files with 53 additions and 2 deletions
@@ -613,9 +613,26 @@ public final class CompletionResolver implements TurnListener {
return null;
}
/** True when the error pattern begins the matched pane line, rather than appearing in prose. */
/**
* True when the error pattern begins the matched pane line, rather than appearing in prose.
*
* <p>Leading terminal chrome is skipped first — box-drawing characters, bullets, gutter bars and
* spaces. #339 introduced this check with a bare {@code lookingAt}, and that rejected a genuine
* error line rendered as {@code "| 503 Service Unavailable: ..."}: the send still failed, but the
* credential outage was never recorded. That is the false negative #339's own invariant 3 called
* worse than the false positive it set out to fix — measured with a throwaway probe on the
* raw-scrape path, which is exactly the path whose comment says to expect leading chrome.
*
* <p>Skipping only a leading run of non-letter, non-digit characters keeps the fix's intent. A
* member's prose ({@code "I checked the retry path. An API Error: makes it back off."}) still
* does not match, because there the pattern sits after words, not after chrome.
*/
private static boolean startsWithBackendError(String line, Pattern pattern) {
return pattern.matcher(line).lookingAt();
int i = 0;
while (i < line.length() && !Character.isLetterOrDigit(line.charAt(i))) {
i++;
}
return pattern.matcher(line.substring(i)).lookingAt();
}
/**
@@ -970,6 +970,40 @@ class CompletionResolverTest {
assertTrue(notified.get(0).contains("503 Service Unavailable"), notified.get(0));
}
/**
* fleetd #339 follow-up: a genuine backend error rendered behind terminal chrome must still
* record the credential outage. #339 added a start-of-line check to stop a member's own prose
* being counted as an outage, and a bare {@code lookingAt} also rejected this — the send failed
* but the sink never fired. #339's invariant 3 named that direction as the worse one: a real
* outage going unrecorded leaves the fleet spawning into a dead credential.
*
* <p>The raw-scrape path is where this matters, because its own comment says to expect leading
* TUI chrome there.
*/
@Test
void aRealErrorBehindTerminalChromeStillNotifiesTheSink() {
String block = """
╭──────────────────────────────────────╮
│ 503 Service Unavailable: upstream credential rejected
""";
FakeHerdr herdr = new FakeHerdr().readText(block);
Rendezvous rendezvous = new Rendezvous();
BackendErrorPatternLookup patterns = target -> Pattern.compile("(?i)503 Service Unavailable");
java.util.List<String> notified = new java.util.ArrayList<>();
BackendErrorSink sink = (target, matchedLine, reason) -> notified.add(target + ": " + matchedLine);
CompletionResolver resolver = new CompletionResolver(new AgentControl(herdr), rendezvous,
ExhaustedPatternLookup.none(), ExhaustionSink.none(), patterns, sink);
var waiter = rendezvous.open("term_a");
resolver.resolve("term_a", new CompletionResolver.InFlight(waiter, null));
assertEquals(Rendezvous.Kind.FAILED, waiter.getNow(null).kind(),
"a real backend error must still fail the send");
assertEquals(1, notified.size(),
"a real error line behind box chrome is still a real outage — it must reach the sink, "
+ "or the fleet keeps spawning into a dead credential");
}
// --- fleetd#201 Unit 1: classification inside the fleetd#164 MIN_TURN_NANOS floor -------------
@Test