diff --git a/11-Features.md b/11-Features.md index b2aa3ab..0682d2b 100644 --- a/11-Features.md +++ b/11-Features.md @@ -54,6 +54,7 @@ six weeks, and the table alone will not carry it. | [See free fleet capacity](#see-free-fleet-capacity) | automatic | CB-573 | `mcp/BridgeMcp` | | [Answer a question on an async delegation](#answer-a-question-on-an-async-delegation) | automatic | CB-574 | `msg/MessageService` | | [Learn why a delegation died](#learn-why-a-delegation-died) | automatic | CB-568 | `inject/CompletionResolver` | +| [Watch the fleet's health](#watch-the-fleets-health) | `health:` | CB-573 | `health/FleetHealthMonitor` | Nearly every knob above lives in one file, on one profile: @@ -933,6 +934,36 @@ was queued but not yet delivered sat until its timeout, which is 30 minutes on a --- +## Watch the fleet's health + +**What.** An opt-in background observer. Each tick it takes **one** `AgentControl.list()` for the whole +fleet and one in-memory roster snapshot, joins them, and classifies every member. A member entering a +fault state logs one WARN; recovering logs one INFO. `bridge_list` reports `healthCoverage`, which is +`off`, `detection-only`, or `full`. + +**On.** A `health:` block in `bridged.yaml` with `enabled: true`. `intervalSeconds` defaults to 30 and +is floored at 15. With no block at all nothing is constructed and no herdr call is ever made. + +**Why.** A fault is usually a *disagreement between two views*, not a value you can read from one of +them. A member that says `BUSY` in the session FSM while herdr says `DONE` has lost its turn boundary — +a real trace sat in that state for eighteen minutes with its ticket still `PENDING` and no fallback +firing. That is why both views must come from the same instant: one list call per tick, never one per +member. Reading panes is the exception, not the method. + +**Gotcha.** Detection and notification are **separate keys** on purpose. An earlier design required a +webhook before `health.enabled` could be turned on, which would have removed real local detection to +avoid a narrower human-notification gap. So health runs with no sink configured and reports +`detection-only` — treat that value as "nobody will be paged", not as "health is off". + +Two more. `tick()` catches `Throwable` and reschedules in a `finally`, because a +`ScheduledExecutorService` never re-runs a task that threw: the earlier version rescheduled as its last +statement, so the first `agents.list()` failure would have stopped health permanently and silently — +exactly when the control link is down, the highest-priority state in the model. And the snapshot fields +this unit cannot yet supply are the named constant `NOT_YET_OBSERVED`, not bare `false`, because to this +classifier `false` means "no fault" rather than "not known yet". + +--- + ## Backfill status This page was started after the fact, so it is **not yet complete**. Entries above are written from