Features: redeploy the daemon safely

Dai Ha
2026-08-15 15:14:10 +02:00
parent f4c70b7786
commit 6fc8603e27
+34
@@ -60,6 +60,7 @@ six weeks, and the table alone will not carry it.
| [Tell a usage-limit refusal from a real reply](#tell-a-usage-limit-refusal-from-a-real-reply) | profile `exhaustedPattern:` | CB-578 | `inject/CompletionResolver` |
| [Stop spawning onto an exhausted account](#stop-spawning-onto-an-exhausted-account) | `quarantineCooldownSeconds:` + profile `credentialId:` | CB-578 | `placement/BackendQuarantine` |
| [See which charter a member got](#see-which-charter-a-member-got) | automatic | CB-571 | `peer/CharterReceipt` |
| [Redeploy the daemon safely](#redeploy-the-daemon-safely) | run the script | — | `scripts/redeploy-bridged.sh` |
Nearly every knob above lives in one file, on one profile:
@@ -1099,6 +1100,39 @@ a `PeerHandle` implementation, this is the line that will not let you skip it.
---
## Redeploy the daemon safely
**What.** `scripts/redeploy-bridged.sh` rebuilds the jar and restarts `bridged` as one command. It
builds *before* it stops anything, waits for the old process to actually exit, restarts from a login
shell with `cwd = bridged/`, then polls `/healthz` and reports the herdr protocol number, a fresh
`bridged listening` line, the config keys accepted or deferred at boot, and any `ERROR` lines since
the restart. `--check` reports state and changes nothing; `--yes` skips the drain prompt;
`--no-build` restarts the jar already on disk.
**On.** Run it. Nothing is automatic — the daemon never restarts itself.
**Why.** A merge is not a deployment: the running daemon holds the jar it was started with, so merged
code does nothing until this runs. That gap has silently shipped inert features more than once — the
whole `health:` stack sat merged and doing nothing for two tickets. The script also exists so the
operator can allow-list **one** auditable command instead of approving a `kill` and a `java -jar`
separately every time, which is what a lead would otherwise have to ask for on every deploy.
**Gotcha.** The check that matters most has no log line anywhere in the daemon: `bridged` inherits
`WORKER_GITEA_TOKEN` from the shell that starts it, via `${SHARED_ENV}/tools/secrets.sh`. Start it
from a non-login shell and the variable is empty — the daemon boots normally, `/healthz` is green,
and the failure surfaces much later as workers that cannot open a PR. `--check` is the only thing
that reports this, and it tests whether the name resolves without ever printing the value.
Second gotcha: a green `/healthz` only proves herdr *answers*. If herdr's protocol number has moved,
every spawn can still fail. The script prints the protocol it saw so you can compare it; prove a real
spawn before trusting the fleet.
The script never escalates to `kill -9`. The shutdown hook releases sessions and worktrees in order,
and a hard kill can leave worktrees and panes behind; if the process will not exit it stops and tells
you rather than forcing it.
---
## Backfill status
This page was started after the fact, so it is **not yet complete**. Entries above are written from