diff --git a/docs/CB-308-Multi-Host-Federation.md b/docs/CB-308-Multi-Host-Federation.md index 18e3eff..1a78ad7 100644 --- a/docs/CB-308-Multi-Host-Federation.md +++ b/docs/CB-308-Multi-Host-Federation.md @@ -1,6 +1,6 @@ # CB-308 — Multi-Host Federation (Stage 5) -**Status:** design note (proposal) +**Status:** design note (proposal) — core design decisions resolved 2026-08-10 (§7) **Depends on:** CB-307 (broker-based reliable delivery) — CB-308 is the multi-host layer built *on* CB-307's broker fabric. **Relates to:** CB-401 (`PeerHandle` opaque id), CB-304 (`rosterView`), CB-306 (spawn-readiness), @@ -143,6 +143,8 @@ extending the same broker from "worker→primary reliability" to "gateway↔gate 5. **Trust** — the broker connection is now the security boundary. A gateway injects env/tokens at daemon privilege (the CB-401 Stage-C concern), so a **remote-triggered spawn/send** needs authn/authz: who may act on which host, and which control channels a gateway will honour. + *Authenticity* is resolved — signed messages, §7.1; *authorization* (who may do what) remains + open — §8. ## 5. The one thing the broker does NOT dissolve @@ -191,7 +193,44 @@ useful), and build CB-308's items on top once the broker fabric exists. Choose C channel naming **multi-host-ready** now (per-agent routing keys, a `roster.*` topic namespace) so CB-308 doesn't have to repaint the topology. -## 7. Open questions +## 7. Resolved design decisions (2026-08-10) + +Settled in a design review of this note + wiki chapter 10. The broker-level operational rules +(inbox caps, TLS + private broker, schema versioning, trace id, exclusive consumers, U8 broadcast) +are recorded in wiki 10 §10; the CB-308-side decisions are below. + +1. **Sender authenticity — sign every message.** Each gateway holds its own signing key and signs + what it publishes (sender gid, `msgId`, timestamp). The receiving gateway verifies the + signature **and** checks against the roster that the claimed sender lives on the signing + gateway's host. This extends the single-host invariant — *identity comes from the connection, + never an argument* — across the broker: cross-host, identity comes from the key. Complements + (not replaces) per-gateway broker logins over TLS. +2. **Profiles are owned by the worker's host.** `bridge_spawn(profile, host)` resolves the name in + the *target* gateway's `bridged.yaml`. Gateways advertise their profile names in presence + heartbeats, so a leader sees what each host offers before spawning; an unknown name is a clear + error from the target. Secrets (base URLs, tokens) never leave the host that uses them. +3. **Repo provisioning — clone from the forge, pinned.** A cross-host spawn names the repo URL and + the exact commit. The target gateway clones from the forge into a local cache (first spawn + only), then cuts a per-worker worktree — the CB-301-ext flow with a clone step in front, + covered by the same repo-scoped forge token (CB-302). Git stays the only channel code moves + through. +4. **Asks are live-only, with expiry.** `ASK`/`ANSWER` (U2) traverse the broker as short-lived + (TTL'd) messages carrying the `turn_id`, and are never held durably — the single-host rule + kept. An answer arriving after its turn ended is **not** injected; it is dropped and the leader + gets a `TOO_LATE` notice, so the one failure case is loud rather than weird. Only terminal + replies are durable. Walkthrough: wiki 10 §7.4. +5. **Spawn dedup — a spawn id, remembered on the target.** The control queue redelivers like any + queue; a replayed `SpawnRequest` must not double-spawn. Requests carry a unique spawn id; the + target gateway keeps a short memory of handled ids and answers a redelivery with the existing + `PeerHandle`. CB-117's orphan reap stays as the backstop. +6. **Broker down — local unaffected, remote fails fast.** The routing fork (§3.2) means same-host + traffic never touches the broker; that is now a written promise. A send to a remote agent while + the broker is unreachable **fails immediately** with a clear error — the gateway never buffers + on the broker's behalf (it stays soft-state, so a crash cannot lose messages it claimed to + deliver). Gateways auto-reconnect; remote hosts read as unknown in the roster meanwhile. Broker + HA is a later ops choice, not a design requirement. + +## 8. Still open - **Directory ground-truth:** pure soft-state presence (heartbeats) vs. also treating broker queue existence as authoritative. Lean soft-state to preserve the persistence boundary; revisit if @@ -199,7 +238,9 @@ CB-308 doesn't have to repaint the topology. - **Global id scheme:** `/` (human-legible, leaks host) vs. opaque UUID (clean, needs the directory to resolve host). Probably UUID in the protocol, host as directory metadata. - **Gateway discovery:** how gateways find the broker and each other (static config vs. discovery). -- **Trust model shape:** per-host shared secret vs. mTLS on the broker vs. a capability token per - control action — ties into CB-401 Stage-C. -- **Failure semantics:** a host/gateway dies mid-turn — how the federated roster reaps it (missed - heartbeat) and whether in-flight primary-bound messages survive (broker durability = yes). +- **Control authorization:** signing (§7.1) settles *who sent it*; still open is *who may do + what* — which leaders may spawn or stop on which hosts (allowlist vs. a capability token per + control action — ties into CB-401 Stage-C). +- **Gateway death mid-turn:** the roster reaps it by missed heartbeat, and in-flight primary-bound + messages survive by broker durability; still open is reconciling *worker* state when the dead + gateway's host comes back (orphaned panes vs. still-valid sessions). diff --git a/wiki b/wiki index 0c896eb..36bb865 160000 --- a/wiki +++ b/wiki @@ -1 +1 @@ -Subproject commit 0c896eb49b042383d0de6535ffcb5344b1e11b5f +Subproject commit 36bb86588aa550e51c4227fc6edc1eebb1b62d6c