From 84c8a2d2f0759f2b1ea6b00f38c95ffb72524d55 Mon Sep 17 00:00:00 2001 From: Dai Ha Date: Tue, 28 Jul 2026 16:26:10 +0200 Subject: [PATCH] =?UTF-8?q?CB-500=20=C2=A711:=20resolve=20distributed-sand?= =?UTF-8?q?box=20topology=20(gateway-per-host=20=C3=97=20local=20sandboxes?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Clarifies the open fork from §4/§9: Development A (sandbox launcher) and CB-308 (per-host federation) COMPOSE — each host runs a bridged gateway whose launcher spawns agents into that host's LOCAL sandboxes; the broker moves messages + presence, never keystrokes. The forcing fact: delivery is herdr keystroke-injection into a locally-owned PTY, so "a sandboxed agent on another host" ≡ "a sandbox spawned by that host's gateway" (a remote container with no local herdr can't be injected into). Rules out a central daemon reaching remote PTYs. Adds Figure 11 (composed topology) + Figure 12 (remote-delegation sequence: the local?inject:publish fork with a sandboxed far side — both injection points stay local, only the middle hop crosses the broker), the two forced reachability changes (host-routable mcpUrl; PTY in the local gateway's herdr), and a maps-to-existing-seams table (CB-308 gateway × Dev-A launcher, CB-117 reap, CB-303 container lifecycle, CB-308 #5 trust). No new pillars. Both diagrams mmdc-validated; §4/§9 updated to point at §11. --- docs/CB-500-Multi-Tier-Coordination.md | 109 ++++++++++++++++++++++++- 1 file changed, 106 insertions(+), 3 deletions(-) diff --git a/docs/CB-500-Multi-Tier-Coordination.md b/docs/CB-500-Multi-Tier-Coordination.md index c9cd5e9..78827ac 100644 --- a/docs/CB-500-Multi-Tier-Coordination.md +++ b/docs/CB-500-Multi-Tier-Coordination.md @@ -160,7 +160,9 @@ CB-306 readiness gate); only the launcher's `buildLaunch` differs — exactly th **Deltas:** a new `kind: sandbox` adapter (extends the same `HerdrPeerLauncher`/`PeerLauncher` base); a spawn-target/role dimension on `SpawnRequest` and the `Worker` profile; optionally a `SANDBOX` `Capability`. Per-role = two profiles → two images; `CompositePeerLauncher` already routes them. If a -sandbox is a *separate host/container*, it reuses CB-308's global id + per-host gateway wholesale. +sandbox is a *separate host/container*, it reuses CB-308's global id + per-host gateway wholesale — +**the distributed case is resolved in §11: a sandbox on another host is one spawned by that host's +gateway, because herdr keystroke-injection needs a locally-owned PTY.** ## 5. Development B — Main-agent pairs @@ -336,8 +338,9 @@ flowchart LR ## 9. Open questions (to resolve at ticket-split) -- **Sandbox mechanism:** container (`docker exec`) vs devcontainer vs a remote host (then it *is* - CB-308). How the role→image mapping is expressed on the profile. +- **Sandbox mechanism:** container (`docker exec`) vs devcontainer — how the role→image mapping is + expressed on the profile. *(Topology **resolved** in §11: distributed = gateway-per-host × local + sandboxes; the remaining choice is only the local launch mechanism, not the shape.)* - **Pair semantics:** are the two mains fully symmetric peers, or is one a co-primary that may also delegate? Affects how `PrimaryRegistry` and the "orchestration tools" identity relax. - **Orchestrator drivenness:** the mains become programmatically spawned/resumed — does the human @@ -355,3 +358,103 @@ role/spawn-target), **the CB-308 substrate** (likely already its own ticket), ** pair), and **C** (orchestrator tier + context scoping) — with the identity clause (§7) as an acceptance criterion on **A** specifically. Sequence per §8; nothing here is a new pillar, so each ticket is an extension of an existing pattern (CB-402 for A, CB-308 for B/C). + +## 11. Distributed sandboxes — the resolved topology + +The follow-up question — *"clarify the architecture when we have distributed agents in sandboxes"* — +resolves the fork left open in §4 and §9. **Decision: Development A (sandbox launcher) and CB-308 +(per-host federation) *compose*, not compete — each host runs a `bridged` gateway whose launcher +spawns agents into that host's *local* sandboxes.** A sandbox is never reached across the network; it +is reached by the gateway sitting next to it. + +### 11.1 The one fact that fixes the shape + +The bus delivers a turn by **herdr keystroke-injection** — `Injector → AgentControl.send` writes into +a PTY that its **local** herdr owns. The broker moves *messages and presence*, **never keystrokes**. +So an agent's PTY must live in a herdr that *some* `bridged` instance drives locally: a remote +container with no local herdr **cannot be injected into**. That rules out a central daemon reaching +remote PTYs, and collapses the design to a single identity: + +> **"a sandboxed agent on another host" ≡ "a sandbox spawned by that host's gateway."** + +```mermaid +flowchart TB + subgraph hostA["HOST A — gateway"] + mA["main / orchestrator
MCP client → LOCAL gateway"] + gA["bridged A
herdr + CompositePeerLauncher
(incl. SandboxLauncher)"] + cBEa["sandbox: backend
(local container)"] + cFEa["sandbox: frontend
(local container)"] + mA --- gA + gA -->|"spawn (docker/devcontainer)
→ PTY in A's herdr"| cBEa + gA --> cFEa + end + subgraph broker["BROKER (AMQP) — CB-307/308 fabric"] + inbox["agent.ID.inbox queues"] + roster["roster.* (federated presence)"] + end + subgraph hostB["HOST B — gateway"] + gB["bridged B
herdr + SandboxLauncher"] + cBEb["sandbox: backend
(local container)"] + gB -->|"spawn → PTY in B's herdr"| cBEb + end + gA <-->|"messages + presence
(NOT keystrokes)"| inbox + gB <-->|"messages + presence"| inbox + gA --- roster + gB --- roster + classDef line fill:#2b6cb0,stroke:#1a365d,color:#ffffff; + class inbox,roster line +``` + +*Figure 11 — the composed topology. Each gateway owns its local herdr and runs a `SandboxLauncher` +(the §4 adapter) that spawns role-specific containers **on its own host**; the broker (blue) carries +only messages + roster between gateways. Keystroke-injection stays strictly local to each gateway.* + +### 11.2 How a delegation reaches a sandboxed agent on another host + +```mermaid +sequenceDiagram + participant MA as main (host A) + participant GA as gateway A + participant BR as broker + participant GB as gateway B + participant SB as sandbox agent (host B, container) + MA->>GA: bridge_send(globalId on B, msg) + GA->>GA: directory lookup - is globalId local? NO + GA->>BR: publish agent.ID.inbox (durable) + BR->>GB: route to the owning gateway + GB->>SB: inject via B's LOCAL herdr (keystrokes) + Note over GB,SB: SandboxLauncher already spawned the container -
its PTY is in B's herdr, CB-306 readiness passed + SB-->>GB: bridge_reply (to B's LOCAL MCP endpoint) + GB->>BR: publish primary-bound (durable, msg id) + BR->>GA: route back to A + Note over GA: held until the main pulls (the main is a client) + MA->>GA: poll / blocking send resolves + GA-->>MA: reply +``` + +*Figure 12 — the `local ? inject : publish` fork (CB-308 §3.2) with a sandboxed far side. Only the +**middle** hop crosses the network via the broker; **both** injection points (into the sandbox on B, +and the drain-nudge back into the main on A) are local herdr writes. This is CB-308's routing rule +unchanged — the sandbox is transparent to it.* + +### 11.3 Two reachability changes any sandbox forces + +| Change | Today | Under sandboxes | +|---|---|---| +| **`mcpUrl`** | `http://127.0.0.1:8765/mcp` (loopback) | must be **host-routable from inside the container** (e.g. `host.docker.internal` or the gateway's LAN IP) — the worker connects to **its own gateway's** MCP, never a remote one. | +| **PTY ownership** | pane in the daemon's herdr | pane is the **container's** attached PTY, in the **local** gateway's herdr (via `docker exec`/devcontainer) — non-negotiable per §11.1. | + +### 11.4 Everything maps to an existing seam (nothing new invented) + +| Concern | Provided by | +|---|---| +| Per-host gateway (owns local herdr + sessions) | **CB-308** (today's `bridged`, evolved) | +| Spawn into a local sandbox / role→image | **Development A** `SandboxLauncher` (§4), routed by `CompositePeerLauncher` | +| Addressing a remote sandboxed agent | **CB-308** global id + federated roster (host + role as metadata) | +| Orphan reap after a gateway restart | **CB-117** per-gateway, summed by the composite — each reaps only its **local** herdr | +| Container up/down | tied to **CB-303** session lifecycle — `SandboxLauncher.stop` tears the container down with the pane | +| Cross-gateway spawn/send trust | **CB-308 item #5** / CB-401 Stage-C — each gateway edge is a trust boundary | + +*The net: distributed sandboxes add **zero** new pillars — they are `CB-308 gateway × Development-A +launcher` at every host, with the §7 ownership line (peer owns the image; the bridge only launches +into it) holding at each gateway.*