Features: #338 — a timed-out queued send cancels its message

Dai Ha
2026-09-04 16:01:54 +07:00
parent c7c142300c
commit 5a59747349
+28
@@ -4193,3 +4193,31 @@ test's own javadoc instead of quietly leaving it. Measured on 2026-09-04: deleti
comparison from `changedDeferredKeys` while leaving `guard` in the deferred set left all 1355 tests
green. Open as fleetd #337. Two tickets running, an honest caveat in a test's javadoc has been the
fastest route to the next bug — hold every checker here to that standard.
---
## A send that times out while still queued now cancels its message
**What it does.** `Injector.enqueue` returns an identity `Delivery` handle. When a send's deadline
passes and its message was never delivered, `MessageService` cancels that exact `Pending` instead of
leaving it in the queue. `queuedDeliveries` still records the health fact — that half is unchanged.
**On.** Always on (fleetd #338).
**Why it exists.** Before this, `TIMED_OUT_QUEUED` told the sender the message did not go, and then
the message went anyway. The injector picked it up on the member's next injectable status and typed
it in — minutes or hours later, after the lead had moved on and usually re-sent the work elsewhere.
Nobody was told. This is not a theoretical path: it is a behaviour hit while operating the fleet,
and a test had been asserting it as correct.
**One thing to know for maintenance.** Cancelling is the strict direction and it costs something: a
member that was about to go idle loses a message it could have taken, and the lead must send again.
That is the right trade because it is loud and recoverable, but it is a trade. If a queued message
starts disappearing more often than expected, this is why.
**The race is resolved deliberately.** Cancellation and `Injector.onStatus` share the target
monitor. If pickup wins, the text has already landed, `cancel` returns `DELIVERED`, and the result
is `TIMED_OUT_WORKING` — not `TIMED_OUT_QUEUED`. Claiming "queued" while the text landed would be
the original bug with a smaller window. Note that `MessageService` reading that return value is
**not** currently pinned by a test: `InjectorTest` covers `cancel` itself, and mutating the caller's
use of it left all 1357 tests green. Open as fleetd #345.