Merge #513: TIMED_OUT_QUEUED has four routes, and the javadoc now says which
The old javadoc said a TIMED_OUT_QUEUED message was "still sitting in the injector's
per-target queue". It is not — send() has already given up on it and it will never
arrive. That wrong claim told an operator to wait for a message that was never coming.
The first fix replaced it with a narrower wrong claim: that Injector.cancel() cancelled
the entry and "the target never saw a word of it". That describes one of four routes.
TIMED_OUT_QUEUED is returned whenever injector.cancel() returns anything but DELIVERED:
CANCELLED the Pending was still queued and this call removed it
NOT_DELIVERED Injector.java:400 - the herdr agent.prompt call threw
NOT_DELIVERED Injector.java:419 - readiness grace expired, never attempted
NOT_DELIVERED Injector.java:691 - drop(), the target is gone
On the last three, cancel() cancels nothing: it reads a state another path already set
(cancellationOf, Injector.java:288-290). And on the herdr-threw route, agent.prompt
pastes and submits in one call, so a throw does not prove the pane stayed clean - an
operator told "never saw a word" will not go and look at the one place the evidence is.
The javadoc now states the two facts that hold on every route - the message will not
arrive later, and it is not in any queue - and attaches "the target saw nothing" only to
the CANCELLED case. Five blocks: the enum constant, queuedDeliveries, hasQueuedDelivery,
hasOrphanedDelegation, and the inline comment in the TimeoutException branch that seeded
the wording.
Comment-only; no logic changed.
Verified on a tree merged with main (fast-forward to cb64bc8):
mvn install exit 0
1744 tests, 0 failures, 0 errors - Maven's own summary and an independent sum over
130 surefire report files agree
no unresolved javadoc reference on the three new links, with a positive control
showing javadoc did analyse MessageService.java
Closes #513.
This commit was merged in pull request #564.
This commit is contained in:
@@ -92,7 +92,19 @@ public final class MessageService {
|
|||||||
QUESTION,
|
QUESTION,
|
||||||
/** Timed out after the message was delivered — the worker is still working. */
|
/** Timed out after the message was delivered — the worker is still working. */
|
||||||
TIMED_OUT_WORKING,
|
TIMED_OUT_WORKING,
|
||||||
/** Timed out before delivery — the message is still queued for the worker. */
|
/**
|
||||||
|
* Timed out with no confirmed delivery. Despite the name, this does not mean the message
|
||||||
|
* is sitting in a queue — on every route it will not arrive later. {@link #send} reaches
|
||||||
|
* this outcome through {@link Injector#cancel}, whose result tells the routes apart:
|
||||||
|
* {@link Injector.Cancellation#CANCELLED} means the message was still queued and this call
|
||||||
|
* removed it, so the target saw nothing; {@link Injector.Cancellation#NOT_DELIVERED} means
|
||||||
|
* an earlier attempt already decided the message's fate — the injector's call to the
|
||||||
|
* target's terminal ({@link dev.ltms.fleet.herdr.AgentControl#send}) threw, or the queue
|
||||||
|
* was cleared because the target never became ready or was abandoned — and {@code cancel}
|
||||||
|
* is only reporting that pre-existing state. On the failed-attempt route the terminal may
|
||||||
|
* already hold a partial paste from before the call threw, so only the {@code CANCELLED}
|
||||||
|
* case establishes that the target saw nothing.
|
||||||
|
*/
|
||||||
TIMED_OUT_QUEUED,
|
TIMED_OUT_QUEUED,
|
||||||
/** Another send to this session was in flight for the whole window. */
|
/** Another send to this session was in flight for the whole window. */
|
||||||
BUSY,
|
BUSY,
|
||||||
@@ -299,12 +311,18 @@ public final class MessageService {
|
|||||||
*/
|
*/
|
||||||
private final ConcurrentHashMap<String, Boolean> strandedReplies = new ConcurrentHashMap<>();
|
private final ConcurrentHashMap<String, Boolean> strandedReplies = new ConcurrentHashMap<>();
|
||||||
/**
|
/**
|
||||||
* Targets whose last send timed out with {@link Outcome#TIMED_OUT_QUEUED} (CB-640) — the
|
* Targets whose last send timed out with {@link Outcome#TIMED_OUT_QUEUED} (CB-640) — {@link
|
||||||
* message never reached the {@link Injector} delivery window before the caller's deadline, so
|
* #send} called {@link Injector#cancel} and got back something other than {@code DELIVERED}.
|
||||||
* it is still sitting in the injector's own per-target queue. Set where {@link #send} already
|
* That covers more than one history: the message may have still been queued and {@code cancel}
|
||||||
* computes {@code wasDelivered} for that outcome; no new queue is kept here, only the fact.
|
* removed it right there, or an earlier attempt may already have failed (the call to the
|
||||||
* Cleared the same way as {@link #strandedReplies}: the next accepted delivery for the target
|
* target's terminal threw) or been abandoned (the target never became ready, or was torn
|
||||||
* ({@link #send} opening a fresh waiter) or a teardown ({@link #abandon}).
|
* down). What holds on every route: the message will not arrive later, and it is not sitting
|
||||||
|
* in any queue. What does NOT hold on every route: that the target saw nothing — a failed
|
||||||
|
* delivery attempt can leave a partial paste behind. Set where {@link #send} already computes
|
||||||
|
* {@code wasDelivered} for that outcome; no queue is kept here, only the fact that the send
|
||||||
|
* ended with no confirmed delivery. Cleared the same way as {@link #strandedReplies}: the next
|
||||||
|
* accepted delivery for the target ({@link #send} opening a fresh waiter) or a teardown
|
||||||
|
* ({@link #abandon}).
|
||||||
*/
|
*/
|
||||||
private final ConcurrentHashMap<String, Boolean> queuedDeliveries = new ConcurrentHashMap<>();
|
private final ConcurrentHashMap<String, Boolean> queuedDeliveries = new ConcurrentHashMap<>();
|
||||||
private final AtomicLong ticketSeq = new AtomicLong();
|
private final AtomicLong ticketSeq = new AtomicLong();
|
||||||
@@ -391,12 +409,17 @@ public final class MessageService {
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Read-only delegation fact for fleet views (CB-640): {@code target}'s last send timed out
|
* Read-only delegation fact for fleet views (CB-640): {@code target}'s last send timed out
|
||||||
* before the {@link Injector} ever delivered it — the caller saw
|
* with no confirmed delivery — the caller saw {@link Outcome#TIMED_OUT_QUEUED} (see the
|
||||||
* {@link Outcome#TIMED_OUT_QUEUED} (see the {@code TimeoutException} branch of {@link #send}),
|
* {@code TimeoutException} branch of {@link #send}). Despite the method's name, this is not
|
||||||
* and the message is still sitting in the injector's per-target queue waiting for the worker
|
* proof that a message is sitting in a queue: {@link Injector#cancel} reports this outcome
|
||||||
* to go idle. Distinct from {@link Outcome#TIMED_OUT_WORKING}, where delivery already happened
|
* either because the message was still queued and got removed right there, or because an
|
||||||
* and only the reply is outstanding. Cleared the next time this target's delivery is accepted
|
* earlier attempt already failed (the call to the target's terminal threw) or was abandoned
|
||||||
* or the target is abandoned — see {@link #queuedDeliveries}.
|
* (the target never became ready, or was torn down). Either way the message will not arrive
|
||||||
|
* later. It does NOT follow that the target saw nothing — on the failed-attempt route the
|
||||||
|
* terminal may already hold a partial paste. Distinct from {@link Outcome#TIMED_OUT_WORKING},
|
||||||
|
* where delivery already happened and only the reply is outstanding. Cleared the next time
|
||||||
|
* this target's delivery is accepted or the target is abandoned — see
|
||||||
|
* {@link #queuedDeliveries}.
|
||||||
*/
|
*/
|
||||||
public boolean hasQueuedDelivery(String target) {
|
public boolean hasQueuedDelivery(String target) {
|
||||||
return target != null && queuedDeliveries.containsKey(target);
|
return target != null && queuedDeliveries.containsKey(target);
|
||||||
@@ -416,15 +439,15 @@ public final class MessageService {
|
|||||||
/**
|
/**
|
||||||
* Read-only delegation fact for fleet views (CB-640): an async ticket is still
|
* Read-only delegation fact for fleet views (CB-640): an async ticket is still
|
||||||
* {@link Phase#PENDING} against {@code target}, yet nothing is actually in flight for it — no
|
* {@link Phase#PENDING} against {@code target}, yet nothing is actually in flight for it — no
|
||||||
* open rendezvous waiter ({@link #hasAcceptedDelivery}) and no message still sitting in the
|
* open rendezvous waiter ({@link #hasAcceptedDelivery}) and no record of a send that ended with
|
||||||
* injector's queue ({@link #hasQueuedDelivery}). A healthy PENDING ticket can briefly look this
|
* no confirmed delivery ({@link #hasQueuedDelivery}). A healthy PENDING ticket can briefly look
|
||||||
* way while its virtual thread has not yet been scheduled or is blocked on the session lock
|
* this way while its virtual thread has not yet been scheduled or is blocked on the session
|
||||||
* behind another send to the same target, so this is a snapshot fact for the health classifier
|
* lock behind another send to the same target, so this is a snapshot fact for the health
|
||||||
* to weigh across ticks, not proof on its own that the ticket is stuck. It also genuinely
|
* classifier to weigh across ticks, not proof on its own that the ticket is stuck. It also
|
||||||
* persists — not just as a passing race — once an async {@code fleet_ask} lapses unanswered:
|
* genuinely persists — not just as a passing race — once an async {@code fleet_ask} lapses
|
||||||
* {@link #ask} clears the ticket's question and returns it to {@code PENDING}, but {@link #send}
|
* unanswered: {@link #ask} clears the ticket's question and returns it to {@code PENDING}, but
|
||||||
* already closed the forward waiter the instant the question surfaced, so the target has
|
* {@link #send} already closed the forward waiter the instant the question surfaced, so the
|
||||||
* neither an accepted nor a queued delivery left to show for it.
|
* target has neither an accepted nor a queued delivery left to show for it.
|
||||||
*
|
*
|
||||||
* <p><strong>Deliberately still {@code question == null} only (fleetd #275).</strong> This
|
* <p><strong>Deliberately still {@code question == null} only (fleetd #275).</strong> This
|
||||||
* method must not also report a still-{@link Phase#ASKING} task as orphaned: the worker may
|
* method must not also report a still-{@link Phase#ASKING} task as orphaned: the worker may
|
||||||
@@ -952,7 +975,10 @@ public final class MessageService {
|
|||||||
log.debug("send to {} timed out (delivered={})", target, wasDelivered);
|
log.debug("send to {} timed out (delivered={})", target, wasDelivered);
|
||||||
if (!wasDelivered) {
|
if (!wasDelivered) {
|
||||||
// CB-640: record that delivery did not happen for fleet health (see
|
// CB-640: record that delivery did not happen for fleet health (see
|
||||||
// queuedDeliveries). The exact Pending was cancelled, so it cannot arrive later.
|
// queuedDeliveries). Whatever injector.cancel() reported above — this call
|
||||||
|
// removed a still-queued Pending, or an earlier attempt already failed or
|
||||||
|
// was abandoned — the send ends with no confirmed delivery and will not
|
||||||
|
// arrive later.
|
||||||
queuedDeliveries.put(target, Boolean.TRUE);
|
queuedDeliveries.put(target, Boolean.TRUE);
|
||||||
}
|
}
|
||||||
return recorded(new Reply(
|
return recorded(new Reply(
|
||||||
|
|||||||
Reference in New Issue
Block a user