Fix OTA retry accounting and radio delivery races

Keep catalog, manifest and leaf requests pending when the local transmit
queue rejects them instead of spending their network retry allowance.

Keep the packet selected before CAD so work becoming due during sensing
cannot bypass its own channel or pacing checks. Retire cancelled radio
retries before waiting for BUSY hardware or their retry deadline.

Wait for active transmission completion before applying firmware. Stop
admitting new transmissions after the apply drain deadline so busy queues
cannot postpone reboot indefinitely, and keep those deadlines armed across
millis rollover. Retain the existing persistence-flush backoff.

Add failing-before/fixed-after regression coverage for all four issues,
including the production OTA apply and TX guards in CI.

Validation: 1,496 native tests, 42 Python integration tests, and five
firmware builds passed across ESP32-S3, nRF52, and STM32 targets.
This commit is contained in:
mikecarper committed 2026-09-15 06:45:28 -07:00
1 parent b2f2b5bf02
commit 5d69aa461f
11 files changed
+312 -24

No files matched your search

+11 -1
View File
@@ -356,6 +356,12 @@ void Dispatcher::loop() {
// arriving during backoff before asking to retune; otherwise BUSY can
// keep the retry waiting forever on an unread RxDone interrupt.
if (isDualRadioActive()) checkRecv();
// A cancelled or expired packet must retire even if the chip never
// becomes available for another retune (including an armed OTA reboot).
if (!isPacketRadioCurrent(outbound) || !allowPacketTransmit(outbound)) {
failOutboundTransmit();
return;
}
if (!millisHasNowPassed(outbound_radio_retry_at)) return;
const auto prepared = _radio->prepareTransmitProfile(outbound->radio_profile,
outbound->radio_reply && outbound->radio_reply_force);
@@ -816,7 +822,11 @@ void Dispatcher::checkSend() {
cad_busy_start = 0; // reset busy state
profile_cad_busy[pending ? pending->radio_profile : 0] = 0;
outbound = _mgr->getNextOutbound(_ms->getMillis());
// Retuning and CAD can advance the clock. Keep the selection timestamp so
// a newly due higher-priority packet cannot skip its own profile, airtime,
// and OTA pacing checks. A manager may also expire/reorder work on peek.
if (pending && _mgr->peekNextOutbound(now) != pending) return;
outbound = _mgr->getNextOutbound(now);
if (outbound) {
outbound_radio_retry_pending = false;
outbound_radio_retry_used = false;