Expose the Full Companion terminal over TCP and add an authenticated, replay-resistant host command service with documented CPU temperature, reboot, and allowlisted program examples.\n\nImprove LoRa OTA diagnostics and persistent CLI handling, reduce clock correction drift to ten minutes, and make mOTA stream flush behavior an explicit transport policy with regression coverage.
ESP-IDF stores the mbedTLS stack error as a positive magnitude (it captures
-ret), so negating it before printing produced "mbedtls:-0xFFFF8100" instead
of "mbedtls:-0x7F00" for the record-buffer allocation failure. %04X is a
minimum width, so nothing masked it.
Normalisation moves into MQTTReplyFormat.h as mbedtlsErrorMagnitude() rather
than staying inline in the bridge: inline is why this survived, since the
existing test passes the already-correct magnitude straight into replyAppendf
and never exercised the caller. It accepts either sign so a later SDK storing
the real negative code still renders, and widens to int64_t before negating
because negating INT32_MIN is undefined behaviour.
MQTTReplyFormat.h also gains the stdint.h it was always missing: it compiled
only because MQTTBridge.cpp pulls stdint in via other headers, and the host
test includes the header standalone.
Cleanup before the commit phase ignored whether the temp was actually
removed. On a fresh install a complete, byte-verified temp that failed
schema verification (or whose read-back failed in finish()) could survive
a failed remove() with no primary to outrank it, and boot recovery then
promoted the very value the CLI had just reported as rolled back.
Both cleanups now return a disposition: success means the temp is gone or
an existing primary is authoritative. A false disposition maps to the new
CleanupIndeterminate, which latches the same indeterminate reply as a
commit that could not be rolled back. A short write is excluded, since it
leaves structurally incomplete JSON that recovery classifies as invalid.
Recovery also stops spending transaction state on an opaque backup: an
uncertain temp beside a FutureUsable or uncertain backup now holds both
names and runs defaults instead of promoting the candidate into the
authoritative name, where the "any primary owns the name" rule would keep
it even after a later boot proved it corrupt.
Recovery preserved a FutureClaimed or Indeterminate temp, but published the
backup into the primary name to run that boot. That spent the one piece of
state saying the candidate had already passed the backup rename: the next
boot saw an ordinary usable primary beside a stray temp, and deleted the
temp precisely when more heap or newer firmware finally made it readable.
The OOM path needed no future firmware to hit it — power cut after the
backup rename, one boot short of classification scratch, and a verified
new image was gone.
Answer an uncertain temp with UseBackupHeld instead: rename nothing, read
the last committed image straight out of /mqtt.json.bak, and hold writes.
The filenames then still describe the interrupted transaction, so a later
boot promotes the candidate through the ordinary temp rule, or falls back
to the backup once the candidate proves definitively corrupt.
Tests: two-boot sequences for Indeterminate and FutureClaimed candidates
that later classify as Usable or FutureUsable, the invalid-candidate
fallback, and the no-usable-backup case where the candidate still takes
the authoritative name.
Publishing moves the old primary to .bak before the verified temp takes
that name, so a failed second rename left the new image exactly where
boot recovery promotes it — while the observer setter told the operator
the change had been rolled back. The refused value came back at the next
reset.
Restore the backup and discard the temp on that path, and distinguish
CommitIndeterminate from CommitFailed when the filesystem cannot be put
back, so the CLI reply says the flash state is unresolved rather than
claiming the change is gone. The indeterminate condition latches for the
boot: the artifact left behind also makes every later transaction fail to
begin, so it cannot clear itself.
Also state the version-first rule the future-version probe depends on.
The probe reads the root version with this firmware's grammar, so a newer
file that introduces unknown syntax ahead of that field reads as corrupt
rather than future and loses its preservation guarantee.
Tests: publish-failure rollback and the indeterminate outcome against a
SPIFFS-shaped store fake; the version-first writer invariant and the cost
of violating it; /prefs.json coverage for the strict shape checks
(deployed-shape file, unknown nested groups, torn files, mismatches).
The usable-clock fallback asked libc only, which does not answer for the case it
was written to cover. On a cold boot ESP32RTCClock::begin() stamps libc with a
2024 placeholder on power-on; AutoDiscoverRTCClock::begin() probes the chip but
never copies its time across, and getCurrentTime() reads the chip directly. So a
Station G3 or T-Beam Supreme that knows exactly what time it is, on a network
with UDP/123 blocked, still failed the plausibility test, left _ntp_synced false,
and brought up no slots — precisely the deployment the fallback exists for.
Ask the RTC when libc is below the floor. libc still wins when it is usable: a
clock SNTP set recently outranks a chip that may have drifted. Accepting the RTC
value then flows through the same block, so settimeofday() repairs libc and the
epoch is written back to the chip.
The choice is chooseFallbackClock() in MQTTConnectionPolicy, host-tested across
the four states including the power-on placeholder and the exact floor. Also
corrects the previous commit's claim that configTime() is called only when a
server replied — the fallback necessarily points it at each server before knowing
that; it is the post-acceptance call that is now conditional.
The corrected-clock path reconnected a disconnected slot whether or not
createSlotAuthToken() had produced anything, which re-presented the credentials
the correction had just invalidated. Minting fails for recoverable reasons —
allocation pressure is treated as recoverable elsewhere in this file — so the
path is reachable, and the reconnect it spends is one that cannot succeed.
Move the decision into MQTTConnectionPolicy as classifyStaleToken(), where the
four outcomes are named and host-tested rather than spelled out in nested
conditions: Defer on a failed mint, Reconnect a client that is down, Bounce a
live session whose broker enforces exp, KeepAlive one whose broker does not.
Deferring leaves the slot to the backoff ladder, which mints again on its next
attempt.
Covers the reviewer's first four cases. The other two — that a completed SNTP
sync is required, and that time(nullptr) reflects the accepted epoch before
_ntp_synced flips — are inside MQTTBridge.cpp, which the native env does not
compile; locking those down needs a seam around the IDF calls that does not
exist yet.
Every ordinary backoff reconnect and every circuit-breaker probe minted a
fresh JWT and re-applied credentials, with no check of whether the existing
token was still valid. setCredentials() always dirties the esp-mqtt config, so
reconnect() then called esp_mqtt_set_config() as well. On a flapping broker
that is a signing plus a configuration-copy cycle on every retry, and these
observers see ~38 genuine reconnects/day per slot.
The no-bounce renewal change (27bd05a1) only stopped the proactive renewal
from tearing down a live session; it left this retry path untouched, which is
why a soak shows renewals neither firing nor failing for hours while drops
continue — each reconnect silently re-mints and pushes the expiry out.
Reuse the credentials when their validity is provable and refresh them
otherwise. canReuseJwtForReconnect() lives with the other policy predicates so
it is host-testable, and it establishes current_time < token_expires_at before
subtracting: token_expires_at is unsigned, so an already-expired token would
otherwise wrap to ~4e9 seconds and read as valid for decades. The
>= kMinimumValidEpoch term also rejects the 0 that a failed renewal writes.
Minting stays the default for every uncertain case — unsynced clock, missing or
insane expiry, empty token, or an expiry inside kJwtReconnectSafetyMarginSecs
(60 s), which covers the handshake itself.
Two paths still always mint, deliberately:
- The circuit-breaker probe. It is the recovery of last resort for a slot
that has already failed repeatedly, quite possibly on auth, and it runs
once per 30 minutes — so a fresh token there costs nothing worth counting
against keeping that path guaranteed-clean.
- Any slot whose last error was a broker refusal. Before this change, minting
on every retry accidentally recovered from server-side credential
invalidation: key rotation, revocation, broker clock skew, or an audience
change after a reconfigure. Reuse would have retried a rejected credential
until it neared expiry — up to 24 h for every preset that leaves
token_lifetime at the default. onError already detects
MQTT_ERROR_TYPE_CONNECTION_REFUSED and only logged it; it now also sets a
per-slot force-mint flag, cleared on a successful connect and wherever the
credentials it referred to are blanked. The flag is volatile because the
esp-mqtt callback sets it and the bridge loop consumes it.
The reconnect log line reports the decision and its outcome — REUSE, MINT with
a reason, and OK/FAILED for the mint — because a silently failed mint is the
case most likely to end in an auth refusal. It never prints the token.
Host tests cover the reuse boundary: exact margin, already-expired, expiry 0,
sub-epoch expiry, empty token, unsynced clock, and the force-mint override.
The real Arduino.h includes stdlib.h, so ConfigSerializer.cpp reaches atoi,
atol and atof through it and compiles on device. The mock supplied only
cstdint, cmath and Stream.h, leaving those undeclared — and since the native
env compiles ConfigSerializer.cpp into every suite via build_src_filter, all
21 suites errored rather than just its own.
Fixing the mock keeps src/ identical to upstream and covers any other source
relying on the same transitive include.
pio test -e native: 297 test cases, 297 succeeded.
Picks up upstream MeshCore 1.17.1.
Notable upstream content:
- 1.17.1 version/build-date bump in the example MyMesh headers.
- nRF52: combine radio entropy with CC310 RNG.
- Companion FEM prefs: load/save of fem_ properties commented out until they
can be set from the client.
- Scoped reply routing: replies no longer dropped when flood.max.unscoped is
low (RoutingPolicy + unit tests).
- nRF52 unused-pin sweep (T1, T-Echo Lite, MeshPocket).
No conflicts.