A neighbour heard before the clock is set carries the firmware's unset-clock
default (1715770351, 15 May 2024). Subtracting that from an NTP-synced clock
published ages of ~806 days for neighbours that had just answered a live scope
query, and a backwards clock step reported 0, i.e. "heard just now".
- finishNeighborDiscover() reports the age as unknown ("heard_secs_ago": null)
when the stored stamp and the current clock come from different epochs, or
when the clock has stepped backwards
- handleNeighborDiscoverResponse() re-stamps heard_timestamp on a zero-hop scope
reply, in both the snapshot and the live table, so entries heal once per
discovery cycle instead of waiting for the neighbour's next advert
- publish ordering places usable ages ahead of unknown ones so a poisoned stamp
cannot displace fresh entries when the JSON buffer truncates
- document the null case, and the always-present total_neighbors /
queried_neighbors / truncated fields the payload sample omitted
- add UPSTREAM_BUGS.md, tracking the monotonic-uptime fix to propose upstream
plus the unclamped subtraction in the companion and CLI readouts
Containment only: upstream still stamps neighbours from the wall clock at
packet-reception time, which on a cold boot always precedes NTP.
Absorbs 106 upstream commits. Seven files conflicted; the substantive one
was upstream's new JSON ConfigSerializer (PR #2982), which replaces the
binary /com_prefs layout with /prefs.json and makes NodePrefs a
ConfigSerializer subclass.
Prefs migration
- Adopt upstream's ConfigSerializer. writeCommonPrefsImage() and its
documented offsets (0-294) are deleted, along with the now-unreachable
saveCommonPrefsImageAtomically()/CommonPrefsFileStore atomic rename path.
- Load order is /prefs.json, then /com_prefs, then /node_prefs. Upstream
dropped the /node_prefs fallback; it is restored here so devices that
never advanced past that filename keep their config.
- Legacy files are never removed, so migration cannot destroy its own
source and a deferred or failed save simply retries next boot.
- /com_prefs is treated as a format migration only, not an "upgrade", so
it does not trip the bridge.source tx->rx flip on existing nodes.
- The MQTTPrefsAtomicStore legacy gate is retained: the observer tail
recovered from an old-format file still commits to /mqtt_prefs before
/prefs.json is written.
- MQTTPrefs and /mqtt_prefs are untouched; savePrefs keeps its save_mqtt
parameter and now returns upstream's bool.
Fixes to upstream code
- RadioPrefs::structure() bound both "rxgain" and "fem_rxgain" to
rx_boosted_gain, so radio_fem_rxgain was never persisted. Bound to the
correct field.
- discovery_mod_timestamp was dropped from structure(); it gates
'since'-filtered DISCOVER replies and is set on every config change, so
losing it would silently stop discovery responses after a reboot. Added
as "disc_mod".
Merge artifacts repaired
- Restored bblanchon/ArduinoJson to the native test env; a clean but wrong
auto-merge at the lib_deps block boundary dropped it and broke all 19
host test suites.
- Migrated the fork's WebConfig UITask screens off the removed
DisplayDriver::Color enum to upstream's UIColor element types.
- Removed duplicate getCADEnabled() definitions in companion MyMesh.cpp
and simple_sensor SensorMesh.cpp that both sides had added.
- Dropped memset(&_prefs, 0, ...) in the four example meshes; NodePrefs now
has a vtable. guard gains an initializer that memset used to provide.
Other resolutions
- simple_room_server keeps both the fork's discover.* commands and
upstream's new room.post.
- docs/payloads.md taken from upstream, undoing content earlier merges had
reverted (Control data section, split login tables).
Verified: 273/273 host tests pass across native and native_kiss_modem;
Heltec v3 repeater, repeater_observer_mqtt, room_server_observer_mqtt,
sensor, and companion_radio_ble all build clean.
Enhance the neighbor discovery JSON structure by introducing a
default_scope field, which indicates the region name this node
floods to by default. This change improves clarity in the
neighbor discovery process and aligns with the unscoped flood
behavior when no default region is set. Updates include
modifications to the MyMesh class and related message building
functions to accommodate the new field.
Add support for named packet types in per-slot filters, allowing users
to specify packet types using descriptive names alongside numeric values.
This improves usability and clarity in configuring MQTT slot filters.
Updates include modifications to the parsing logic, WebConfig interface,
and related documentation to reflect the new naming conventions.
Introduce new methods to manage neighbor discovery JSON budget and
entries in MyMesh. This includes tracking the number of queried and
published neighbors, measuring JSON sizes, and handling truncation
when the buffer limit is reached. These improvements optimize the
neighbor discovery process and ensure efficient JSON message
construction for MQTT communications.
Add functionality to manage neighbor discovery requests in MyMesh.
This includes handling the status of requests, updating the state
upon transmission success or failure, and implementing a timeout
mechanism for neighbor discovery queries. The changes improve the
efficiency and reliability of neighbor discovery operations in
MQTT-enabled environments.
Improve the response message for the `alert test` command to clarify
that a successful test send does not guarantee automatic alerts are
enabled. This change helps prevent confusion for operators regarding
the status of the alert system after testing.
Port the neighbor-discovery state machine from mqtt-bridge-implementation-flex,
adapted to this branch's MyMesh (WebConfig members shifted the insertion points;
applied by content).
- Two-stage periodic refresh in loop() driven by mqtt_neighbors_interval:
stage 1 is a zero-hop sendNodeDiscoverReq() (reuses the existing 60s window),
stage 2 (startNeighborDiscover) fires one anon-regions scope query per heard
neighbour, then finishNeighborDiscover() builds the table JSON via
MQTTMessageBuilder::buildNeighborsMessage and hands it to
bridge->requestPublishNeighbors().
- Peer overlay at NEIGHBOR_DISCOVER_PEER_BASE lets scope-query RESPONSE packets
from non-ACL neighbours decrypt: searchPeersByHash prepends heard neighbours
(bounded by MAX_CLIENTS), getPeerSharedSecret derives the secret on the fly,
and onPeerDataRecv routes both overlay-index and ACL-client-that-is-a-neighbour
responses into handleNeighborDiscoverResponse.
- Entries ordered most- to least-useful (recent, then stronger SNR) so the JSON
builder's tail-drop keeps the useful head.
- `discover.scopes` CLI command (manual trigger), with a WITH_MQTT_BRIDGE stub
replying "requires PSRAM" on non-PSRAM builds.
- Reports schedule to the bridge each loop via setNeighborsSchedule().
- Uses ArduinoJson v7 JsonDocument (not deprecated DynamicJsonDocument).
All gated on WITH_MQTT_NEIGHBORS. Reuses the existing MQTTBridge* member.
Verified: T_Beam_S3_Supreme_SX1262_repeater_observer_mqtt builds [SUCCESS].
First upstream merge since the 2026-06-06 base (191 upstream commits). 14 files
conflicted; resolutions below.
Fleet-critical check (Constraint 1): upstream reordered NodePrefs members
(rx_boosted_gain / path_hash_mode moved to the struct tail) but did NOT change
/com_prefs. Persistence is written field-by-field at explicit offsets, so member
order is in-memory only. Verified the fork's writeCommonPrefsImage() is
byte-identical to upstream's inline writer at every offset (79 pad, 121, 122,
290-294). No migration needed.
Resolutions:
- CommonCLI.h: kept the fork's NodePrefs (superset) and adopted upstream's
setRxBoostedGain(bool)->bool signature change, which CommonCLI.cpp now uses to
report unsupported. Corrected a stale comment claiming rx_boosted_gain lives at
offset 79 (it is a pad; the field is at 290).
- CommonCLI.cpp: kept the fork's legacy /com_prefs migration and the extracted
writeCommonPrefsImage() call.
- UITask.cpp: three-way merge - upstream's drawTextCentered + powering-off
screen, plus the fork's WITH_WEBCONFIG portal/reboot screens.
- ESP32Board.cpp, MeshCore.h, platformio.ini: kept both sides (fork OTA additions
alongside upstream powerOff/enterDeepSleep and Packet.cpp).
- MicroNMEALocationProvider.h: took upstream's claim/release and added the
_claims member they depend on.
- MyMesh.cpp/.h (repeater + room server): kept the fork's superset defaults.
- Removed duplicate declarations auto-merge produced: RadioLibWrapper::_cad_enabled
and MyMesh::getCADEnabled().
Verification: native suite 15/15 (incl. upstream's new test_mesh_tables), both
MQTT smoke builds green, ArduinoJson pin check passes. Hardware validation next.
An operator who triggers `ota update` via remote management only sees the
command's reply; the actual OTA runs ~2.5s later and reboots on success,
so the start/fail notifications land outside the reply window. Mirror the
key OTA milestones onto the configured alert channel (in addition to the
Serial log):
- START ("OTA update starting") from beginDeferredOtaUpdate(), i.e. while
the loop still runs -- a flood queued at the deferred fire could never
transmit before the flash blocks the loop / reboots on success.
- FAIL ("OTA aborted: ...") at both abort points (teardown barrier
withheld flashing; preflight/download error).
Success has no message: a successful flash reboots into the new image, so
the node returning on the new version is the signal.
New MyMesh::otaAlert() gates on the `alert on/off` master switch and rides
the configured alert scope (AlertReporter::sendText -> sendChannel ->
resolveAlertScope); no-op when alerts are off or no channel is set. Only
these start/fail milestones -- routine slot connect/disconnect is
unaffected (stays in AlertReporter's fault logic). Documented in ALERTS.md.
Both observer firmwares build.
Wire the Phase 4 MQTTLifecycle state machine into MQTTBridge to replace the
blind vTaskDelete teardown that could kill the MQTT task mid-mbedTLS and then
free client buffers on a corrupted heap (the observed OTA teardown panic).
- MQTTBridge owns a MQTTLifecycle::Coordinator driven only by the loop task
(Core 1) from begin()/end(); a nested LifecycleOps binds the host-tested Ops
spec to FreeRTOS/PsychicMqttClient.
- end() requests a cooperative stop; the MQTT task (Core 0) tears down its own
clients where the mbedTLS contexts live, acks via _stop_acked, and
self-terminates. end() waits (bounded) for the ack, then frees queue/buffers.
- Bounded stop timeout -> reviewed fallback (force kill + Core-1 teardown) sets
a dirty latch that withholds OTA flashing.
- begin() gains an idempotent double-call guard and syncs the Coordinator to
Running.
- OTA teardown barrier: simple_repeater's deferred flash aborts/resumes unless
end() reported a clean stop (canFlashAfterStop()).
Scope: minimal cooperative-shutdown unit. The volatile NTP/reconfigure handshake
replacement and the plain-data snapshot / consumer repointing (MQTT_OWNERSHIP.md
sections 1-3) are deferred. MQTT_STOP_TIMEOUT_MS is a Phase-0 placeholder pending
on-hardware characterization.
Native suite green (incl. test_mqtt_lifecycle); both observer firmware smoke
builds compile. Not yet hardware-validated (Phase 7 gate).
Include detailed instructions for local testing of observer and WiFi
functionality without hardware. Document the use of a mock backend and
Wokwi ESP32-S3 simulation for easier development and testing.
Enhance the MQTT implementation documentation to improve developer
experience and facilitate testing workflows.
agessaman
mikecarper
Jarosław Domański
Scott Powell