ArduinoJson v7 hands out document-pool blocks in fixed 4096-byte chunks, so
the pool is not bounded by the size of the text it serialises to. Budgeting
it at NEIGHBORS_JSON_BUFFER_SIZE starved it once the table grew: a 50-entry
table needs 12541 B of pool against the 10240 B cap, and a starved allocator
sets doc.overflowed(), which makes buildNeighborsMessage return 0 and drop
the entire publish rather than truncating the tail.
Repeaters with roughly 40 or more neighbours therefore published no neighbors
message at all, silently, while smaller tables published normally.
Give the pool its own NEIGHBORS_DOC_POOL_BUDGET and add an explicit
NEIGHBORS_MAX_PUBLISH_ENTRIES cap alongside the existing text-size check.
Board manifest for esp32-c6-devkitm-1 defaults build.flash_mode to qio,
which this module's flash chip does not support -- causes a boot
crash-loop (repeated USB-Serial-JTAG reconnects) on real hardware.
Override with board_build.flash_mode = dio in the common M5Stack_Unit_C6L
section so it applies to all envs (ble/usb/repeater/room_server).
Adds ThinkNode_M7_repeater_observer_mqtt and
ThinkNode_M7_room_server_observer_mqtt, following the Station G3 observer
overlay: adafruit-full cert bundle, MQTT bridge + SNMP, pinned observer
lib_deps, and the quieter default logging profile.
WiFi-only for now. The M7 has an onboard CH390 Ethernet controller (used by
ThinkNode_M7_companion_radio_ethernet, and an lwIP netif so the MQTT data path
would work over it), but the bridge's link management is bound to the WiFi
station API, so Ethernet cannot carry MQTT yet.
The board has PSRAM, so MAX_NEIGHBOURS=50 enables WITH_MQTT_NEIGHBORS —
verified present in both images. Builds use 46.5% of the 3.19 MB app slot at
13.7% RAM, and build.sh emits the full artifact set (.bin, -merged.bin,
.partsig). The partition table is inherited from [ThinkNode_M7] unchanged, so
no merged first flash is needed; its signature is byte-identical to the T-Beam
Supreme observer builds (both default_8MB.csv).
Also fixes two pre-existing build failures on this board:
- [ThinkNode_M7] re-added a blanket helpers/*.cpp after arduino_base excludes
the MQTT-only sources, so every M7 env failed on a missing Timezone.h. The
glob was otherwise fully redundant with the base, so it is dropped rather
than re-excluding the two files.
- ThinkNode_M7_companion_radio_ethernet sets DISPLAY_CLASS=NullDisplayDriver
but never compiled NullDisplayDriver.cpp, which defines UIColor::window_bkg,
so it failed at link. The ble and wifi envs both already list it.
All nine M7 envs now build. Observer env discovery picks up both new envs, and
the ArduinoJson pin check covers both new sections.
A neighbour heard before the clock is set carries the firmware's unset-clock
default (1715770351, 15 May 2024). Subtracting that from an NTP-synced clock
published ages of ~806 days for neighbours that had just answered a live scope
query, and a backwards clock step reported 0, i.e. "heard just now".
- finishNeighborDiscover() reports the age as unknown ("heard_secs_ago": null)
when the stored stamp and the current clock come from different epochs, or
when the clock has stepped backwards
- handleNeighborDiscoverResponse() re-stamps heard_timestamp on a zero-hop scope
reply, in both the snapshot and the live table, so entries heal once per
discovery cycle instead of waiting for the neighbour's next advert
- publish ordering places usable ages ahead of unknown ones so a poisoned stamp
cannot displace fresh entries when the JSON buffer truncates
- document the null case, and the always-present total_neighbors /
queried_neighbors / truncated fields the payload sample omitted
- add UPSTREAM_BUGS.md, tracking the monotonic-uptime fix to propose upstream
plus the unclamped subtraction in the companion and CLI readouts
Containment only: upstream still stamps neighbours from the wall clock at
packet-reception time, which on a cold boot always precedes NTP.
The [Xiao_S3_WIO] base block lost its sensor wiring when upstream
3dd6dc02 ("xiao_s3: use environment sensor manager and add sensor
role") was resolved to our side in a later merge. Only the .ini half
was dropped -- target.h/target.cpp kept the EnvironmentSensorManager,
so builds still compiled helpers/sensors with an empty SENSOR_TABLE:
no ENV_INCLUDE_* macros meant the table held only its sentinel, so
begin() scanned the I2C bus and initialized nothing and querySensors()
emitted nothing above channel 1.
Reported as an INA226 missing from channel 2 on a MQTT Observer build,
working on stock firmware. Affected every Xiao_S3_WIO env, including
the dedicated Xiao_S3_WIO_sensor role.
Restores sensor_base build_flags/lib_deps, -UENV_INCLUDE_GPS and the
PIN_BOARD_SDA/SCL defines, making the base block identical to upstream
again. Also picks up the commented rs232 pin relocation from the same
upstream commit, since 5,6 is now the I2C bus.
Flash on Xiao_S3_WIO_repeater_observer_mqtt: 42.7% -> 44.3%.
The [Xiao_S3_WIO] base block lost its sensor wiring when upstream
3dd6dc02 ("xiao_s3: use environment sensor manager and add sensor
role") was resolved to our side in a later merge. Only the .ini half
was dropped -- target.h/target.cpp kept the EnvironmentSensorManager,
so builds still compiled helpers/sensors with an empty SENSOR_TABLE:
no ENV_INCLUDE_* macros meant the table held only its sentinel, so
begin() scanned the I2C bus and initialized nothing and querySensors()
emitted nothing above channel 1.
Reported as an INA226 missing from channel 2 on a MQTT Observer build,
working on stock firmware. Affected every Xiao_S3_WIO env, including
the dedicated Xiao_S3_WIO_sensor role.
Restores sensor_base build_flags/lib_deps, -UENV_INCLUDE_GPS and the
PIN_BOARD_SDA/SCL defines, making the base block identical to upstream
again. Also picks up the commented rs232 pin relocation from the same
upstream commit, since 5,6 is now the I2C bus.
Flash on Xiao_S3_WIO_repeater_observer_mqtt: 42.7% -> 44.3%.
Adds Station_G3_ESP32_repeater_observer_mqtt and
Station_G3_ESP32_room_server_observer_mqtt, mirroring the known-good
Station G2 observer envs. The two boards are the same family — ESP32-S3,
qio_opi, PSRAM, 16 MB flash, SH1106 display — so the overlay ports across
unchanged: default_16MB.csv partitions, adafruit-full cert bundle,
MQTT bridge + SNMP, and the same lib_deps pinning.
Verified the G3 partition signature is byte-identical to the G2 observer's,
so the OTA partition-compatibility gate treats them the same. Both envs
build clean (1.67 MB of the 6.25 MB app slot, 28% RAM) and produce the full
CI artifact set via build.sh: .bin, -merged.bin and .partsig. Observer env
discovery now finds 32 envs (was 30) and the ArduinoJson pin check covers
both new sections.
Docs: added the G3 build commands and the two partition-table rows to
MQTT_IMPLEMENTATION.md.
Absorbs 106 upstream commits. Seven files conflicted; the substantive one
was upstream's new JSON ConfigSerializer (PR #2982), which replaces the
binary /com_prefs layout with /prefs.json and makes NodePrefs a
ConfigSerializer subclass.
Prefs migration
- Adopt upstream's ConfigSerializer. writeCommonPrefsImage() and its
documented offsets (0-294) are deleted, along with the now-unreachable
saveCommonPrefsImageAtomically()/CommonPrefsFileStore atomic rename path.
- Load order is /prefs.json, then /com_prefs, then /node_prefs. Upstream
dropped the /node_prefs fallback; it is restored here so devices that
never advanced past that filename keep their config.
- Legacy files are never removed, so migration cannot destroy its own
source and a deferred or failed save simply retries next boot.
- /com_prefs is treated as a format migration only, not an "upgrade", so
it does not trip the bridge.source tx->rx flip on existing nodes.
- The MQTTPrefsAtomicStore legacy gate is retained: the observer tail
recovered from an old-format file still commits to /mqtt_prefs before
/prefs.json is written.
- MQTTPrefs and /mqtt_prefs are untouched; savePrefs keeps its save_mqtt
parameter and now returns upstream's bool.
Fixes to upstream code
- RadioPrefs::structure() bound both "rxgain" and "fem_rxgain" to
rx_boosted_gain, so radio_fem_rxgain was never persisted. Bound to the
correct field.
- discovery_mod_timestamp was dropped from structure(); it gates
'since'-filtered DISCOVER replies and is set on every config change, so
losing it would silently stop discovery responses after a reboot. Added
as "disc_mod".
Merge artifacts repaired
- Restored bblanchon/ArduinoJson to the native test env; a clean but wrong
auto-merge at the lib_deps block boundary dropped it and broke all 19
host test suites.
- Migrated the fork's WebConfig UITask screens off the removed
DisplayDriver::Color enum to upstream's UIColor element types.
- Removed duplicate getCADEnabled() definitions in companion MyMesh.cpp
and simple_sensor SensorMesh.cpp that both sides had added.
- Dropped memset(&_prefs, 0, ...) in the four example meshes; NodePrefs now
has a vtable. guard gains an initializer that memset used to provide.
Other resolutions
- simple_room_server keeps both the fork's discover.* commands and
upstream's new room.post.
- docs/payloads.md taken from upstream, undoing content earlier merges had
reverted (Control data section, split login tables).
Verified: 273/273 host tests pass across native and native_kiss_modem;
Heltec v3 repeater, repeater_observer_mqtt, room_server_observer_mqtt,
sensor, and companion_radio_ble all build clean.
Port the observer doc cleanup from observer-firmware (4beb5472) onto dev,
keeping dev-specific content intact.
- move internal mechanics into MQTT_INTERNALS.md: deferred construction,
runtime slot memory, backoff/circuit breaker, message building, command
namespacing, and the legacy prefs key mapping
- drop the duplicated First-Time Setup, Command Architecture, and SNMP
Monitoring sections, folding their unique steps into Quick Start
- condense the partition/NVS prose and merge the five custom-broker
examples into one Custom Brokers section
- promote the preset list to a top-level Broker Presets section with an
Extra setup column, and document the presets the table was missing:
meshcore-fi, corecomms, mesh-chaun14, wcmesh, and meshtexas
- fix the coloradomesh port (443, not 1883)
- point flasher and changelog links at observer.gessaman.com
- restore the nbr: status field, which the code still emits, and correct
the neighbors topic to QoS 0 retained per allow_retain
Dev-specific content is preserved as-is: per-slot packet filters, the Web
Configuration Portal, local testing without hardware, MeshRank's raw
exclusion, and neighbors default_scope.
CustomLR1110::startReceive() passed RADIOLIB_LR11X0_IRQ_PREAMBLE_DETECTED
(1<<4 = 16) as RadioLib's first argument, which is the RX *timeout*, not an
IRQ mask. At the LR11x0's 30.52us tick that armed the receiver for ~488us, so
it dropped out of RX before any packet could arrive and the node received
nothing at all -- while transmitting normally.
Symptoms on a SenseCAP T1000-E: tx_air_secs rising, rx_air_secs stuck at 0,
recv_errors 0, and the noise floor pinned at the -120 clamp because
getCurrentRSSI() never sampled a live receiver.
Pass RADIOLIB_LR11X0_RX_TIMEOUT_INF (continuous RX), which is what
LR11x0::startReceive() itself uses, keeping the PREAMBLE_DETECTED flag in the
reported IRQ flags as intended.
Introduced in ea5d7c8b ("LR1110: add PREAMBLE_DETECTED to reported irq flags").
Verified on two T1000-E units: with only the repeater fixed it began receiving
(last_rssi -29, SNR 17.0) while the unfixed companion stayed deaf; fixing both
brought up the link in each direction.
MeshRank slots previously took packets only. The broker tolerates status and
neighbors (and its maintainer intends to look at using the neighbors data), so
those now publish under meshrank/uplink/{token}/{device}/{type}, using the same
type suffixes as the MeshCore layout. Raw stays excluded: it is the
highest-volume topic and the broker does not consume it, so `set mqtt.raw on`
has no effect on a MeshRank slot.
All MeshRank gating funnels through the topic router, so relaxing that single
guard was sufficient - publishStatusToSlot, publishStatus, publishRaw,
publishNeighbors, and eligiblePacketSlots already skip slots that cannot form a
topic. A per-slot token is still required. Because eligiblePacketSlots resolves
topic support before serialising, raw JSON is never built for a MeshRank slot
rather than built and discarded.
observer-firmware sends raw to MeshRank; this supersedes that decision, so raw
must stay excluded when the branches merge. MeshRankTakesEveryTypeExceptRaw
fails if a merge re-enables it.
Enhance the neighbor discovery JSON structure by introducing a
default_scope field, which indicates the region name this node
floods to by default. This change improves clarity in the
neighbor discovery process and aligns with the unscoped flood
behavior when no default region is set. Updates include
modifications to the MyMesh class and related message building
functions to accommodate the new field.
agessaman
Scott Powell
Huw Duddy
mikecarper
IoTThinks
Liam Cottle
me
Kevin Le
Mike Damon
taco