Integrate upstream 1.18.0, delayed direct ACK forwarding, zero-scope routing, Waveshare LR1121 and dynamic WiFi variants. Preserve fork chemistry-aware power management and bounded multi-interface UI; retain retry/dedup safety while adding upstream ACK delay. Qualify LR1121 band power and HP-only Morse handling with native regressions.
Add the framed BLE mOTA transport and companion control path, complete the full-profile build and runtime fixes, and document and test the OTA workflow.
Nothing has ever read it. The slot count comes from two other places: the
persisted array size (MAX_MQTT_SLOTS / RUNTIME_MQTT_SLOTS in MQTTPresets.h) and
the runtime cap MQTTBridge::getMaxActiveSlots(), which answers 5 with PSRAM and
2 without. A flag reading MAX_MQTT_BROKERS=3 sitting in every observer env
implies a third, lower limit that does not exist — it cost me a wrong answer
about how many MQTT slots a Heltec V4 exposes.
36 lines across 15 variants, all of them =3. Removing an unread -D cannot change
code: heltec_v4_repeater_observer_mqtt builds to a byte-identical size before and
after (1647744). The image checksum does differ, but so does it between two
clean builds of untouched source — ESP-IDF stamps the app descriptor with the
build time — so size is the meaningful comparison here.
All 606 envs still parse; heltec_v4, heltec_v3, station_g2, rak3112 and
xiao_s3_wio observer targets all build.
Neighbors publication was gated on BOARD_HAS_PSRAM. Removing that gate alone
was not enough: the feature built but was inert without PSRAM, because three
allocation sites asked for MALLOC_CAP_SPIRAM (which returns null with no
PSRAM), the bridge's persistent buffer was allocated inside a BOARD_HAS_PSRAM
block, and neighborDiscoverReady() rejected every pass at runtime on
psramFound(). The entry table also did not fit: finishNeighborDiscover put
pubkey_hex[50][65] plus entries[50] on the stack, a 4752-byte frame against
the mesh loop task's 8 KB.
- Gate on MAX_NEIGHBOURS plus PSRAM or an explicit per-variant
MQTT_NEIGHBORS_WITHOUT_PSRAM opt-in.
- Move the entry table and its hex strings into one heap block sized to the
pass; the frame drops from 4752 to 304 bytes.
- Prefer PSRAM and fall back to internal DRAM in the mesh-side allocations
and the ArduinoJson pool; hoist the bridge's persistent buffer out of the
BOARD_HAS_PSRAM block (psram_malloc already falls back).
- Keep the runtime psramFound() check only where the buffers are sized for
PSRAM, so a board whose PSRAM failed to init still refuses.
- Size for internal DRAM without PSRAM: 4 KB text buffer and 20 entries per
publish, keeping the pool to a single block and the peak near 13 KB rather
than ~35 KB. Oversized tables truncate and report total_neighbors as before.
Enabled on the ESP32-S3 observer envs (Heltec V3/WSL3, RAK3112, Heltec
Tracker v1.1/v2). Left off for the classic ESP32 T-LoRa V2.1-1.6, which is
already limited to one active TLS slot.
Costs ~7.4 KB static DRAM on repeaters and ~9.6 KB on room servers. The
prefs layout is unchanged, so this is neutral for existing devices.
Absorbs 106 upstream commits. Seven files conflicted; the substantive one
was upstream's new JSON ConfigSerializer (PR #2982), which replaces the
binary /com_prefs layout with /prefs.json and makes NodePrefs a
ConfigSerializer subclass.
Prefs migration
- Adopt upstream's ConfigSerializer. writeCommonPrefsImage() and its
documented offsets (0-294) are deleted, along with the now-unreachable
saveCommonPrefsImageAtomically()/CommonPrefsFileStore atomic rename path.
- Load order is /prefs.json, then /com_prefs, then /node_prefs. Upstream
dropped the /node_prefs fallback; it is restored here so devices that
never advanced past that filename keep their config.
- Legacy files are never removed, so migration cannot destroy its own
source and a deferred or failed save simply retries next boot.
- /com_prefs is treated as a format migration only, not an "upgrade", so
it does not trip the bridge.source tx->rx flip on existing nodes.
- The MQTTPrefsAtomicStore legacy gate is retained: the observer tail
recovered from an old-format file still commits to /mqtt_prefs before
/prefs.json is written.
- MQTTPrefs and /mqtt_prefs are untouched; savePrefs keeps its save_mqtt
parameter and now returns upstream's bool.
Fixes to upstream code
- RadioPrefs::structure() bound both "rxgain" and "fem_rxgain" to
rx_boosted_gain, so radio_fem_rxgain was never persisted. Bound to the
correct field.
- discovery_mod_timestamp was dropped from structure(); it gates
'since'-filtered DISCOVER replies and is set on every config change, so
losing it would silently stop discovery responses after a reboot. Added
as "disc_mod".
Merge artifacts repaired
- Restored bblanchon/ArduinoJson to the native test env; a clean but wrong
auto-merge at the lib_deps block boundary dropped it and broke all 19
host test suites.
- Migrated the fork's WebConfig UITask screens off the removed
DisplayDriver::Color enum to upstream's UIColor element types.
- Removed duplicate getCADEnabled() definitions in companion MyMesh.cpp
and simple_sensor SensorMesh.cpp that both sides had added.
- Dropped memset(&_prefs, 0, ...) in the four example meshes; NodePrefs now
has a vtable. guard gains an initializer that memset used to provide.
Other resolutions
- simple_room_server keeps both the fork's discover.* commands and
upstream's new room.post.
- docs/payloads.md taken from upstream, undoing content earlier merges had
reverted (Control data section, split login tables).
Verified: 273/273 host tests pass across native and native_kiss_modem;
Heltec v3 repeater, repeater_observer_mqtt, room_server_observer_mqtt,
sensor, and companion_radio_ble all build clean.
Implement SNMP functionality in MyMesh to provide radio diagnostics
and statistics updates. This includes a new method for formatting
radio diagnostic replies and periodic updates to the SNMP agent
with radio stats, enhancing monitoring capabilities for users
with SNMP enabled.
Add support for neighbor discovery and management in the MyMesh class.
This includes structures for neighbor information, methods for handling
neighbor advertisements, and control data reception. The new features
allow for dynamic neighbor management, enhancing the mesh network's
capabilities. Additionally, the maximum number of neighbors is set to
50 across various platform configurations.
The observer fetches its manifest from <OTA_MANIFEST_BASE>/<OTA_VARIANT>.json,
so that URL IS the release channel. It was hardcoded to the production channel
in 28 identical places across variants/*/platformio.ini, which made a parallel
(e.g. beta) channel impossible: both channels build the same env names, so beta
devices would read the production manifest and silently flash themselves onto
production firmware.
Inject it from build.sh instead, symmetric with OTA_VARIANT (which no .ini
declares), defaulting to the production URL. Set OTA_MANIFEST_BASE_URL to
publish a parallel channel.
Removed from the .ini files rather than overridden: PLATFORMIO_BUILD_FLAGS
cannot reliably override a -D from build_flags, because SCons reorders -U/-D
and the -U can land after both -Ds, leaving the macro undefined. Verified
empirically before choosing this approach.
A default in a header was deliberately NOT added: leaving both macros undefined
on a plain 'pio run' is what keeps ESP32Board.cpp's 'ERR: OTA not configured
(build via build.sh)' guard firing, so locally built firmware is never OTA-armed.
Verified on Heltec_v3_repeater_observer_mqtt:
- plain 'pio run' -> OTA disarmed, no manifest URL in the binary
- build.sh (default) -> https://observer.gessaman.com/v
- OTA_MANIFEST_BASE_URL set -> beta URL only, production URL absent
First upstream merge since the 2026-06-06 base (191 upstream commits). 14 files
conflicted; resolutions below.
Fleet-critical check (Constraint 1): upstream reordered NodePrefs members
(rx_boosted_gain / path_hash_mode moved to the struct tail) but did NOT change
/com_prefs. Persistence is written field-by-field at explicit offsets, so member
order is in-memory only. Verified the fork's writeCommonPrefsImage() is
byte-identical to upstream's inline writer at every offset (79 pad, 121, 122,
290-294). No migration needed.
Resolutions:
- CommonCLI.h: kept the fork's NodePrefs (superset) and adopted upstream's
setRxBoostedGain(bool)->bool signature change, which CommonCLI.cpp now uses to
report unsupported. Corrected a stale comment claiming rx_boosted_gain lives at
offset 79 (it is a pad; the field is at 290).
- CommonCLI.cpp: kept the fork's legacy /com_prefs migration and the extracted
writeCommonPrefsImage() call.
- UITask.cpp: three-way merge - upstream's drawTextCentered + powering-off
screen, plus the fork's WITH_WEBCONFIG portal/reboot screens.
- ESP32Board.cpp, MeshCore.h, platformio.ini: kept both sides (fork OTA additions
alongside upstream powerOff/enterDeepSleep and Packet.cpp).
- MicroNMEALocationProvider.h: took upstream's claim/release and added the
_claims member they depend on.
- MyMesh.cpp/.h (repeater + room server): kept the fork's superset defaults.
- Removed duplicate declarations auto-merge produced: RadioLibWrapper::_cad_enabled
and MyMesh::getCADEnabled().
Verification: native suite 15/15 (incl. upstream's new test_mesh_tables), both
MQTT smoke builds green, ArduinoJson pin check passes. Hardware validation next.
Replace direct JSON construction in MQTTMessageBuilder with calls to
MQTTPayloadBuilder for building status, packet, and raw messages.
This change improves code maintainability and reduces duplication by
centralizing message formatting logic. Additionally, update platformio.ini
to include ArduinoJson dependency for JSON handling.
Include detailed instructions for local testing of observer and WiFi
functionality without hardware. Document the use of a mock backend and
Wokwi ESP32-S3 simulation for easier development and testing.
Enhance the MQTT implementation documentation to improve developer
experience and facilitate testing workflows.
The 30s "MQTT: Memory" line was useful during the outbox/sync-publish
investigation but is spam for production. Gate the periodic logMemoryStatus()
call in the MQTT task loop behind MQTT_MEMORY_DEBUG (a dedicated diagnostics
flag, not enabled by plain MQTT_DEBUG or production builds) and stop the
heltec_v3 variant from force-enabling MQTT_MEMORY_DEBUG on its observer_mqtt
envs (now commented out to match heltec_v4). logMemoryStatus() itself is kept
intact for opt-in debugging.
Expose the same data on demand via a new `get mqtt.stats` CLI command backed by
MQTTBridge::formatMqttStatsReply(): free/max heap, queue depth, outbox total,
and per-slot publish ok/err counts (1-based, matching the msgs: line). Fits the
160-byte reply buffer at 6 slots; returns "(bridge not running)" when down.
Integrates Mike Carper's transmission-reliability work (direct retries with
SNR-adaptive backoff and CR, flood retry controls, alt-path replies, flood
channel gates) plus its upstream/dev base (as of 2026-07-06) on top of the
observer stack. The flex branch is untouched; this branch is the experimental
integration line.
Conflict-resolution decisions, for future re-syncs (git rerere is enabled and
has recorded these):
- NodePrefs adopts keymind/upstream member order + retry/flood tail. Member
order is in-memory only: /com_prefs stays field-by-field with the same
canonical file order both sides already share through offset 294; keymind
appends the retry tail at 295+ (new canonical size 676). Flex fleet files
load unchanged; retry fields default via direct_retry_prefs_magic.
- loadPrefsInt keeps the fork's legacy MQTT-gap recovery but moves its
detection boundary from 'extra > 5' to 'extra > LEGACY_MQTT_GAP_3SLOT (864)'
so keymind-size tails take the normal read path. COM_PREFS_TAIL_BYTES
retired.
- Dispatcher/RadioLibWrappers: fork watchdog additions + keymind TX overrides
and CAD busy counter are unioned; RxReservePacketManager::queueOutbound
follows upstream's new bool return (false when shedding).
- CommonCLI: fork observer dispatch + versioned /mqtt_prefs machinery kept;
keymind CLI (retry/flood/radioat) taken; duplicate CAD/FEM handlers and
sanitise lines deduped (kept keymind placements to minimize future diff).
- MyMesh (repeater/room): fork alerter/OTA/observer wiring kept; keymind
scheduled-radio system replaces the old pending_* temp-radio members;
applySavedRadioParams() replaces raw setParams at startup.
- ESP32Board: fork manifest-OTA + keymind powerOff/deepSleep and stopOTAUpdate
unioned; startOTAUpdate keeps fork's STA-aware IP with keymind's idempotency
guard.
- build.sh taken wholesale from keymind (fleet builds use GH Actions).
Verified: Heltec_v3_repeater_observer_mqtt, Heltec_v3_repeater,
heltec_v4_repeater_observer_mqtt, Heltec_v3_room_server_observer_mqtt build;
host migration harness (13 MQTT + non-MQTT scenarios) passes with updated
expectations (676-byte /com_prefs; odd-size files normalize on save, not
load); native unit tests 13/13.
Enhanced the firmware versioning system by appending a build number
suffix when available, allowing for better tracking of published builds.
This change improves the OTA update process by providing clearer
versioning information in the embedded firmware string.
Added functionality to support pull-based OTA updates by fetching firmware
from a manifest. The new `otaFromManifest` method allows the system to
check for available updates and flash the firmware if necessary. This
enhancement improves the update process for observer builds using the
MQTT bridge, ensuring a more seamless firmware management experience.
- Introduced `Heltec_T190_room_server_observer_mqtt` and `Heltec_WSL3_room_server_observer_mqtt` environments.
- Configured SSL certificate handling and embedded files for both environments.
- Set compile-time flags for MQTT settings, including advertisement names, passwords, and MQTT bridge options.
- Updated library dependencies to include necessary MQTT and JSON libraries.
Update the PsychicMqttClient to ensure that QoS0 messages are enqueued with durable outbox storage. This change addresses issues with false-failure semantics in certain connected paths, improving message flow reliability. Additionally, modify platformio.ini to include SSL certificate generation and adjust build flags for reduced verbosity and enhanced functionality.
mikecarper
Rastislav Vysoky
liamcottle
Michael Graff
agessaman
Kevin Le
Valentin Kivachuk Burda
Scott Powell