Picks up upstream MeshCore 1.17.0 and the dev commits that followed it.
Notable upstream content:
- 1.17.0 version/build-date bump in the example MyMesh headers.
- anon_req hardening: reply_path_len is now uint8_t with an isValidPathLen()
bounds check and a 0xFF sentinel; reply_path_hash_size is gone.
- LR2021 support (Meshnology W12, Seeed MeshTracker X1) incl. side detectors,
multi-SF and the new `extra.sf` get/set CLI.
- CustomLFS 0.2.2 -> 0.2.3 (GD25Q64C support).
- kiss_modem envs for several nRF52 variants; ThinkNode M6 GPS/flash fixes.
Conflicts resolved:
- platformio.ini: took upstream's CustomLFS 0.2.3, kept our lib_ignore for
the vendored PsychicMqttClient.
- CommonCLI.h: kept both upstream's USE_LR2021 configSideDetectors() hook and
our fault-alert callbacks.
NodePrefs gains upstream's extra_sf[4], which is not registered with
ConfigSerializer, so /prefs.json layout is unchanged.
Verified: 275/275 native host tests pass; Heltec_v3_repeater_observer_mqtt,
Heltec_v3_repeater and heltec_v4_repeater_observer_mqtt all build.
Three fork-introduced dependencies had leaked into non-ESP32 builds, breaking
every nRF52 and RP2040 target since 7e4f75c9 (2026-04-10):
- The `memory` CLI command called ESP.getFreeHeap()/heap_caps_* unguarded from
shared CommonCLI.cpp. ESP32 output is unchanged; other platforms now report
newlib arena stats, omitting min-ever-free and largest-free-block rather than
substituting numbers that mean something different.
- The vendored PsychicMqttClient (ESP-IDF esp-mqtt) was pulled in by the LDF on
nRF52; nrf52_base now lib_ignores it, matching how RP2040 variants ignore BLE.
- JWTHelper.cpp and MQTTMessageBuilder.cpp are excluded in arduino_base, but 22
variants re-glob helpers/*.cpp after inheriting it, which undoes the exclusion.
Guarding the file contents on WITH_MQTT_BRIDGE is robust against any variant's
filter, and matches helpers/esp32/WebConfigServer.cpp.
Verified: RAK4631 repeater + room server, Heltec T114, T1000-E, Wio WM1110,
Xiao nRF52 and ThinkNode M1 all build. ESP32 observer builds are byte-identical
to before (RAM and flash), and the native suite stays at 267/267.
RP2040 still fails separately: four boards declare the pre-force_ap
startOTAUpdate signature. Not addressed here.
Absorbs 106 upstream commits. Seven files conflicted; the substantive one
was upstream's new JSON ConfigSerializer (PR #2982), which replaces the
binary /com_prefs layout with /prefs.json and makes NodePrefs a
ConfigSerializer subclass.
Prefs migration
- Adopt upstream's ConfigSerializer. writeCommonPrefsImage() and its
documented offsets (0-294) are deleted, along with the now-unreachable
saveCommonPrefsImageAtomically()/CommonPrefsFileStore atomic rename path.
- Load order is /prefs.json, then /com_prefs, then /node_prefs. Upstream
dropped the /node_prefs fallback; it is restored here so devices that
never advanced past that filename keep their config.
- Legacy files are never removed, so migration cannot destroy its own
source and a deferred or failed save simply retries next boot.
- /com_prefs is treated as a format migration only, not an "upgrade", so
it does not trip the bridge.source tx->rx flip on existing nodes.
- The MQTTPrefsAtomicStore legacy gate is retained: the observer tail
recovered from an old-format file still commits to /mqtt_prefs before
/prefs.json is written.
- MQTTPrefs and /mqtt_prefs are untouched; savePrefs keeps its save_mqtt
parameter and now returns upstream's bool.
Fixes to upstream code
- RadioPrefs::structure() bound both "rxgain" and "fem_rxgain" to
rx_boosted_gain, so radio_fem_rxgain was never persisted. Bound to the
correct field.
- discovery_mod_timestamp was dropped from structure(); it gates
'since'-filtered DISCOVER replies and is set on every config change, so
losing it would silently stop discovery responses after a reboot. Added
as "disc_mod".
Merge artifacts repaired
- Restored bblanchon/ArduinoJson to the native test env; a clean but wrong
auto-merge at the lib_deps block boundary dropped it and broke all 19
host test suites.
- Migrated the fork's WebConfig UITask screens off the removed
DisplayDriver::Color enum to upstream's UIColor element types.
- Removed duplicate getCADEnabled() definitions in companion MyMesh.cpp
and simple_sensor SensorMesh.cpp that both sides had added.
- Dropped memset(&_prefs, 0, ...) in the four example meshes; NodePrefs now
has a vtable. guard gains an initializer that memset used to provide.
Other resolutions
- simple_room_server keeps both the fork's discover.* commands and
upstream's new room.post.
- docs/payloads.md taken from upstream, undoing content earlier merges had
reverted (Control data section, split login tables).
Verified: 273/273 host tests pass across native and native_kiss_modem;
Heltec v3 repeater, repeater_observer_mqtt, room_server_observer_mqtt,
sensor, and companion_radio_ble all build clean.
Enhance the MQTT preferences handling by establishing a downgrade
contract that ensures compatibility between different firmware versions.
This contract allows nodes to read settings from newer builds while
safeguarding against data loss during downgrades. The implementation
includes updates to the classification logic, ensuring that longer
payloads from newer versions are handled correctly without rejecting
files, thus preserving critical WiFi credentials and broker settings.
Add support for named packet types in per-slot filters, allowing users
to specify packet types using descriptive names alongside numeric values.
This improves usability and clarity in configuring MQTT slot filters.
Updates include modifications to the parsing logic, WebConfig interface,
and related documentation to reflect the new naming conventions.
Enhance the documentation for the `start ota` command to clarify its
behavior when connected to a Wi-Fi network versus when using the
`start ota ap` option. This provides users with better guidance on
how to initiate OTA updates under different network conditions.
Update commit logic to remove existing destination before renaming
the temporary file on ESP32. This change addresses the SPIFFS_ERR_CONFLICTING_NAME
issue, ensuring that the new preferences are published correctly.
Add tests to validate the new behavior and confirm that the
temporary file is handled appropriately.
Append mqtt_neighbors_enabled(u8) + mqtt_neighbors_interval(u32) to the
observer tail of MQTTPrefs. The layout is kept byte-identical to the flex
neighbors build: the enable flag lands in the old struct's zeroed trailing
padding (offset 2857) and the interval begins exactly at the former baseline
(2860), so sizeof grows 2860 -> 2864 (net +4 bytes). offsetof static_asserts
lock the layout so a mismatch fails the build.
The codec now accepts three v1 payload sizes: register 2860 as a
"pre-neighbors" Current payload so an in-lineage upgrade reads its existing
/mqtt_prefs and defaults the neighbors tail (off / 24h). Because 2864 is the
shared Current baseline, a /mqtt_prefs written by either the flex build or this
firmware is interchangeable.
Add the 12/24/336h interval constants, neighbors defaults, and a load-time
interval clamp that keeps persisted values inside the signed-delta millis()
scheduling window. Extend the host codec suite with a pre-neighbors migration
case and neighbors round-trip coverage.
Saving observer prefs logged an ESP32 error line on every save:
[E][vfs_api.cpp:182] remove(): /mqtt_prefs.tmp does not exists or is directory
MQTTPrefsFileStore::begin() cleared a stale transaction with an unconditional
remove("/mqtt_prefs.tmp"). On the normal path there is no stale tmp - commit()
renames it away - so the remove always failed and the ESP32 VFS layer logged it
at [E] level. The save itself succeeded; the noise just reads as a fault in the
serial log at exactly the moment an operator is watching a config change.
Guard each remove on exists(), at all four sites: begin() and abort() for both
the /mqtt_prefs and /com_prefs stores. Semantics are unchanged - a genuinely
stale tmp is still cleared, and a failure to clear it still aborts the
transaction - it just stops issuing a syscall that can only fail.
Fixed here on webconfig (the 1.16.0-based line that carries the atomic store)
so it flows to flex with the rest of that work. NOT applicable to
mqtt-bridge-implementation-flex today: flex has no .tmp/rename handling at all,
so the code path does not exist there.
Verified: Heltec_v3_repeater_observer_mqtt builds; hardware confirmation of the
silenced log pending.
First upstream merge since the 2026-06-06 base (191 upstream commits). 14 files
conflicted; resolutions below.
Fleet-critical check (Constraint 1): upstream reordered NodePrefs members
(rx_boosted_gain / path_hash_mode moved to the struct tail) but did NOT change
/com_prefs. Persistence is written field-by-field at explicit offsets, so member
order is in-memory only. Verified the fork's writeCommonPrefsImage() is
byte-identical to upstream's inline writer at every offset (79 pad, 121, 122,
290-294). No migration needed.
Resolutions:
- CommonCLI.h: kept the fork's NodePrefs (superset) and adopted upstream's
setRxBoostedGain(bool)->bool signature change, which CommonCLI.cpp now uses to
report unsupported. Corrected a stale comment claiming rx_boosted_gain lives at
offset 79 (it is a pad; the field is at 290).
- CommonCLI.cpp: kept the fork's legacy /com_prefs migration and the extracted
writeCommonPrefsImage() call.
- UITask.cpp: three-way merge - upstream's drawTextCentered + powering-off
screen, plus the fork's WITH_WEBCONFIG portal/reboot screens.
- ESP32Board.cpp, MeshCore.h, platformio.ini: kept both sides (fork OTA additions
alongside upstream powerOff/enterDeepSleep and Packet.cpp).
- MicroNMEALocationProvider.h: took upstream's claim/release and added the
_claims member they depend on.
- MyMesh.cpp/.h (repeater + room server): kept the fork's superset defaults.
- Removed duplicate declarations auto-merge produced: RadioLibWrapper::_cad_enabled
and MyMesh::getCADEnabled().
Verification: native suite 15/15 (incl. upstream's new test_mesh_tables), both
MQTT smoke builds green, ArduinoJson pin check passes. Hardware validation next.
Rework the /mqtt_prefs load/save path so preference migrations are
crash-safe and, for the first time, unit-testable on the host.
Most of this is extraction. The multi-format migration that previously
lived inline in CommonCLI.cpp (and could only run on-device) is moved
into three dependency-free headers so it can be exercised without
Arduino, a filesystem, or the radio stack:
- MQTTPrefsStorage.h frozen layout structs for every /mqtt_prefs
format ever shipped, with static_asserts that
fail the build if any on-flash offset changes.
- MQTTPrefsCodec.h pure format classification, field-copy
migration, and plausibility validation.
- MQTTPrefsAtomicStore.h transactional writer plus the power-cut
upgrade gate, both host-testable.
New behavior, beyond the refactor:
- Atomic writes: /mqtt_prefs is written to /mqtt_prefs.tmp, verified,
then published with an atomic rename; the writer never removes the
existing file. A failed or interrupted save leaves the current
config intact.
- Power-cut ordering: LegacyUpgradeGate guarantees /mqtt_prefs is
durably committed before the legacy /com_prefs (or /node_prefs)
carrying the observer tail is compacted or removed, so an
interrupted two-file upgrade retries on the next boot without
losing settings.
- Corrupt, unsupported-version, and newer-than-known files are
preserved and the device boots on in-RAM defaults, rather than
overwriting a file this firmware cannot fully decode.
- Headerless legacy formats are validated for plausibility before
they are trusted and rewritten (raw prefs carry no checksum).
The full historical format matrix is migrated forward to the versioned
v1 layout: pre-slot (including pre-wifi-power), 3-slot (base and
token/topic tails), and headerless 6-slot (base, audience, rx, ntp).
Scope note: only /mqtt_prefs and the one-time /node_prefs -> /com_prefs
name migration use the atomic path. Ordinary /com_prefs saves remain a
direct rewrite, unchanged by this commit.
Tests: adds two host GoogleTest suites (pio test -e native).
- test_mqtt_prefs_codec: format classification, migration fixtures,
v1 header integrity, downgrade preservation.
- test_mqtt_prefs_atomic_store: transactional writes, short-write
detection, begin/finish/rename failure cleanup, original-file
preservation.
mikecarper
Kevin Le
Jarosław Domański
Jarosław Domański
agessaman
taco
IoTThinks
Scott Powell