diff --git a/MQTT_IMPLEMENTATION.md b/MQTT_IMPLEMENTATION.md index 6dbf4f7b7..1908e8744 100644 --- a/MQTT_IMPLEMENTATION.md +++ b/MQTT_IMPLEMENTATION.md @@ -172,6 +172,16 @@ Fresh installs default to slot 1 `analyzer-us`, slot 2 `analyzer-eu`, and slots - Slots configured beyond what the device supports show as `(inactive)` in `get mqtt.status` - Slot configuration is preserved in preferences — moving the firmware to a PSRAM device activates the rest +**Neighbors publication without PSRAM.** PSRAM boards get `WITH_MQTT_NEIGHBORS` +automatically. Non-PSRAM boards opt in per variant with +`-D MQTT_NEIGHBORS_WITHOUT_PSRAM=1`, which is set on the ESP32-S3 observer envs +(Heltec V3/WSL3, RAK3112, Heltec Tracker v1.1/v2). It costs ~7.4 KB of static DRAM +(~9.6 KB on room servers) and up to ~13 KB transiently per publish, and caps the +table at 20 entries — so the feature is deliberately **not** enabled on the classic +ESP32 T-LoRa V2.1–1.6 observer builds, which are already down to one active TLS +slot. Publishing peaks while the table is built, so keep the slot guidance above +in mind: the peak lands on the same internal heap the TLS stack draws from. + ## Build Configuration To build the MQTT bridge firmware: @@ -209,7 +219,7 @@ pio run -e ThinkNode_M7_room_server_observer_mqtt `ThinkNode_M7_companion_radio_ethernet` uses it, but the MQTT bridge's link management is bound to the WiFi station API, so the observer envs uplink over WiFi. See `UPSTREAM_BUGS.md` for the Ethernet gap. The board has PSRAM, so these builds -get the PSRAM-only neighbors publication (`WITH_MQTT_NEIGHBORS`). +get neighbors publication (`WITH_MQTT_NEIGHBORS`) automatically. **TLora naming:** The env prefix `LilyGo_TLora_V2_1_1_6` is LilyGo’s **T-LoRa V2.1–1.6** board (SX1276); PlatformIO selects **`ttgo-lora32-v1`** (TTGO LoRa32 V1.0). **MQTT observer** envs extend a slim base **without** `sensor_base` so the image fits `min_spiffs`; **all other** `LilyGo_TLora_V2_1_1_6_*` targets still use optional I2C environmental sensors as before. The **`lilygo_tlora_c6`** variant is separate hardware (ESP32-C6). @@ -500,8 +510,8 @@ These settings apply across all MQTT slots: - `get mqtt.rx` - Get RX packet uplinking setting (on/off) - `get mqtt.tx` - Get TX packet uplinking setting (on/off/advert) - `get mqtt.interval` - Get status publish interval -- `get mqtt.neighbors` - Get periodic neighbors publishing setting (on/off; PSRAM only) -- `get mqtt.neighbors.interval` - Get neighbors publish interval in hours (PSRAM only) +- `get mqtt.neighbors` - Get periodic neighbors publishing setting (on/off; neighbors-enabled builds) +- `get mqtt.neighbors.interval` - Get neighbors publish interval in hours (neighbors-enabled builds) - `get mqtt.ntp` - Get effective NTP server hostname - `get mqtt.ntp.diag` - Probe every configured NTP server for connectivity (does not change the clock; serial console shows each server's reported time, LoRa shows a compact ` ok|fail` list) - `get mqtt.owner` - Get owner public key (serial console only) @@ -519,8 +529,8 @@ These settings apply across all MQTT slots: - `advert` - Uplink only this node's own advert packets (self-originated) - `off` - Disable TX packet uplinking - `set mqtt.interval ` - Set status publish interval (1-60 minutes) -- `set mqtt.neighbors on|off` - Enable/disable periodic neighbors publishing (PSRAM only; read live, no restart) -- `set mqtt.neighbors.interval ` - Set neighbors publish interval (12-336 hours, default 24; PSRAM only) +- `set mqtt.neighbors on|off` - Enable/disable periodic neighbors publishing (neighbors-enabled builds; read live, no restart) +- `set mqtt.neighbors.interval ` - Set neighbors publish interval (12-336 hours, default 24; neighbors-enabled builds) - `set mqtt.ntp ` - Set custom NTP server (validated with immediate sync); `none` reverts to default - `set mqtt.owner <64-hex-char-public-key>` - Set owner public key - `set mqtt.email ` - Set owner email address @@ -698,7 +708,7 @@ Full packet data with RF characteristics and metadata. Minimal raw packet data for map integration. ### Neighbors Topic: `meshcore/{IATA}/{DEVICE_PUBLIC_KEY}/neighbors` -Cached zero-hop repeater neighbors with SNR, last-heard age, and flood-allowed scopes. Published on `discover.scopes` or periodically when `mqtt.neighbors` is enabled (PSRAM observer builds only). Goes to every configured slot's `neighbors` topic at QoS 0, retained only where the preset allows it. +Cached zero-hop repeater neighbors with SNR, last-heard age, and flood-allowed scopes. Published on `discover.scopes` or periodically when `mqtt.neighbors` is enabled (observer builds with neighbors compiled in; non-PSRAM builds cap the table at 20 entries and set `truncated`). Goes to every configured slot's `neighbors` topic at QoS 0, retained only where the preset allows it. Periodic publishing first runs a 60-second zero-hop neighbor refresh equivalent to `discover.neighbors`, then queries the refreshed table for scopes and publishes when the scope-query phase completes. @@ -784,7 +794,7 @@ While `mqtt.neighbors` is on, `get mqtt.status` appends `nbr: /` — } ``` -### Neighbors Message (PSRAM observer builds) +### Neighbors Message (neighbors-enabled observer builds) ```json { "timestamp": "2024-01-01T12:00:00.000000+00:00", diff --git a/MQTT_INTERNALS.md b/MQTT_INTERNALS.md index 46109c93e..d0de03985 100644 --- a/MQTT_INTERNALS.md +++ b/MQTT_INTERNALS.md @@ -74,11 +74,22 @@ defined in `MQTTBridge.h`). It spans two subsystems and two cores: the snapshot and `neighbours[]` — proof of reception, and the only thing that heals a stamp taken before the clock was set. After all responses land or a 30 s window expires, `finishNeighborDiscover()` builds the JSON with - `MQTTMessageBuilder::buildNeighborsMessage` into a transient PSRAM buffer and hands it - to the bridge. Ages that still span a clock epoch publish as `null` rather than a + `MQTTMessageBuilder::buildNeighborsMessage` into a transient buffer (PSRAM where + available, internal DRAM otherwise) and hands it to the bridge. The entry table and its + hex strings share one heap block sized to the pass — at `MAX_NEIGHBOURS` they reach + ~4.5 KB, which does not fit the mesh loop task's 8 KB stack. Ages that still span a clock epoch publish as `null` rather than a fabricated delta (`neighborHeardAgeUsable`, see `UPSTREAM_BUGS.md` #1). -- **Bridge side, handoff**: `requestPublishNeighbors(json, len)` (Core 1) memcpys into a - persistent ~10 KB PSRAM buffer (`NEIGHBORS_JSON_BUFFER_SIZE`) and sets +- **Buffer sizing**: `NEIGHBORS_JSON_BUFFER_SIZE` is 10 KB with PSRAM and 4 KB without, + since a non-PSRAM board pays for the persistent buffer, the transient build buffer and + the ArduinoJson pool out of the same internal DRAM each TLS slot needs ~40 KB of + (~13 KB peak instead of ~35 KB). The pool has its own budget + (`NEIGHBORS_DOC_POOL_BUDGET`) because ArduinoJson v7 hands out pool blocks in fixed + 4096-byte chunks, so a table that just fits the text buffer can still need well over + it in pool — and a starved pool sets `doc.overflowed()`, which drops the entire publish + instead of truncating it. `NEIGHBORS_MAX_PUBLISH_ENTRIES` (20 without PSRAM) keeps the + pool inside a single block. +- **Bridge side, handoff**: `requestPublishNeighbors(json, len)` (Core 1) memcpys into the + persistent buffer (`NEIGHBORS_JSON_BUFFER_SIZE`, PSRAM where available) and sets `_neighbors_publish_pending` with a release store; the MQTT task (`mqttTaskLoop`, Core 0) consumes it with an acquire load, calls `publishNeighbors()`, and clears the flag. A second snapshot is dropped while one is in flight. `publishNeighbors()` sends QoS 1, diff --git a/docs/cli_commands.md b/docs/cli_commands.md index bfab6ae76..f3779c8a8 100644 --- a/docs/cli_commands.md +++ b/docs/cli_commands.md @@ -136,9 +136,10 @@ scopes and publishes the assembled table to the MQTT `neighbors` topic once. **Usage:** - `discover.scopes` -**Note:** Requires a PSRAM board with the MQTT bridge running. On non-PSRAM MQTT -builds it replies `Err - not supported (requires PSRAM)`. If a `discover.neighbors` -refresh is already in flight, the scope pass is queued behind it. +**Note:** Requires an MQTT observer build with the neighbors feature compiled in +(all PSRAM boards, plus non-PSRAM boards built with `MQTT_NEIGHBORS_WITHOUT_PSRAM`). +Elsewhere it replies `Err - neighbors not enabled in this build`. If a +`discover.neighbors` refresh is already in flight, the scope pass is queued behind it. --- @@ -1160,8 +1161,12 @@ region save **Default:** `off` -> **Note:** Requires a PSRAM board. On non-PSRAM MQTT builds this replies -> `Err - not supported (requires PSRAM)`. The setting is read live by the mesh +> **Note:** Requires a build with the neighbors feature compiled in (all PSRAM +> boards, plus non-PSRAM boards built with `MQTT_NEIGHBORS_WITHOUT_PSRAM`); +> elsewhere this replies `Err - neighbors not enabled in this build`. Non-PSRAM +> builds publish at most 20 neighbours per pass to bound internal-DRAM use, and +> set `truncated` with the true `total_neighbors` when the table is larger. +> The setting is read live by the mesh > loop — no restart required; enabling it triggers a discovery on the next pass. > While enabled, `get mqtt.status` gains a trailing `nbr: /` field > (time to next publish, and how the last publish went). diff --git a/examples/simple_repeater/MyMesh.cpp b/examples/simple_repeater/MyMesh.cpp index c7ed77152..efb6532b6 100644 --- a/examples/simple_repeater/MyMesh.cpp +++ b/examples/simple_repeater/MyMesh.cpp @@ -1651,7 +1651,7 @@ void MyMesh::handleCommand(uint32_t sender_timestamp, char *command, char *reply } #elif defined(WITH_MQTT_BRIDGE) } else if (memcmp(command, "discover.scopes", 15) == 0) { - strcpy(reply, "Err - not supported (requires PSRAM)"); + strcpy(reply, "Err - neighbors not enabled in this build"); #endif } else{ _cli.handleCommand(sender_timestamp, command, reply); // common CLI commands @@ -2021,34 +2021,61 @@ static bool neighborPublishEntryComesBefore( } #if defined(ESP_PLATFORM) +// Neighbors allocations prefer PSRAM where it exists and otherwise come from +// internal DRAM, so MQTT_NEIGHBORS_WITHOUT_PSRAM boards can build the table too. +#if defined(BOARD_HAS_PSRAM) +static const uint32_t kNeighborsAllocCaps = MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT; +#else +static const uint32_t kNeighborsAllocCaps = MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT; +#endif + +static void* neighborsAlloc(size_t size) { + if (size == 0) return nullptr; + void* p = heap_caps_malloc(size, kNeighborsAllocCaps); +#if defined(BOARD_HAS_PSRAM) + if (!p) p = heap_caps_malloc(size, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT); +#endif + return p; +} + +static void neighborsFree(void* ptr) { + if (ptr) heap_caps_free(ptr); +} + // ArduinoJson v7 JsonDocument has no real capacity cap (DynamicJsonDocument(N) -// is a no-op shim). Keep the pool off internal DRAM and soft-cap peak growth to -// the publish buffer size. used only rises on allocate — conservative for this -// single-shot doc (overflow path removes+breaks, so no further growth after free). +// is a no-op shim), so soft-cap peak pool growth to NEIGHBORS_DOC_POOL_BUDGET. +// used only rises on allocate — conservative for this single-shot doc (overflow +// path removes+breaks, so no further growth after free). struct NeighborsDocAllocator : ArduinoJson::Allocator { size_t used = 0; static const size_t kBudget = MQTTBridge::NEIGHBORS_DOC_POOL_BUDGET; void* allocate(size_t size) override { if (used >= kBudget || size > kBudget - used) return nullptr; - void* p = heap_caps_malloc(size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT); + void* p = neighborsAlloc(size); if (p) used += size; return p; } void deallocate(void* ptr) override { - heap_caps_free(ptr); + neighborsFree(ptr); } void* reallocate(void* ptr, size_t new_size) override { size_t old_size = ptr ? heap_caps_get_allocated_size(ptr) : 0; size_t next_used = (used >= old_size) ? (used - old_size) : 0; if (next_used >= kBudget || new_size > kBudget - next_used) return nullptr; - void* p = heap_caps_realloc(ptr, new_size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT); + void* p = heap_caps_realloc(ptr, new_size, kNeighborsAllocCaps); +#if defined(BOARD_HAS_PSRAM) + if (!p) p = heap_caps_realloc(ptr, new_size, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT); +#endif if (p) used = next_used + new_size; return p; } }; +#else +static void* neighborsAlloc(size_t size) { return size ? malloc(size) : nullptr; } +static void neighborsFree(void* ptr) { free(ptr); } #endif // Build the neighbors-table JSON and hand it to the bridge, then reschedule. @@ -2059,80 +2086,69 @@ void MyMesh::finishNeighborDiscover() { char timestamp[40]; MQTTMessageBuilder::formatIsoTimestampForMqtt(getRTCClock()->getCurrentTime(), 0, nullptr, timestamp, sizeof(timestamp)); - char pubkey_hex[MAX_NEIGHBOURS][65]; - MQTTMessageBuilder::NeighborsMessageEntry entries[MAX_NEIGHBOURS]; - uint32_t now_secs = getRTCClock()->getCurrentTime(); + // The entry table plus one hex string each reaches ~4.5 KB at MAX_NEIGHBOURS, + // which does not fit the mesh loop task's 8 KB stack, so both share a single + // heap block sized to this pass. Publishing is skipped if either alloc fails. + const int publish_count = neighbor_discover_publish_count; + const size_t hex_size = PUB_KEY_SIZE * 2 + 1; + const size_t entries_bytes = + sizeof(MQTTMessageBuilder::NeighborsMessageEntry) * publish_count; + void* scratch = neighborsAlloc(entries_bytes + hex_size * publish_count); + char* json_buf = (char*)neighborsAlloc(MQTTBridge::NEIGHBORS_JSON_BUFFER_SIZE); - for (int i = 0; i < neighbor_discover_publish_count; i++) { - auto& entry = neighbor_discover[i]; - mesh::Utils::toHex(pubkey_hex[i], entry.id.pub_key, PUB_KEY_SIZE); - entries[i].pubkey_hex = pubkey_hex[i]; - entries[i].snr = entry.snr / 4.0f; - bool heard_known = neighborHeardAgeUsable(entry.heard_timestamp, now_secs); - entries[i].heard_unknown = !heard_known; - entries[i].heard_secs_ago = heard_known ? (now_secs - entry.heard_timestamp) : 0; - entries[i].scopes = entry.scopes; - switch (entry.status) { - case ND_RESPONDED: entries[i].status = "responded"; break; - case ND_SEND_FAILED: entries[i].status = "send_failed"; break; - default: entries[i].status = "timeout"; break; + if (json_buf && (scratch || publish_count == 0)) { + auto* entries = (MQTTMessageBuilder::NeighborsMessageEntry*)scratch; + char* pubkey_hex = (char*)scratch + entries_bytes; + uint32_t now_secs = getRTCClock()->getCurrentTime(); + + for (int i = 0; i < publish_count; i++) { + auto& entry = neighbor_discover[i]; + char* hex = &pubkey_hex[i * hex_size]; + mesh::Utils::toHex(hex, entry.id.pub_key, PUB_KEY_SIZE); + entries[i].pubkey_hex = hex; + entries[i].snr = entry.snr / 4.0f; + bool heard_known = neighborHeardAgeUsable(entry.heard_timestamp, now_secs); + entries[i].heard_unknown = !heard_known; + entries[i].heard_secs_ago = heard_known ? (now_secs - entry.heard_timestamp) : 0; + entries[i].scopes = entry.scopes; + switch (entry.status) { + case ND_RESPONDED: entries[i].status = "responded"; break; + case ND_SEND_FAILED: entries[i].status = "send_failed"; break; + default: entries[i].status = "timeout"; break; + } + } + + // insertion sort: most useful first (JSON builder drops the tail on overflow) + for (int i = 1; i < publish_count; i++) { + MQTTMessageBuilder::NeighborsMessageEntry entry = entries[i]; + int j = i; + while (j > 0 && neighborPublishEntryComesBefore(entry, entries[j - 1])) { + entries[j] = entries[j - 1]; + j--; + } + entries[j] = entry; + } + +#if defined(ESP_PLATFORM) + NeighborsDocAllocator doc_alloc; + JsonDocument doc(&doc_alloc); +#else + JsonDocument doc; +#endif + int json_len = MQTTMessageBuilder::buildNeighborsMessage( + doc, neighbor_discover_origin, self_pubkey_hex, timestamp, self_scopes_buf, + self_default_scope_buf, entries, publish_count, + json_buf, MQTTBridge::NEIGHBORS_JSON_BUFFER_SIZE, + neighbor_discover_count, neighbor_discover_queried_count, + neighbor_discover_truncated); + + if (json_len > 0 && bridge) { + bridge->requestPublishNeighbors(json_buf, (size_t)json_len); } } - // insertion sort: most useful first (JSON builder drops the tail on overflow) - for (int i = 1; i < neighbor_discover_publish_count; i++) { - MQTTMessageBuilder::NeighborsMessageEntry entry = entries[i]; - int j = i; - while (j > 0 && neighborPublishEntryComesBefore(entry, entries[j - 1])) { - entries[j] = entries[j - 1]; - j--; - } - entries[j] = entry; - } - -#if defined(ESP_PLATFORM) - char* json_buf = (char*)heap_caps_malloc(MQTTBridge::NEIGHBORS_JSON_BUFFER_SIZE, MALLOC_CAP_SPIRAM); -#else - char* json_buf = (char*)malloc(MQTTBridge::NEIGHBORS_JSON_BUFFER_SIZE); -#endif - if (!json_buf) { - neighbor_discover_active = false; - neighbor_discover_count = 0; - neighbor_discover_next = 0; - neighbor_discover_publish_count = 0; - neighbor_discover_queried_count = 0; - neighbor_discover_json_size = 0; - neighbor_discover_truncated = false; - neighbor_discover_until = 0; - neighbor_discover_request = NULL; - if (_cli.getObserverPrefs()->mqtt_neighbors_enabled) { - next_neighbors_publish = futureMillis(_cli.getObserverPrefs()->mqtt_neighbors_interval); - } - return; - } - -#if defined(ESP_PLATFORM) - NeighborsDocAllocator doc_alloc; - JsonDocument doc(&doc_alloc); -#else - JsonDocument doc; -#endif - int json_len = MQTTMessageBuilder::buildNeighborsMessage( - doc, neighbor_discover_origin, self_pubkey_hex, timestamp, self_scopes_buf, - self_default_scope_buf, entries, neighbor_discover_publish_count, - json_buf, MQTTBridge::NEIGHBORS_JSON_BUFFER_SIZE, - neighbor_discover_count, neighbor_discover_queried_count, - neighbor_discover_truncated); - - if (json_len > 0 && bridge) { - bridge->requestPublishNeighbors(json_buf, (size_t)json_len); - } - -#if defined(ESP_PLATFORM) - heap_caps_free(json_buf); -#else - free(json_buf); -#endif + neighborsFree(scratch); + neighborsFree(json_buf); neighbor_discover_active = false; neighbor_discover_count = 0; @@ -2204,9 +2220,12 @@ void MyMesh::loopNeighborDiscover() { } } -// Shared precondition for starting a discovery: PSRAM present + bridge running. +// Shared precondition for starting a discovery: usable buffers + bridge running. +// PSRAM builds size their neighbors buffers for PSRAM, so a board whose PSRAM +// failed to init must not silently spend that much internal DRAM here. +// MQTT_NEIGHBORS_WITHOUT_PSRAM builds are already sized for internal DRAM. bool MyMesh::neighborDiscoverReady(char* reply) { -#if defined(ESP_PLATFORM) +#if defined(ESP_PLATFORM) && defined(BOARD_HAS_PSRAM) if (!psramFound()) { strcpy(reply, "Err - PSRAM not available"); return false; } #endif if (!bridge || !bridge->isRunning()) { strcpy(reply, "Err - MQTT bridge not running"); return false; } diff --git a/examples/simple_room_server/MyMesh.cpp b/examples/simple_room_server/MyMesh.cpp index 302db12d7..f2e447c6e 100644 --- a/examples/simple_room_server/MyMesh.cpp +++ b/examples/simple_room_server/MyMesh.cpp @@ -1477,7 +1477,7 @@ void MyMesh::handleCommand(uint32_t sender_timestamp, char *command, char *reply } #elif defined(WITH_MQTT_BRIDGE) } else if (memcmp(command, "discover.scopes", 15) == 0) { - strcpy(reply, "Err - not supported (requires PSRAM)"); + strcpy(reply, "Err - neighbors not enabled in this build"); #endif } else if (strncmp(command, "room.post", 9) == 0) { char* msg = command + 9; @@ -1905,34 +1905,61 @@ static bool neighborPublishEntryComesBefore( } #if defined(ESP_PLATFORM) +// Neighbors allocations prefer PSRAM where it exists and otherwise come from +// internal DRAM, so MQTT_NEIGHBORS_WITHOUT_PSRAM boards can build the table too. +#if defined(BOARD_HAS_PSRAM) +static const uint32_t kNeighborsAllocCaps = MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT; +#else +static const uint32_t kNeighborsAllocCaps = MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT; +#endif + +static void* neighborsAlloc(size_t size) { + if (size == 0) return nullptr; + void* p = heap_caps_malloc(size, kNeighborsAllocCaps); +#if defined(BOARD_HAS_PSRAM) + if (!p) p = heap_caps_malloc(size, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT); +#endif + return p; +} + +static void neighborsFree(void* ptr) { + if (ptr) heap_caps_free(ptr); +} + // ArduinoJson v7 JsonDocument has no real capacity cap (DynamicJsonDocument(N) -// is a no-op shim). Keep the pool off internal DRAM and soft-cap peak growth to -// the publish buffer size. used only rises on allocate — conservative for this -// single-shot doc (overflow path removes+breaks, so no further growth after free). +// is a no-op shim), so soft-cap peak pool growth to NEIGHBORS_DOC_POOL_BUDGET. +// used only rises on allocate — conservative for this single-shot doc (overflow +// path removes+breaks, so no further growth after free). struct NeighborsDocAllocator : ArduinoJson::Allocator { size_t used = 0; static const size_t kBudget = MQTTBridge::NEIGHBORS_DOC_POOL_BUDGET; void* allocate(size_t size) override { if (used >= kBudget || size > kBudget - used) return nullptr; - void* p = heap_caps_malloc(size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT); + void* p = neighborsAlloc(size); if (p) used += size; return p; } void deallocate(void* ptr) override { - heap_caps_free(ptr); + neighborsFree(ptr); } void* reallocate(void* ptr, size_t new_size) override { size_t old_size = ptr ? heap_caps_get_allocated_size(ptr) : 0; size_t next_used = (used >= old_size) ? (used - old_size) : 0; if (next_used >= kBudget || new_size > kBudget - next_used) return nullptr; - void* p = heap_caps_realloc(ptr, new_size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT); + void* p = heap_caps_realloc(ptr, new_size, kNeighborsAllocCaps); +#if defined(BOARD_HAS_PSRAM) + if (!p) p = heap_caps_realloc(ptr, new_size, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT); +#endif if (p) used = next_used + new_size; return p; } }; +#else +static void* neighborsAlloc(size_t size) { return size ? malloc(size) : nullptr; } +static void neighborsFree(void* ptr) { free(ptr); } #endif // Build the neighbors-table JSON and hand it to the bridge, then reschedule. @@ -1943,80 +1970,69 @@ void MyMesh::finishNeighborDiscover() { char timestamp[40]; MQTTMessageBuilder::formatIsoTimestampForMqtt(getRTCClock()->getCurrentTime(), 0, nullptr, timestamp, sizeof(timestamp)); - char pubkey_hex[MAX_NEIGHBOURS][65]; - MQTTMessageBuilder::NeighborsMessageEntry entries[MAX_NEIGHBOURS]; - uint32_t now_secs = getRTCClock()->getCurrentTime(); + // The entry table plus one hex string each reaches ~4.5 KB at MAX_NEIGHBOURS, + // which does not fit the mesh loop task's 8 KB stack, so both share a single + // heap block sized to this pass. Publishing is skipped if either alloc fails. + const int publish_count = neighbor_discover_publish_count; + const size_t hex_size = PUB_KEY_SIZE * 2 + 1; + const size_t entries_bytes = + sizeof(MQTTMessageBuilder::NeighborsMessageEntry) * publish_count; + void* scratch = neighborsAlloc(entries_bytes + hex_size * publish_count); + char* json_buf = (char*)neighborsAlloc(MQTTBridge::NEIGHBORS_JSON_BUFFER_SIZE); - for (int i = 0; i < neighbor_discover_publish_count; i++) { - auto& entry = neighbor_discover[i]; - mesh::Utils::toHex(pubkey_hex[i], entry.id.pub_key, PUB_KEY_SIZE); - entries[i].pubkey_hex = pubkey_hex[i]; - entries[i].snr = entry.snr / 4.0f; - bool heard_known = neighborHeardAgeUsable(entry.heard_timestamp, now_secs); - entries[i].heard_unknown = !heard_known; - entries[i].heard_secs_ago = heard_known ? (now_secs - entry.heard_timestamp) : 0; - entries[i].scopes = entry.scopes; - switch (entry.status) { - case ND_RESPONDED: entries[i].status = "responded"; break; - case ND_SEND_FAILED: entries[i].status = "send_failed"; break; - default: entries[i].status = "timeout"; break; + if (json_buf && (scratch || publish_count == 0)) { + auto* entries = (MQTTMessageBuilder::NeighborsMessageEntry*)scratch; + char* pubkey_hex = (char*)scratch + entries_bytes; + uint32_t now_secs = getRTCClock()->getCurrentTime(); + + for (int i = 0; i < publish_count; i++) { + auto& entry = neighbor_discover[i]; + char* hex = &pubkey_hex[i * hex_size]; + mesh::Utils::toHex(hex, entry.id.pub_key, PUB_KEY_SIZE); + entries[i].pubkey_hex = hex; + entries[i].snr = entry.snr / 4.0f; + bool heard_known = neighborHeardAgeUsable(entry.heard_timestamp, now_secs); + entries[i].heard_unknown = !heard_known; + entries[i].heard_secs_ago = heard_known ? (now_secs - entry.heard_timestamp) : 0; + entries[i].scopes = entry.scopes; + switch (entry.status) { + case ND_RESPONDED: entries[i].status = "responded"; break; + case ND_SEND_FAILED: entries[i].status = "send_failed"; break; + default: entries[i].status = "timeout"; break; + } + } + + // insertion sort: most useful first (JSON builder drops the tail on overflow) + for (int i = 1; i < publish_count; i++) { + MQTTMessageBuilder::NeighborsMessageEntry entry = entries[i]; + int j = i; + while (j > 0 && neighborPublishEntryComesBefore(entry, entries[j - 1])) { + entries[j] = entries[j - 1]; + j--; + } + entries[j] = entry; + } + +#if defined(ESP_PLATFORM) + NeighborsDocAllocator doc_alloc; + JsonDocument doc(&doc_alloc); +#else + JsonDocument doc; +#endif + int json_len = MQTTMessageBuilder::buildNeighborsMessage( + doc, neighbor_discover_origin, self_pubkey_hex, timestamp, self_scopes_buf, + self_default_scope_buf, entries, publish_count, + json_buf, MQTTBridge::NEIGHBORS_JSON_BUFFER_SIZE, + neighbor_discover_count, neighbor_discover_queried_count, + neighbor_discover_truncated); + + if (json_len > 0 && bridge) { + bridge->requestPublishNeighbors(json_buf, (size_t)json_len); } } - // insertion sort: most useful first (JSON builder drops the tail on overflow) - for (int i = 1; i < neighbor_discover_publish_count; i++) { - MQTTMessageBuilder::NeighborsMessageEntry entry = entries[i]; - int j = i; - while (j > 0 && neighborPublishEntryComesBefore(entry, entries[j - 1])) { - entries[j] = entries[j - 1]; - j--; - } - entries[j] = entry; - } - -#if defined(ESP_PLATFORM) - char* json_buf = (char*)heap_caps_malloc(MQTTBridge::NEIGHBORS_JSON_BUFFER_SIZE, MALLOC_CAP_SPIRAM); -#else - char* json_buf = (char*)malloc(MQTTBridge::NEIGHBORS_JSON_BUFFER_SIZE); -#endif - if (!json_buf) { - neighbor_discover_active = false; - neighbor_discover_count = 0; - neighbor_discover_next = 0; - neighbor_discover_publish_count = 0; - neighbor_discover_queried_count = 0; - neighbor_discover_json_size = 0; - neighbor_discover_truncated = false; - neighbor_discover_until = 0; - neighbor_discover_request = NULL; - if (_cli.getObserverPrefs()->mqtt_neighbors_enabled) { - next_neighbors_publish = futureMillis(_cli.getObserverPrefs()->mqtt_neighbors_interval); - } - return; - } - -#if defined(ESP_PLATFORM) - NeighborsDocAllocator doc_alloc; - JsonDocument doc(&doc_alloc); -#else - JsonDocument doc; -#endif - int json_len = MQTTMessageBuilder::buildNeighborsMessage( - doc, neighbor_discover_origin, self_pubkey_hex, timestamp, self_scopes_buf, - self_default_scope_buf, entries, neighbor_discover_publish_count, - json_buf, MQTTBridge::NEIGHBORS_JSON_BUFFER_SIZE, - neighbor_discover_count, neighbor_discover_queried_count, - neighbor_discover_truncated); - - if (json_len > 0 && bridge) { - bridge->requestPublishNeighbors(json_buf, (size_t)json_len); - } - -#if defined(ESP_PLATFORM) - heap_caps_free(json_buf); -#else - free(json_buf); -#endif + neighborsFree(scratch); + neighborsFree(json_buf); neighbor_discover_active = false; neighbor_discover_count = 0; @@ -2088,9 +2104,12 @@ void MyMesh::loopNeighborDiscover() { } } -// Shared precondition for starting a discovery: PSRAM present + bridge running. +// Shared precondition for starting a discovery: usable buffers + bridge running. +// PSRAM builds size their neighbors buffers for PSRAM, so a board whose PSRAM +// failed to init must not silently spend that much internal DRAM here. +// MQTT_NEIGHBORS_WITHOUT_PSRAM builds are already sized for internal DRAM. bool MyMesh::neighborDiscoverReady(char* reply) { -#if defined(ESP_PLATFORM) +#if defined(ESP_PLATFORM) && defined(BOARD_HAS_PSRAM) if (!psramFound()) { strcpy(reply, "Err - PSRAM not available"); return false; } #endif if (!bridge || !bridge->isRunning()) { strcpy(reply, "Err - MQTT bridge not running"); return false; } diff --git a/src/helpers/CommonCLI_Observer.cpp b/src/helpers/CommonCLI_Observer.cpp index eb75a273b..c1b7e74a1 100644 --- a/src/helpers/CommonCLI_Observer.cpp +++ b/src/helpers/CommonCLI_Observer.cpp @@ -289,7 +289,7 @@ bool CommonCLI::handleObserverSetCmd(uint32_t sender_timestamp, const char* conf #elif defined(WITH_MQTT_BRIDGE) } else if (memcmp(config, "mqtt.neighbors.interval ", 24) == 0 || memcmp(config, "mqtt.neighbors ", 15) == 0) { - strcpy(reply, "Err - not supported (requires PSRAM)"); + strcpy(reply, "Err - neighbors not enabled in this build"); #endif } else if (memcmp(config, "mqtt.ntp ", 9) == 0) { const char* host = &config[9]; @@ -837,7 +837,7 @@ bool CommonCLI::handleObserverGetCmd(uint32_t sender_timestamp, const char* conf #elif defined(WITH_MQTT_BRIDGE) } else if (memcmp(config, "mqtt.neighbors.interval", 23) == 0 || memcmp(config, "mqtt.neighbors", 14) == 0) { - strcpy(reply, "Err - not supported (requires PSRAM)"); + strcpy(reply, "Err - neighbors not enabled in this build"); #endif } else if (memcmp(config, "mqtt.ntp.diag", 13) == 0 && (config[13] == '\0' || config[13] == ' ')) { #ifdef ESP_PLATFORM diff --git a/src/helpers/bridges/MQTTBridge.cpp b/src/helpers/bridges/MQTTBridge.cpp index 0da28d581..9f1ecab3d 100644 --- a/src/helpers/bridges/MQTTBridge.cpp +++ b/src/helpers/bridges/MQTTBridge.cpp @@ -727,18 +727,23 @@ void MQTTBridge::allocateRuntimeBuffers() { _publish_json_buffer, PUBLISH_JSON_BUFFER_SIZE, psram_malloc)); _status_json_buffer = static_cast(MQTTRuntimeBufferLifecycle::allocateIfMissing( _status_json_buffer, STATUS_JSON_BUFFER_SIZE, psram_malloc)); -#if defined(WITH_MQTT_NEIGHBORS) - // Persistent neighbors JSON buffer. Unlike status/packet there is no stack - // fallback: the feature is PSRAM-gated, so a nullptr simply disables publishing - // (requestPublishNeighbors/publishNeighbors both no-op on nullptr). - _neighbors_json_buffer = static_cast(MQTTRuntimeBufferLifecycle::allocateIfMissing( - _neighbors_json_buffer, NEIGHBORS_JSON_BUFFER_SIZE, psram_malloc)); -#endif MQTT_DEBUG_PRINTLN("Runtime buffers: raw=%s publish=%s status=%s", _last_raw_data ? "PSRAM" : "unavailable", _publish_json_buffer ? "PSRAM" : "stack fallback", _status_json_buffer ? "PSRAM" : "stack fallback"); #endif + +#if defined(WITH_MQTT_NEIGHBORS) + // Persistent neighbors JSON buffer, heap-allocated on every board: too large to + // keep inline in the bridge object the way the non-PSRAM status/packet buffers + // are. psram_malloc() falls back to internal DRAM, so this works without PSRAM. + // Unlike status/packet there is no stack fallback — a nullptr simply disables + // publishing (requestPublishNeighbors/publishNeighbors both no-op on nullptr). + _neighbors_json_buffer = static_cast(MQTTRuntimeBufferLifecycle::allocateIfMissing( + _neighbors_json_buffer, NEIGHBORS_JSON_BUFFER_SIZE, psram_malloc)); + MQTT_DEBUG_PRINTLN("Neighbors buffer: %s", + _neighbors_json_buffer ? "ready" : "unavailable"); +#endif } void MQTTBridge::releaseRuntimeBuffers() { @@ -749,13 +754,15 @@ void MQTTBridge::releaseRuntimeBuffers() { _publish_json_buffer, psram_free)); _status_json_buffer = static_cast(MQTTRuntimeBufferLifecycle::release( _status_json_buffer, psram_free)); + #endif + #if defined(WITH_MQTT_NEIGHBORS) + // Paired with the unconditional allocation in allocateRuntimeBuffers(). _neighbors_json_buffer = static_cast(MQTTRuntimeBufferLifecycle::release( _neighbors_json_buffer, psram_free)); _neighbors_publish_len = 0; _neighbors_publish_pending.store(false, std::memory_order_release); #endif - #endif // Never pair a newly allocated raw buffer with metadata from a prior bridge // run. This also makes non-PSRAM restarts discard their stale raw cache. diff --git a/src/helpers/bridges/MQTTBridge.h b/src/helpers/bridges/MQTTBridge.h index 446d864ad..143aae36b 100644 --- a/src/helpers/bridges/MQTTBridge.h +++ b/src/helpers/bridges/MQTTBridge.h @@ -38,11 +38,15 @@ class MeshSNMPAgent; // Forward declaration #ifdef WITH_MQTT_BRIDGE -// Periodic neighbors publication is PSRAM-only: it needs a persistent ~10 KB JSON -// buffer plus a second transient one while the mesh builds the table, and it keys -// off the mesh neighbor cache (sized by MAX_NEIGHBOURS). Every neighbors-specific -// member, method, and code block in this bridge is gated on WITH_MQTT_NEIGHBORS. -#if defined(BOARD_HAS_PSRAM) && defined(MAX_NEIGHBOURS) && MAX_NEIGHBOURS > 0 +// Periodic neighbors publication keys off the mesh neighbor cache (sized by +// MAX_NEIGHBOURS) and needs a persistent JSON buffer plus a second transient one +// while the mesh builds the table. PSRAM boards get it automatically; non-PSRAM +// boards must opt in per variant with MQTT_NEIGHBORS_WITHOUT_PSRAM, which spends +// internal DRAM the TLS stack also needs (see NEIGHBORS_JSON_BUFFER_SIZE). Every +// neighbors-specific member, method, and code block here is gated on +// WITH_MQTT_NEIGHBORS. +#if defined(MAX_NEIGHBOURS) && MAX_NEIGHBOURS > 0 && \ + (defined(BOARD_HAS_PSRAM) || defined(MQTT_NEIGHBORS_WITHOUT_PSRAM)) #define WITH_MQTT_NEIGHBORS 1 #endif @@ -513,17 +517,38 @@ public: #if defined(WITH_MQTT_NEIGHBORS) // Single source of truth for the neighbors JSON size, used by both the bridge's // persistent buffer and the mesh's transient build buffer. + #if defined(BOARD_HAS_PSRAM) static const size_t NEIGHBORS_JSON_BUFFER_SIZE = 10240; + #else + // Without PSRAM all three neighbors allocations (persistent buffer, transient + // build buffer, ArduinoJson pool) come out of internal DRAM, which each TLS + // slot also needs ~40 KB of. A quarter-size buffer keeps the peak near 13 KB + // instead of ~35 KB; the builder drops the tail and sets "truncated" once the + // next entry will not fit, so the table degrades instead of failing. + static const size_t NEIGHBORS_JSON_BUFFER_SIZE = 4096; + #endif // The ArduinoJson pool is NOT bounded by the text buffer: v7 hands out pool // blocks in fixed 4096-byte chunks, so a table that just fits the text buffer // can still need well over it in pool. Budgeting the pool at the text size // starves it, and a starved pool sets doc.overflowed() — which drops the whole - // publish instead of truncating. Measured need is 12541 B for 50 entries. + // publish instead of truncating. Measured need is 12541 B for 50 entries and + // 4286 B for 21, so allow a spare block over each. + #if defined(BOARD_HAS_PSRAM) static const size_t NEIGHBORS_DOC_POOL_BUDGET = 16384; + #else + static const size_t NEIGHBORS_DOC_POOL_BUDGET = 12288; + #endif - // Entries per publish; the neighbour cache and text buffer are the real limit. + // Entries per publish. One 4096-byte pool block holds 21, so staying under + // that keeps the non-PSRAM pool to a single block (~4.3 KB rather than + // ~8.4 KB) and the whole publish peak near 13 KB of internal DRAM. PSRAM + // boards are limited only by the text buffer and the neighbour cache. + #if defined(BOARD_HAS_PSRAM) static const int NEIGHBORS_MAX_PUBLISH_ENTRIES = MAX_NEIGHBOURS; + #else + static const int NEIGHBORS_MAX_PUBLISH_ENTRIES = 20; + #endif // Called by the mesh (Core 1) once a neighbor-discovery pass has built the // table JSON. Copies it into the persistent PSRAM buffer and raises the diff --git a/variants/heltec_tracker_v2/platformio.ini b/variants/heltec_tracker_v2/platformio.ini index 63aab2972..03169c05f 100644 --- a/variants/heltec_tracker_v2/platformio.ini +++ b/variants/heltec_tracker_v2/platformio.ini @@ -185,6 +185,8 @@ build_flags = -D ADMIN_PASSWORD='"password"' -D MAX_NEIGHBOURS=50 -D WITH_MQTT_BRIDGE=1 +; Non-PSRAM board: neighbors table costs ~4 KB of internal DRAM (see MQTTBridge.h). + -D MQTT_NEIGHBORS_WITHOUT_PSRAM=1 -D MAX_MQTT_BROKERS=3 -D MQTT_MAX_PACKET_SIZE=1024 -D MQTT_DEBUG=1 @@ -221,6 +223,8 @@ build_flags = -D ADMIN_PASSWORD='"password"' -D MAX_NEIGHBOURS=50 -D WITH_MQTT_BRIDGE=1 +; Non-PSRAM board: neighbors table costs ~4 KB of internal DRAM (see MQTTBridge.h). + -D MQTT_NEIGHBORS_WITHOUT_PSRAM=1 -D MAX_MQTT_BROKERS=3 -D MQTT_MAX_PACKET_SIZE=1024 -D MQTT_DEBUG=1 @@ -257,6 +261,8 @@ build_flags = -D ADMIN_PASSWORD='"password"' -D ROOM_PASSWORD='"hello"' -D WITH_MQTT_BRIDGE=1 +; Non-PSRAM board: neighbors table costs ~4 KB of internal DRAM (see MQTTBridge.h). + -D MQTT_NEIGHBORS_WITHOUT_PSRAM=1 -D MAX_NEIGHBOURS=50 -D MAX_MQTT_BROKERS=3 -D MQTT_MAX_PACKET_SIZE=1024 @@ -294,6 +300,8 @@ build_flags = -D ADMIN_PASSWORD='"password"' -D ROOM_PASSWORD='"hello"' -D WITH_MQTT_BRIDGE=1 +; Non-PSRAM board: neighbors table costs ~4 KB of internal DRAM (see MQTTBridge.h). + -D MQTT_NEIGHBORS_WITHOUT_PSRAM=1 -D MAX_NEIGHBOURS=50 -D MAX_MQTT_BROKERS=3 -D MQTT_MAX_PACKET_SIZE=1024 diff --git a/variants/heltec_v3/platformio.ini b/variants/heltec_v3/platformio.ini index 1c6a72606..305d5782c 100644 --- a/variants/heltec_v3/platformio.ini +++ b/variants/heltec_v3/platformio.ini @@ -123,6 +123,8 @@ build_flags = -D ADMIN_PASSWORD='"password"' -D MAX_NEIGHBOURS=50 -D WITH_MQTT_BRIDGE=1 +; Non-PSRAM board: neighbors table costs ~4 KB of internal DRAM (see MQTTBridge.h). + -D MQTT_NEIGHBORS_WITHOUT_PSRAM=1 -D MAX_MQTT_BROKERS=3 -D MQTT_MAX_PACKET_SIZE=1024 -D MQTT_DEBUG=1 @@ -203,6 +205,8 @@ build_flags = -D ADMIN_PASSWORD='"password"' -D ROOM_PASSWORD='"hello"' -D WITH_MQTT_BRIDGE=1 +; Non-PSRAM board: neighbors table costs ~4 KB of internal DRAM (see MQTTBridge.h). + -D MQTT_NEIGHBORS_WITHOUT_PSRAM=1 -D MAX_NEIGHBOURS=50 -D MAX_MQTT_BROKERS=3 -D MQTT_MAX_PACKET_SIZE=1024 @@ -497,6 +501,8 @@ build_flags = -D ADMIN_PASSWORD='"password"' -D MAX_NEIGHBOURS=50 -D WITH_MQTT_BRIDGE=1 +; Non-PSRAM board: neighbors table costs ~4 KB of internal DRAM (see MQTTBridge.h). + -D MQTT_NEIGHBORS_WITHOUT_PSRAM=1 -D MAX_MQTT_BROKERS=3 -D MQTT_MAX_PACKET_SIZE=1024 -D MQTT_DEBUG=1 @@ -546,6 +552,8 @@ build_flags = -D ADMIN_PASSWORD='"password"' -D ROOM_PASSWORD='"hello"' -D WITH_MQTT_BRIDGE=1 +; Non-PSRAM board: neighbors table costs ~4 KB of internal DRAM (see MQTTBridge.h). + -D MQTT_NEIGHBORS_WITHOUT_PSRAM=1 -D MAX_NEIGHBOURS=50 -D MAX_MQTT_BROKERS=3 -D MQTT_MAX_PACKET_SIZE=1024 diff --git a/variants/rak3112/platformio.ini b/variants/rak3112/platformio.ini index 200f5d20b..6cc8b9148 100644 --- a/variants/rak3112/platformio.ini +++ b/variants/rak3112/platformio.ini @@ -113,6 +113,8 @@ build_flags = -D ADMIN_PASSWORD='"password"' -D MAX_NEIGHBOURS=50 -D WITH_MQTT_BRIDGE=1 +; Non-PSRAM board: neighbors table costs ~4 KB of internal DRAM (see MQTTBridge.h). + -D MQTT_NEIGHBORS_WITHOUT_PSRAM=1 -D MAX_MQTT_BROKERS=3 -D MQTT_MAX_PACKET_SIZE=1024 -D MQTT_DEBUG=1 @@ -176,6 +178,8 @@ build_flags = -D ADMIN_PASSWORD='"password"' -D ROOM_PASSWORD='"hello"' -D WITH_MQTT_BRIDGE=1 +; Non-PSRAM board: neighbors table costs ~4 KB of internal DRAM (see MQTTBridge.h). + -D MQTT_NEIGHBORS_WITHOUT_PSRAM=1 -D MAX_NEIGHBOURS=50 -D MAX_MQTT_BROKERS=3 -D MQTT_MAX_PACKET_SIZE=1024