Merge pull request #38 from agessaman/feat/packet-filters

Enhance MQTT slot filters with named types and downgrade support
This commit is contained in:
Adam Gessaman authored and GitHub committed 2026-07-28 09:10:46 -07:00
commit 971a3b1464
20 files changed
+1414 -66

No files matched your search

+64
View File
@@ -248,6 +248,7 @@ The MQTT bridge comes with the following defaults for fresh installs (unless ove
- **Slot 1**: `analyzer-us`
- **Slot 2**: `analyzer-eu`
- **Slots 3-6**: `none` (disabled)
- **Per-slot packet filters**: `all` (every payload type is uploaded)
- **WiFi SSID**: (blank — must be configured)
- **WiFi Password**: (blank — optional for open networks)
- **WiFi Power Save**: `none` (no power save)
@@ -272,6 +273,7 @@ Each slot (1-6) supports the following commands:
- `get mqttN.token` - Get per-slot token (e.g., MeshRank account token)
- `get mqttN.topic` - Get custom topic template for slot N
- `get mqttN.audience` - Get JWT audience for slot N (custom slots only)
- `get mqttN.filter` - Get the slot's packet-type allowlist (`all`, `none`, or numeric CSV)
#### Set Commands
- `set mqttN.preset <name>` - Set slot N to a built-in preset. Use any `name` from the [preset table](#slot-based-preset-system) (run `get mqtt.presets` on-device for the full list). Most presets need no further configuration; the exceptions are:
@@ -287,9 +289,71 @@ Each slot (1-6) supports the following commands:
- `set mqttN.topic <template>` - Set custom topic template (custom preset only, see below)
- `set mqttN.audience <audience>` - Set JWT audience for custom slot (enables Ed25519 JWT auth)
- `set mqttN.audience` - Clear JWT audience (reverts to username/password auth)
- `set mqttN.filter <all|none|list>` - Select payload types uploaded to this slot
**Note:** Custom server/port settings only apply when the slot's preset is `custom`. Username/password also apply to built-in presets that use per-slot credentials (e.g. `inwmesh`); other userpass presets (`tennmesh`, `nashmesh`, `ctmesh`) ship fixed credentials in firmware.
#### Per-broker packet filters
Each slot has an independent allowlist. List entries may be payload-type names
or numbers, and the two can be mixed. These are all equivalent, sending only
text messages and adverts to slot 1:
```bash
set mqtt1.filter txt_msg,advert
set mqtt1.filter 2,4
set mqtt1.filter advert, 2
```
Use `none` when a broker should remain connected for status/neighbors but
receive no packet traffic. A bare `set mqttN.filter` resets the slot to `all`.
`get mqttN.filter` always answers in the canonical numeric form (`all`, `none`,
or an ascending CSV), whichever spelling was used to set it.
| Type | Name | Type | Name |
|------|------|------|------|
| 0 | `req` | 8 | `path` |
| 1 | `response` | 9 | `trace` |
| 2 | `txt_msg` | 10 | `multipart` |
| 3 | `ack` | 11 | `control` |
| 4 | `advert` | 12-14 | reserved (number only) |
| 5 | `grp_txt` | 15 | `raw_custom` |
| 6 | `grp_data` | | |
| 7 | `anon_req` | | |
Names are lowercase and exact; types 12-14 are reserved upstream and have no
name, so they are selectable by number only.
The filter applies to both structured `packets` and `raw` publications for RX
packets and for TX packets permitted by `mqtt.tx`. It does not affect local
packet processing, forwarding, capture logs, status, or neighbors. Changes
apply live without reconnecting the broker.
A rejected packet is dropped before it is copied into the publish queue, so a
narrow filter saves the queue slot and the per-packet work, not just the
upload. `get mqtt.stats` reports the running count as `filt=<n>`, and a slot
whose filter is not `all` shows it in `get mqttN.diag` and on the WebConfig
Stats tab — a filtered slot otherwise looks identical to an idle healthy one.
The diag reply is capped at 160 characters. When a slot is also reporting a
long error tail, the filter is summarised as `filter:<n>/16` rather than
listed, because a list clipped mid-way would read as a different, valid
allowlist. `get mqttN.filter` always gives the exact value.
In WebConfig the allowlist is a checkbox per type under each configured slot,
with **All** / **None** shortcuts. Clearing every box is `none` (nothing
uploaded).
**Downgrade note:** rolling back to any build from this release onward is safe —
the older firmware reads the settings it understands and simply ignores the
packet filters, which revert to `all` if it saves.
Rolling back to a build released *before* this one is the case to watch: that
firmware rejects the longer settings file outright and falls back to defaults,
losing the stored WiFi credentials along with the broker config. Slots left at
the `all` default keep the file in the shorter layout those builds can read, so
if you may need to roll a node back that far, reset every slot to `all` first.
#### Example: Configure MeshRank on Slot 3
```bash
set mqtt3.preset meshrank
+42 -3
View File
@@ -100,11 +100,50 @@ fall back to defaults (no downgrade, no misread). `saveMQTTPrefs()` also refuses
write while such a file is present (`_mqtt_prefs_hold`), so a `set` command after a
firmware downgrade can't clobber the newer config — observer settings changed in that
state simply don't persist. The frozen legacy layouts are pinned with `static_assert`s
in `CommonCLI.h`, so every target build re-verifies the fleet's file offsets.
in `MQTTPrefsStorage.h`, so every target build re-verifies the fleet's file offsets.
Adding a field to the current version stays backward compatible: append it to the end
of `MQTTPrefs`. An older, shorter payload still loads and the missing tail keeps its
default; a newer, longer one is truncated harmlessly.
of `MQTTPrefs`, give the older exact payload length an explicit decoder boundary, and
leave the missing tail at its default. The packet-filter addition follows that rule:
the prior 2864-byte v1 payload loads with all six filters set to `all`, while the
full payload is 2876 bytes.
#### The downgrade contract
**Within a version tag the layout is append-only, and a longer payload is always
readable.** A file written by a later build starts with this binary's exact baseline,
so `classify()` reads that prefix and ignores the tail. A downgraded node keeps its
WiFi credentials, broker slots, and every other setting it understands; the only thing
it loses is the settings the newer build added.
That asymmetry is the whole point. Refusing the file costs the operator the network
itself — `/mqtt_prefs` holds `wifi_ssid`/`wifi_password` as well as the broker config,
so a node that falls back to defaults has no WiFi, no portal, and no OTA, recoverable
only over serial. Reading it costs a feature's settings. Losing later settings is the
acceptable half of that trade; losing the node is not.
The tail survives until something actually writes. `saveMQTTPrefs()` rewrites at this
binary's own length, so a rollback that changes no observer setting and is later rolled
forward keeps the newer fields intact — only an explicit `set` while downgraded drops
them. The boot log says so when it happens.
**A change that is not a pure append MUST bump `MQTT_PREFS_VERSION`.** The version
check is what makes the rule above safe: a different tag is still refused outright and
the file preserved, because the bytes may no longer mean what this binary thinks. Never
reorder, resize, or repurpose an existing field within a version.
Note the rule is only as old as the build that implements it. Firmware already deployed
carries the *previous* decoder, which rejects any longer v1 payload — so rolling back
from this build to one shipped before it still falls back to defaults.
`MQTTPrefsCodec::payloadLenFor()` covers that gap from the writing side: it returns the
shortest length that still round-trips the configuration, so a node keeps writing 2864
bytes until a packet filter actually holds something, and clearing the last non-default
filter puts it back. That mitigation can be retired once no supported downgrade target
predates the contract; the contract itself is the durable half.
Shorter payloads keep their existing, stricter treatment: a short length must match a
boundary that really shipped (`MQTT_PREFS_V1_*_PAYLOAD_SIZE`), because raw prefs have
no checksum and an arbitrary short size cannot be trusted to mean anything.
### Settings upgrade / migration
+10 -2
View File
@@ -1515,9 +1515,17 @@ void MyMesh::buildStatsJson(char* buf, size_t buf_size) {
for (int i = 0; i < RUNTIME_MQTT_SLOTS; i++) {
MQTTBridge::SlotStatusSnapshot s;
if (!MQTTBridge::getSlotStatusSnapshot(i, &s)) continue;
// "filt" is omitted for the all-types default, so the portal only has to
// render the exception and the JSON stays inside the stats buffer.
char filt[24];
filt[0] = '\0';
if (s.filter_mask != MQTTPacketFilter::kAllPacketTypes) {
snprintf(filt, sizeof(filt), ",\"filt\":%u", (unsigned)s.filter_mask);
}
int n = snprintf(buf + pos, buf_size - pos,
"%s{\"n\":%d,\"name\":\"%s\",\"state\":\"%s\",\"ok\":%lu,\"err\":%lu}",
first ? "" : ",", i + 1, s.name, s.state, s.publish_ok, s.publish_err);
"%s{\"n\":%d,\"name\":\"%s\",\"state\":\"%s\",\"ok\":%lu,\"err\":%lu%s}",
first ? "" : ",", i + 1, s.name, s.state, s.publish_ok, s.publish_err,
filt);
if (n < 0 || n >= (int)(buf_size - pos)) break;
pos += n;
first = false;
+10 -2
View File
@@ -1310,9 +1310,17 @@ void MyMesh::buildStatsJson(char* buf, size_t buf_size) {
for (int i = 0; i < RUNTIME_MQTT_SLOTS; i++) {
MQTTBridge::SlotStatusSnapshot s;
if (!MQTTBridge::getSlotStatusSnapshot(i, &s)) continue;
// "filt" is omitted for the all-types default, so the portal only has to
// render the exception and the JSON stays inside the stats buffer.
char filt[24];
filt[0] = '\0';
if (s.filter_mask != MQTTPacketFilter::kAllPacketTypes) {
snprintf(filt, sizeof(filt), ",\"filt\":%u", (unsigned)s.filter_mask);
}
int n = snprintf(buf + pos, buf_size - pos,
"%s{\"n\":%d,\"name\":\"%s\",\"state\":\"%s\",\"ok\":%lu,\"err\":%lu}",
first ? "" : ",", i + 1, s.name, s.state, s.publish_ok, s.publish_err);
"%s{\"n\":%d,\"name\":\"%s\",\"state\":\"%s\",\"ok\":%lu,\"err\":%lu%s}",
first ? "" : ",", i + 1, s.name, s.state, s.publish_ok, s.publish_err,
filt);
if (n < 0 || n >= (int)(buf_size - pos)) break;
pos += n;
first = false;
+78 -3
View File
@@ -47,9 +47,17 @@ LEN_LIMITS = {
"name": 31, "wifi.ssid": 31, "wifi.pwd": 63, "mqtt.origin": 31,
"mqtt.email": 63, "mqtt.ntp": 63, "timezone": 31, "snmp.community": 23,
}
# "filter" is absent on purpose: it is a bitmask, not a text buffer, so it has
# no destination-buffer limit. It is still bounded by the shared CLI command
# budget below, like every other key.
SLOT_LEN_LIMITS = {"server": 63, "username": 31, "password": 63,
"token": 47, "topic": 95, "audience": 63}
# BatchEntry::cmd[160] in WebConfigServer.cpp holds "set <key> <value>" plus a
# NUL. Over-long values are rejected there rather than truncated, because a
# clipped value can still be valid and would persist as a different setting.
BATCH_CMD_SIZE = 160
# Preset names + what the UI must collect (mirrors handlePresets()).
PRESETS = (
[(n, "none") for n in (
@@ -99,7 +107,8 @@ def default_config(setup_mode):
def _slot():
return {"preset": "none", "server": "", "port": 8883, "username": "",
"password": "", "token": "", "topic": "", "audience": ""}
"password": "", "token": "", "topic": "", "audience": "",
"filter": "all"}
class State:
@@ -253,6 +262,52 @@ def apply_set(cfg, key, val):
return True, "OK" # unknown-but-allowlisted: accept (mock is lenient here)
# Payload-type names accepted alongside the decimal form. Mirrors
# namedPacketTypes() in src/helpers/MQTTPacketFilter.h; 12-14 are reserved
# upstream and stay reachable by number only.
PACKET_TYPE_NAMES = {
"req": 0, "response": 1, "txt_msg": 2, "ack": 3, "advert": 4,
"grp_txt": 5, "grp_data": 6, "anon_req": 7, "path": 8, "trace": 9,
"multipart": 10, "control": 11, "raw_custom": 15,
}
def packet_filter_mask(text):
"""Canonical filter text -> bitmask, for the stats payload."""
if text == "all":
return 0xFFFF
if text == "none":
return 0
mask = 0
for token in text.split(","):
mask |= 1 << int(token)
return mask
def canonical_packet_filter(val):
"""Mirror of MQTTPacketFilter::parse + ::format. Returns None if invalid."""
stripped = val.strip()
if stripped == "" or stripped == "all":
return "all"
if stripped == "none":
return "none"
mask = 0
for part in stripped.split(","):
token = part.strip()
if re.fullmatch(r"[0-9]+", token):
packet_type = int(token)
if packet_type > 15:
return None
elif token in PACKET_TYPE_NAMES:
packet_type = PACKET_TYPE_NAMES[token]
else:
return None
mask |= 1 << packet_type
if mask == 0xFFFF:
return "all"
return ",".join(str(i) for i in range(16) if mask & (1 << i))
def apply_slot_set(cfg, idx, field, val):
slot = cfg["mqtt"]["slots"][idx]
if field in SLOT_LEN_LIMITS and len(val) > SLOT_LEN_LIMITS[field]:
@@ -266,6 +321,13 @@ def apply_slot_set(cfg, idx, field, val):
return False, "Error: port must be between 1 and 65535"
slot["port"] = p
return True, "OK"
if field == "filter":
canonical = canonical_packet_filter(val)
if canonical is None:
return False, ("Error: filter must be all, none, or a CSV of "
"types 0-15 / names (advert,txt_msg,...)")
slot["filter"] = canonical
return True, "OK - slot %d packet types: %s" % (idx + 1, canonical)
if field in ("preset", "server", "username", "password", "token", "topic", "audience"):
slot[field] = val
if field == "token":
@@ -424,6 +486,13 @@ class Handler(BaseHTTPRequestHandler):
# drop unchanged secrets (sentinel), like the firmware does
entries = [(k, v) for k, v in setmap.items()
if not (is_secret_key(k) and v == SENTINEL)]
# Same command-budget rejection the firmware applies while building
# BatchEntry::cmd; CR/LF are stripped there and don't count.
for k, v in entries:
prefix = "password " if k == "password" else "set %s " % k
stripped = str(v).replace("\r", "").replace("\n", "")
if len(prefix) + len(stripped) > BATCH_CMD_SIZE - 1:
return self._json(400, {"error": "value too long", "key": k[:32]})
if not entries and not reboot:
return self._json(400, {"error": "no changes"})
# apply now, but expose as pending->done to exercise polling
@@ -473,8 +542,14 @@ class Handler(BaseHTTPRequestHandler):
for i, s in enumerate(ST.cfg["mqtt"]["slots"]):
if s["preset"] == "none":
continue
slots.append({"n": i + 1, "name": s["preset"], "state": "ok",
"ok": 100 + up, "err": 0})
row = {"n": i + 1, "name": s["preset"], "state": "ok",
"ok": 100 + up, "err": 0}
# Mirrors buildStatsJson(): "filt" carries the raw mask and is
# omitted entirely for the all-types default.
mask = packet_filter_mask(s.get("filter", "all"))
if mask != 0xFFFF:
row["filt"] = mask
slots.append(row)
return {
"uptime_s": up, "batt_mv": 4020, "heap_free": 142000, "heap_min": 118000,
"heap_max_alloc": 96000, "noise": -98, "rssi": -71, "snr": 9.5,
+16 -3
View File
@@ -793,6 +793,16 @@ void CommonCLI::loadMQTTPrefs(
MESH_DEBUG_PRINTLN("MQTT: /mqtt_prefs read failed, using defaults (file preserved)");
} else {
has_observer_fields = plan.observer_fields_present;
// Written by a later build with appended fields. Everything this
// binary knows loaded normally; say so, because the next `set` will
// rewrite the file at this length and drop the newer settings.
if (file_size - sizeof(MQTTPrefsHeader) > plan.payload_len) {
MESH_DEBUG_PRINTLN(
"MQTT: /mqtt_prefs written by newer firmware (%u > %u bytes); "
"config loaded, newer settings ignored and dropped on next save",
(unsigned)(file_size - sizeof(MQTTPrefsHeader)),
(unsigned)plan.payload_len);
}
}
if (file) file.close();
} else if (plan.rewrite_legacy) {
@@ -944,12 +954,15 @@ bool CommonCLI::saveMQTTPrefs(FILESYSTEM* fs) {
}
// Write header and payload sequentially so the transaction needs no second
// full-size (2.8 KiB) staging buffer on constrained targets.
const MQTTPrefsHeader header = MQTTPrefsCodec::makeHeader();
// full-size (2.8 KiB) staging buffer on constrained targets. The length is
// the shortest that still round-trips this config, so a node with default
// packet filters keeps writing a payload older firmware can read.
const size_t payload_len = MQTTPrefsCodec::payloadLenFor(_mqtt_prefs);
const MQTTPrefsHeader header = MQTTPrefsCodec::makeHeader(payload_len);
MQTTPrefsFileStore store(fs);
const MQTTPrefsAtomicStore::Result result = MQTTPrefsAtomicStore::write(
store, (const uint8_t *)&header, sizeof(header),
(const uint8_t *)&_mqtt_prefs, sizeof(_mqtt_prefs));
(const uint8_t *)&_mqtt_prefs, payload_len);
if (!MQTTPrefsAtomicStore::committed(result)) {
MESH_DEBUG_PRINTLN("MQTT: atomic /mqtt_prefs save failed at %s; source preserved",
mqttPrefsSaveResultName(result));
+35
View File
@@ -25,6 +25,7 @@
#include "bridges/MQTTBridge.h"
#include "MQTTConnectionPolicy.h" // classifySlotActivation() — "will this slot connect here?"
#include "MQTTDefaults.h"
#include "MQTTPacketFilter.h"
#endif
// Local copy of the busted-libc-safe atoi (the original in CommonCLI.cpp is static).
@@ -529,6 +530,32 @@ bool CommonCLI::handleObserverSetCmd(uint32_t sender_timestamp, const char* conf
savePrefs();
_callbacks->restartBridgeSlot(slot);
sprintf(reply, "OK - slot %d JWT audience cleared (using username/password auth)", slot + 1);
} else if (strcmp(subcmd, "filter") == 0 ||
strncmp(subcmd, "filter ", 7) == 0) {
// Empty/bare input resets to the backwards-compatible all-types default.
const char* filter_value = subcmd[6] == '\0' ? "" : &subcmd[7];
uint16_t filter_mask = 0;
if (!MQTTPacketFilter::parse(filter_value, &filter_mask)) {
strcpy(reply, "Error: filter must be all, none, or a CSV of types 0-15 / names (advert,txt_msg,...)");
} else {
_mqtt_prefs.mqtt_slot_packet_filter[slot] = filter_mask;
savePrefs();
char filter_text[MQTTPacketFilter::kFilterTextSize];
MQTTPacketFilter::format(filter_mask, filter_text, sizeof(filter_text));
snprintf(reply, 160, "OK - slot %d packet types: %s", slot + 1, filter_text);
// A non-default filter extends /mqtt_prefs past what pre-filter
// firmware can read (see MQTTPrefsCodec::payloadLenFor), so say when
// that cost buys nothing: slots beyond the runtime array are never
// published to on this board, the same warning `preset` gives.
if (slot >= RUNTIME_MQTT_SLOTS &&
filter_mask != MQTTPacketFilter::kAllPacketTypes) {
size_t used = strlen(reply);
if (used < 158) {
snprintf(reply + used, 160 - used,
" (slot inactive on this hardware; blocks firmware rollback)");
}
}
}
} else {
sprintf(reply, "unknown config: %s", config);
}
@@ -869,6 +896,14 @@ bool CommonCLI::handleObserverGetCmd(uint32_t sender_timestamp, const char* conf
} else {
strcpy(reply, "> (not set - custom slots use username/password auth)");
}
} else if (strcmp(subcmd, "filter") == 0) {
char filter_text[MQTTPacketFilter::kFilterTextSize];
if (MQTTPacketFilter::format(_mqtt_prefs.mqtt_slot_packet_filter[slot],
filter_text, sizeof(filter_text))) {
snprintf(reply, 160, "> %s", filter_text);
} else {
strcpy(reply, "Error: invalid stored packet filter");
}
} else if (memcmp(subcmd, "diag", 4) == 0) {
MQTTBridge::formatSlotDiagReply(reply, 160, slot);
} else {
+4
View File
@@ -5,6 +5,7 @@
#include <string.h>
#include <MeshCore.h>
#include "CommonCLI.h"
#include "MQTTPacketFilter.h"
#include "MQTTPresets.h"
// Compile-time defaults for fresh /mqtt_prefs (override via platformio build_flags).
@@ -82,6 +83,9 @@ static inline void applyMQTTDefaults(MQTTPrefs* prefs) {
MQTT_DEFAULT_SLOT5_PRESET);
mqttDefaultSlotPreset(prefs->mqtt_slot_preset[5], sizeof(prefs->mqtt_slot_preset[5]),
MQTT_DEFAULT_SLOT6_PRESET);
for (int i = 0; i < MQTT_PREFS_SLOT_COUNT; ++i) {
prefs->mqtt_slot_packet_filter[i] = MQTTPacketFilter::kAllPacketTypes;
}
if (MQTT_DEFAULT_IATA[0] != '\0') {
strncpy(prefs->mqtt_iata, MQTT_DEFAULT_IATA, sizeof(prefs->mqtt_iata) - 1);
+226
View File
@@ -0,0 +1,226 @@
#pragma once
#include <stddef.h>
#include <stdint.h>
#include <string.h>
// Pure per-broker packet-type allowlist helpers. MeshCore payload types occupy
// the low four bits of the packet header, so a uint16_t stores the complete
// 0..15 allowlist without dynamic allocation.
namespace MQTTPacketFilter {
static const uint8_t kMinPacketType = 0;
static const uint8_t kMaxPacketType = 15;
static const uint16_t kAllPacketTypes = 0xFFFFu;
// "0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15" plus the terminator.
static const size_t kFilterTextSize = 38;
inline bool isAsciiSpace(char c) {
return c == ' ' || c == '\t' || c == '\r' || c == '\n';
}
inline bool tokenEquals(const char* begin, size_t len, const char* token) {
return token != nullptr && strlen(token) == len && memcmp(begin, token, len) == 0;
}
// Payload-type spellings accepted alongside the decimal form, mirroring the
// PAYLOAD_TYPE_* names in src/Packet.h. Types 12-14 are reserved upstream and
// have no name, so they stay reachable only by number.
struct NamedPacketType {
const char* name;
uint8_t type;
};
inline const NamedPacketType* namedPacketTypes(size_t* count_out) {
static const NamedPacketType kNames[] = {
{"req", 0}, {"response", 1}, {"txt_msg", 2}, {"ack", 3},
{"advert", 4}, {"grp_txt", 5}, {"grp_data", 6}, {"anon_req", 7},
{"path", 8}, {"trace", 9}, {"multipart", 10}, {"control", 11},
{"raw_custom", 15},
};
if (count_out != nullptr) *count_out = sizeof(kNames) / sizeof(kNames[0]);
return kNames;
}
// Resolve one already-trimmed list entry to a payload type. Decimal and named
// spellings are interchangeable within a list; "all"/"none" are whole-value
// keywords and deliberately do not resolve here, so "all,2" stays invalid.
inline bool resolveToken(const char* begin, size_t len, uint8_t* type_out) {
if (begin == nullptr || type_out == nullptr || len == 0) return false;
if (*begin >= '0' && *begin <= '9') {
unsigned value = 0;
for (size_t i = 0; i < len; ++i) {
if (begin[i] < '0' || begin[i] > '9') return false;
value = value * 10u + static_cast<unsigned>(begin[i] - '0');
if (value > kMaxPacketType) return false;
}
*type_out = static_cast<uint8_t>(value);
return true;
}
size_t name_count = 0;
const NamedPacketType* names = namedPacketTypes(&name_count);
for (size_t i = 0; i < name_count; ++i) {
if (tokenEquals(begin, len, names[i].name)) {
*type_out = names[i].type;
return true;
}
}
return false;
}
// Parse an allowlist value. Empty input means "all" so WebConfig can clear a
// field and retain the same backwards-compatible default as an older
// /mqtt_prefs file. Keywords and type names are deliberately lowercase; "all"
// and "none" cannot be mixed into a list. Entries may be decimal (0..15) or
// named, may carry surrounding ASCII whitespace, and may be repeated.
inline bool parse(const char* input, uint16_t* mask_out) {
if (input == nullptr || mask_out == nullptr) return false;
const char* begin = input;
while (*begin && isAsciiSpace(*begin)) begin++;
const char* end = begin + strlen(begin);
while (end > begin && isAsciiSpace(end[-1])) end--;
const size_t len = static_cast<size_t>(end - begin);
if (len == 0 || tokenEquals(begin, len, "all")) {
*mask_out = kAllPacketTypes;
return true;
}
if (tokenEquals(begin, len, "none")) {
*mask_out = 0;
return true;
}
uint16_t parsed = 0;
const char* cursor = begin;
for (;;) {
const char* comma = cursor;
while (comma < end && *comma != ',') comma++;
const char* token_begin = cursor;
const char* token_end = comma;
while (token_begin < token_end && isAsciiSpace(*token_begin)) token_begin++;
while (token_end > token_begin && isAsciiSpace(token_end[-1])) token_end--;
uint8_t type = 0;
if (!resolveToken(token_begin, static_cast<size_t>(token_end - token_begin), &type)) {
return false;
}
parsed |= static_cast<uint16_t>(1u << type);
if (comma >= end) break;
cursor = comma + 1;
}
*mask_out = parsed;
return true;
}
// Format masks deterministically for CLI/API output. Subsets are emitted in
// ascending order; the two useful extremes use concise keywords.
inline bool format(uint16_t mask, char* output, size_t output_size) {
if (output == nullptr || output_size == 0) return false;
output[0] = '\0';
const char* keyword = nullptr;
if (mask == kAllPacketTypes) keyword = "all";
else if (mask == 0) keyword = "none";
if (keyword != nullptr) {
const size_t len = strlen(keyword);
if (output_size <= len) return false;
memcpy(output, keyword, len + 1);
return true;
}
char formatted[kFilterTextSize];
size_t pos = 0;
bool first = true;
for (uint8_t type = kMinPacketType; type <= kMaxPacketType; ++type) {
if ((mask & static_cast<uint16_t>(1u << type)) == 0) continue;
if (!first) formatted[pos++] = ',';
if (type >= 10) formatted[pos++] = '1';
formatted[pos++] = static_cast<char>('0' + (type % 10));
first = false;
}
formatted[pos] = '\0';
if (output_size <= pos) return false;
memcpy(output, formatted, pos + 1);
return true;
}
// The keyword shortcut above means format() never actually emits the whole
// 0..15 list, but the emit loop is written for every type, so the internal
// buffer must still hold it: 22 digits + 15 separators + NUL.
static_assert(kFilterTextSize >= 38, "filter text buffer must hold the full 0..15 list");
inline bool allows(uint16_t mask, uint8_t packet_type) {
return packet_type <= kMaxPacketType &&
(mask & static_cast<uint16_t>(1u << packet_type)) != 0;
}
// How many types a mask allows. Lets a length-constrained reply fall back to
// "N/16" instead of a clipped list, which would read as a different allowlist.
inline uint8_t countTypes(uint16_t mask) {
uint8_t total = 0;
for (uint8_t type = kMinPacketType; type <= kMaxPacketType; ++type) {
if ((mask & static_cast<uint16_t>(1u << type)) != 0) total++;
}
return total;
}
// Conservative "could any configured broker want this type?" mask, used to
// reject a packet before it is copied into the publish queue. Disabled slots
// contribute nothing; a slot whose topic style later turns out not to support
// the publication is still counted here, so the gate never drops a packet the
// per-slot pass would have published.
inline uint16_t enabledUnion(const uint16_t* masks, const bool* enabled, size_t count) {
if (masks == nullptr) return 0;
uint16_t combined = 0;
for (size_t i = 0; i < count; ++i) {
if (enabled != nullptr && !enabled[i]) continue;
combined = static_cast<uint16_t>(combined | masks[i]);
}
return combined;
}
// True when every slot still carries the default all-types mask, i.e. nothing
// depends on the packet-filter tail of /mqtt_prefs being written.
inline bool allMasksDefault(const uint16_t* masks, size_t count) {
if (masks == nullptr) return true;
for (size_t i = 0; i < count; ++i) {
if (masks[i] != kAllPacketTypes) return false;
}
return true;
}
// Stage one of the publish gate: everything decidable before a packet is
// serialised or a topic is built. Cheap enough to run per packet per slot.
inline bool slotCandidate(bool slot_enabled, uint16_t mask, uint8_t packet_type) {
return slot_enabled && allows(mask, packet_type);
}
// The complete gate, once the slot's topic support is known. Eligibility
// deliberately excludes connection state: a temporarily disconnected broker
// that is configured for this type still requires the shared queue's existing
// bounded retry policy.
//
// eligiblePacketSlots() calls this directly, using slotCandidate() first only
// to skip the topic build for slots the mask already rejects. A slot that
// passes both therefore has its topic built again in the publish loop; that
// second build is the accepted cost of not carrying six 128-byte topics on the
// MQTT task stack.
inline bool slotEligible(bool slot_enabled, bool topic_supported,
uint16_t mask, uint8_t packet_type) {
return slotCandidate(slot_enabled, mask, packet_type) && topic_supported;
}
// A fully filtered/topic-incompatible packet is intentionally complete. Once
// any eligible target exists, at least one actual publish must succeed.
inline bool publishComplete(bool has_eligible_target, bool any_publish_succeeded) {
return any_publish_succeeded || !has_eligible_target;
}
} // namespace MQTTPacketFilter
+52 -9
View File
@@ -2,6 +2,7 @@
#include <string.h>
#include "MQTTPacketFilter.h"
#include "MQTTPrefsStorage.h"
#ifdef WITH_MQTT_BRIDGE
@@ -37,23 +38,44 @@ struct DecodePlan {
static const size_t kV1PreObserverPayloadSize = MQTT_PREFS_V1_PRE_OBSERVER_PAYLOAD_SIZE;
static const size_t kV1PreNeighborsPayloadSize = MQTT_PREFS_V1_PRE_NEIGHBORS_PAYLOAD_SIZE;
static const size_t kV1PreFilterPayloadSize = MQTT_PREFS_V1_PRE_FILTER_PAYLOAD_SIZE;
static const size_t kV1BaselinePayloadSize = MQTT_PREFS_V1_FULL_PAYLOAD_SIZE;
static const size_t kEncodedSize = sizeof(MQTTPrefsHeader) + kV1BaselinePayloadSize;
inline MQTTPrefsHeader makeHeader() {
// Shortest payload length that still round-trips this configuration.
//
// The packet-filter tail is the only optional part of the current layout, and
// its default (all types) is exactly what a pre-filter decoder supplies for a
// missing tail. So a device whose filters are all default keeps writing the
// 2864-byte payload that pre-filter firmware can still read. That matters
// because /mqtt_prefs also carries the WiFi credentials: an unrecognised
// longer payload sends older firmware to defaults with no network, and it
// refuses to overwrite the file, so the node cannot be recovered over the air.
// Touching any filter opts that node into the longer payload — a deliberate,
// operator-initiated trade rather than a side effect of upgrading.
inline size_t payloadLenFor(const MQTTPrefs& prefs) {
return MQTTPacketFilter::allMasksDefault(prefs.mqtt_slot_packet_filter,
MQTT_PREFS_SLOT_COUNT)
? kV1PreFilterPayloadSize
: kV1BaselinePayloadSize;
}
inline MQTTPrefsHeader makeHeader(size_t payload_len) {
MQTTPrefsHeader header;
memcpy(header.magic, MQTT_PREFS_MAGIC, sizeof(header.magic));
header.version = MQTT_PREFS_VERSION;
header.payload_len = static_cast<uint16_t>(kV1BaselinePayloadSize);
header.payload_len = static_cast<uint16_t>(payload_len);
return header;
}
inline size_t encode(const MQTTPrefs& prefs, uint8_t* output, size_t output_size) {
if (output == nullptr || output_size < kEncodedSize) return 0;
const MQTTPrefsHeader header = makeHeader();
const size_t payload_len = payloadLenFor(prefs);
const size_t encoded_size = sizeof(MQTTPrefsHeader) + payload_len;
if (output == nullptr || output_size < encoded_size) return 0;
const MQTTPrefsHeader header = makeHeader(payload_len);
memcpy(output, &header, sizeof(header));
memcpy(output + sizeof(header), &prefs, sizeof(prefs));
return kEncodedSize;
memcpy(output + sizeof(header), &prefs, payload_len);
return encoded_size;
}
inline bool isMagicPrefix(const uint8_t* input, size_t available) {
@@ -92,14 +114,35 @@ inline DecodePlan classify(const uint8_t* prefix, size_t prefix_read, size_t fil
if (header.payload_len != payload_available) {
return corruptPlan();
}
// A same-version append is still unknown to this binary. Holding the
// file prevents a downgrade from discarding it on the next CLI save.
// A longer same-version payload was written by a later build that
// appended fields. Within a version tag the layout is append-only, so
// every byte this binary knows is present and correctly positioned — read
// the baseline prefix and ignore the tail.
//
// This is the downgrade contract, and it is deliberately asymmetric:
// refusing the file would cost the operator WiFi credentials and every
// broker slot (a node with no network and no portal, recoverable only
// over serial), whereas reading it costs only the settings the newer
// build added. Losing a later feature's settings is the acceptable half.
//
// The tail survives until something actually writes: saveMQTTPrefs()
// rewrites at this binary's own length, so a rollback that changes no
// observer setting and is later rolled forward keeps the newer fields
// intact. Only an explicit `set` while downgraded drops them.
//
// A layout change that is NOT a pure append must bump MQTT_PREFS_VERSION;
// the version check above is what makes this rule safe.
if (header.payload_len > kV1BaselinePayloadSize) {
return {Source::UnsupportedVersion, false, true, false, 0};
return {Source::Current, false, false, true, kV1BaselinePayloadSize};
}
if (header.payload_len == kV1BaselinePayloadSize) {
return {Source::Current, false, false, true, kV1BaselinePayloadSize};
}
if (header.payload_len == kV1PreFilterPayloadSize) {
// Written before the per-slot packet-filter tail. Defaults supply an
// all-types mask for every slot.
return {Source::Current, false, false, true, kV1PreFilterPayloadSize};
}
if (header.payload_len == kV1PreNeighborsPayloadSize) {
// Written by observer/webconfig firmware before the neighbors tail
// existed. The observer fields ARE present; only the neighbors tail is
+25 -4
View File
@@ -117,6 +117,11 @@ struct MQTTPrefs {
// interchangeable (see the offsetof static_asserts below).
uint8_t mqtt_neighbors_enabled;
uint32_t mqtt_neighbors_interval;
// Per-slot payload-type allow masks. Bit N controls MeshCore packet type N
// for both packets and raw MQTT topics. Appended so older v1 payloads load
// with the default all-types masks intact.
uint16_t mqtt_slot_packet_filter[MQTT_PREFS_SLOT_COUNT];
};
// Neighbor discovery is scheduled with the wrap-safe millis() helpers, whose
@@ -129,14 +134,28 @@ static const uint32_t MQTT_NEIGHBORS_MIN_INTERVAL_MS = MQTT_NEIGHBORS_MIN_INTERV
static const uint32_t MQTT_NEIGHBORS_MAX_INTERVAL_MS = MQTT_NEIGHBORS_MAX_INTERVAL_HOURS * 3600000UL;
static const uint32_t MQTT_NEIGHBORS_DEFAULT_INTERVAL_MS = MQTT_NEIGHBORS_DEFAULT_INTERVAL_HOURS * 3600000UL;
// Version-1 has three payload layouts this firmware can decode. Never infer a
// compatible payload from an arbitrary shorter size: raw prefs have no checksum.
// Version-1 has four payload layouts this firmware can decode. Never infer a
// compatible payload from an arbitrary SHORTER size: raw prefs have no
// checksum, so a short length has to match a boundary that was really shipped.
//
// A LONGER v1 payload is different and is always readable: within a version tag
// the layout is append-only, so a later build's file still starts with this
// binary's exact baseline. classify() reads that prefix and ignores the tail
// rather than rejecting the file — see the downgrade contract there. Any change
// that is not a pure append MUST bump MQTT_PREFS_VERSION instead.
// - PRE_OBSERVER (2736): stops before the observer tail (snmp_*/alert_*).
// - PRE_NEIGHBORS (2860): full observer tail, no neighbors fields yet.
// - FULL (2864): current baseline, with the neighbors tail.
// - PRE_FILTER (2864): neighbors tail, no per-slot packet filters.
// - FULL (2876): current baseline, with six uint16_t filter masks.
//
// FULL is the maximum written, not the default: MQTTPrefsCodec::payloadLenFor()
// keeps emitting PRE_FILTER while every slot holds the all-types default, so a
// node that never touches a filter stays readable by pre-filter firmware. See
// the rollback note there — /mqtt_prefs also carries the WiFi credentials.
static const size_t MQTT_PREFS_V1_PRE_OBSERVER_PAYLOAD_SIZE = 2736;
static const size_t MQTT_PREFS_V1_PRE_NEIGHBORS_PAYLOAD_SIZE = 2860;
static const size_t MQTT_PREFS_V1_FULL_PAYLOAD_SIZE = 2864;
static const size_t MQTT_PREFS_V1_PRE_FILTER_PAYLOAD_SIZE = 2864;
static const size_t MQTT_PREFS_V1_FULL_PAYLOAD_SIZE = 2876;
// /mqtt_prefs starts with a self-describing 8-byte header. Headerless files
// are deployed legacy layouts and continue to be distinguished by size.
@@ -266,6 +285,8 @@ static_assert(offsetof(MQTTPrefs, mqtt_neighbors_enabled) == 2857,
"neighbors enable flag must sit at the flex-compatible offset");
static_assert(offsetof(MQTTPrefs, mqtt_neighbors_interval) == MQTT_PREFS_V1_PRE_NEIGHBORS_PAYLOAD_SIZE,
"neighbors interval offset must equal the pre-neighbors payload size");
static_assert(offsetof(MQTTPrefs, mqtt_slot_packet_filter) == MQTT_PREFS_V1_PRE_FILTER_PAYLOAD_SIZE,
"packet filters must begin at the pre-filter payload boundary");
static_assert(sizeof(OldMQTTPrefs) == 472, "frozen pre-slot /mqtt_prefs layout changed");
static_assert(sizeof(PreWifiPowerOldMQTTPrefs) == 472, "frozen pre-WiFi-power /mqtt_prefs layout changed");
static_assert(offsetof(OldMQTTPrefs, wifi_power_save) == 144,
+1
View File
@@ -30,6 +30,7 @@ static const char* const WC_ALLOWED_SET_KEYS[] = {
};
static const char* const WC_ALLOWED_SLOT_KEYS[] = {
"preset", "server", "port", "username", "password", "token", "topic", "audience",
"filter",
};
// True when `key` is a well-formed per-slot key ("mqttN.<field>" with N in
+170 -25
View File
@@ -330,10 +330,17 @@ void MQTTBridge::formatMqttStatsReply(char* buf, size_t bufsize) {
// drops=<outbox>/<skipped>: outbox-cap drops vs. memory-pressure skips.
int pos = 0;
replyAppendf(buf, bufsize, &pos, "> Free=%d Max=%d q:%d/%d Outbox=%u drops=%lu/%d |",
replyAppendf(buf, bufsize, &pos, "> Free=%d Max=%d q:%d/%d Outbox=%u drops=%lu/%d",
(int)ESP.getFreeHeap(), (int)ESP.getMaxAllocHeap(),
q, MAX_QUEUE_SIZE, (unsigned)outbox_total,
outbox_drops, b->_skipped_publishes);
// filt=<n>: packets the per-slot type filters rejected before the queue.
// Omitted while zero so an unfiltered node's reply keeps its former length —
// the per-slot list below is what usually gets clamped away first.
if (b->_filtered_packets > 0) {
replyAppendf(buf, bufsize, &pos, " filt=%lu", b->_filtered_packets);
}
replyAppendf(buf, bufsize, &pos, " |");
for (int i = 0; i < RUNTIME_MQTT_SLOTS; i++) {
if (!b->_slots[i].enabled || !b->_slots[i].client) continue;
replyAppendf(buf, bufsize, &pos, " s%d=%lu/%lu", i + 1,
@@ -370,6 +377,9 @@ bool MQTTBridge::getSlotStatusSnapshot(int slot_index, SlotStatusSnapshot* out)
}
out->publish_ok = slot.client ? slot.client->getPublishOk() : 0;
out->publish_err = slot.client ? slot.client->getPublishErr() : 0;
// Lets the portal show why a healthy slot is quiet.
out->filter_mask = b->_obs ? b->_obs->mqtt_slot_packet_filter[slot_index]
: MQTTPacketFilter::kAllPacketTypes;
return true;
}
@@ -476,6 +486,17 @@ void MQTTBridge::formatSlotDiagReply(char* buf, size_t bufsize, int slot_index)
// replyAppendf clamps pos on every call, so the chained appends below can't
// walk past the reply buffer even if the accumulated text exceeds it (A1).
// A non-default filter is the one healthy-looking reason for a slot to stop
// publishing, so it has to appear — but appended last. replyAppendf clamps at
// the 160-byte reply, and an error tail (TLS + mbedTLS + errno + age) can
// already reach ~117 chars, so putting the filter first would push the
// operator's diagnostic detail off the end of a failing slot's line.
const uint16_t filter_mask = b->_obs ? b->_obs->mqtt_slot_packet_filter[slot_index]
: MQTTPacketFilter::kAllPacketTypes;
char filter_text[MQTTPacketFilter::kFilterTextSize];
const bool show_filter = filter_mask != MQTTPacketFilter::kAllPacketTypes &&
MQTTPacketFilter::format(filter_mask, filter_text, sizeof(filter_text));
int pos = 0;
replyAppendf(buf, bufsize, &pos, "> mqtt%d: %s", slot_index + 1, state);
if (slot.disconnect_count > 0) {
@@ -486,14 +507,10 @@ void MQTTBridge::formatSlotDiagReply(char* buf, size_t bufsize, int slot_index)
}
}
// If connected with no errors, we're done
// Connected with no errors: nothing more to say about the connection.
if (slot.connected && slot.last_error_time == 0) {
replyAppendf(buf, bufsize, &pos, ", no errors");
return;
}
// Show last error if we have one
if (slot.last_error_time > 0) {
} else if (slot.last_error_time > 0) {
// TLS error with human-friendly description
if (slot.last_tls_err != 0) {
const char* desc = tlsErrorStr(slot.last_tls_err);
@@ -523,6 +540,26 @@ void MQTTBridge::formatSlotDiagReply(char* buf, size_t bufsize, int slot_index)
} else if (!slot.connected) {
replyAppendf(buf, bufsize, &pos, ", no error info");
}
// Appended last so it never displaces connection diagnostics. replyAppendf
// clamps rather than overflows, but a clipped type list is worse than no
// list: "…,13,14," parses as a real, different allowlist, and this is the
// one line an operator reads to find out why a slot is quiet. So the exact
// text is only emitted when it fits whole; otherwise fall back to a count,
// which cannot be misread. `get mqttN.filter` always has the exact value.
if (show_filter) {
static const int kFilterLabelLen = (int)sizeof(", filter:") - 1;
const int remaining = pos < (int)bufsize ? (int)bufsize - 1 - pos : 0;
const uint8_t allowed = MQTTPacketFilter::countTypes(filter_mask);
const int compact_len = kFilterLabelLen + (allowed >= 10 ? 5 : 4); // "N/16"
if (kFilterLabelLen + (int)strlen(filter_text) <= remaining) {
replyAppendf(buf, bufsize, &pos, ", filter:%s", filter_text);
} else if (compact_len <= remaining) {
replyAppendf(buf, bufsize, &pos, ", filter:%u/16", (unsigned)allowed);
}
// Neither fits: the slot is reporting so much error detail that the filter
// is the least useful field on the line. Omit it rather than mislead.
}
}
// Bounded cooperative-stop timeout for end() (see MQTTLifecycle::Coordinator).
@@ -2699,15 +2736,16 @@ void MQTTBridge::loop() {
void MQTTBridge::onPacketReceived(mesh::Packet *packet) {
if (!_initialized || !_obs->mqtt_packets_enabled || !_obs->mqtt_rx_enabled) return;
// Check if we have any enabled slots to send to
bool has_valid_slots = false;
for (int i = 0; i < RUNTIME_MQTT_SLOTS; i++) {
if (_slots[i].enabled && _slots[i].client) {
has_valid_slots = true;
break;
}
// Drop before the queue copy when no configured slot allows this payload
// type. A QueuedPacket carries the packet plus up to 256 bytes of raw radio
// data and crosses to Core 0, so filtering here (not at publish time) is
// what actually saves the queue slot and the memcpy. This also subsumes the
// older "is any slot configured at all?" pre-check.
bool filtered = false;
if (!shouldQueuePacketType(packet->getPayloadType(), filtered)) {
if (filtered) _filtered_packets++;
return;
}
if (!has_valid_slots) return;
// Queue packet for transmission
queuePacket(packet, false);
@@ -2725,6 +2763,13 @@ void MQTTBridge::sendPacket(mesh::Packet *packet) {
if (!_identity || memcmp(_identity->pub_key, packet->payload, PUB_KEY_SIZE) != 0) return;
}
// Same pre-queue filter gate as the RX path.
bool filtered = false;
if (!shouldQueuePacketType(packet->getPayloadType(), filtered)) {
if (filtered) _filtered_packets++;
return;
}
// Queue packet for transmission
queuePacket(packet, true);
}
@@ -2816,7 +2861,9 @@ void MQTTBridge::processPacketQueue() {
_last_raw_timestamp = millis();
}
bool packet_eligible = false;
bool packet_published = publishPacket(&queued.packet_copy, queued.is_tx,
packet_eligible,
queued.has_raw_data ? queued.raw_data : nullptr,
queued.has_raw_data ? queued.raw_len : 0,
queued.snr, queued.rssi);
@@ -2824,15 +2871,21 @@ void MQTTBridge::processPacketQueue() {
// Publish raw if enabled (live from prefs so `set mqtt.raw` applies without
// a bridge restart)
bool raw_eligible = false;
bool raw_published = false;
if (_obs->mqtt_raw_enabled) {
raw_published = publishRaw(&queued.packet_copy);
raw_published = publishRaw(&queued.packet_copy, raw_eligible);
}
bool any_published = MQTTPacketQueuePolicy::queuedPacketPublished(packet_published, raw_published);
// Decide intentional completion once across the entire queue item. An
// ineligible raw path (for example, MeshRank's packets-only topic style)
// must not hide a failed eligible structured publish.
const bool queue_complete = MQTTPacketFilter::publishComplete(
packet_eligible || raw_eligible,
MQTTPacketQueuePolicy::queuedPacketPublished(packet_published, raw_published));
const MQTTPacketQueuePolicy::RetryDecision retry =
MQTTPacketQueuePolicy::retryDecision(
any_published, queued.retry_attempts,
queue_complete, queued.retry_attempts,
static_cast<uint32_t>(now_ms));
if (retry.action == MQTTPacketQueuePolicy::RetryAction::Schedule) {
queued.retry_attempts = retry.retry_attempts;
@@ -2943,22 +2996,30 @@ void MQTTBridge::processPacketQueue() {
_last_raw_timestamp = millis();
}
bool packet_eligible = false;
bool packet_published = publishPacket(&queued.packet_copy, queued.is_tx,
packet_eligible,
queued.has_raw_data ? queued.raw_data : nullptr,
queued.has_raw_data ? queued.raw_len : 0,
queued.snr, queued.rssi);
// No taskYIELD() on non-ESP32 platforms (non-FreeRTOS, cooperative scheduling not needed)
// Live from prefs so `set mqtt.raw` applies without a bridge restart.
bool raw_eligible = false;
bool raw_published = false;
if (_obs->mqtt_raw_enabled) {
raw_published = publishRaw(&queued.packet_copy);
raw_published = publishRaw(&queued.packet_copy, raw_eligible);
}
bool any_published = MQTTPacketQueuePolicy::queuedPacketPublished(packet_published, raw_published);
// Decide intentional completion once across the entire queue item. An
// ineligible raw path (for example, MeshRank's packets-only topic style)
// must not hide a failed eligible structured publish.
const bool queue_complete = MQTTPacketFilter::publishComplete(
packet_eligible || raw_eligible,
MQTTPacketQueuePolicy::queuedPacketPublished(packet_published, raw_published));
const MQTTPacketQueuePolicy::RetryDecision retry =
MQTTPacketQueuePolicy::retryDecision(
any_published, queued.retry_attempts,
queue_complete, queued.retry_attempts,
static_cast<uint32_t>(now_ms));
if (retry.action == MQTTPacketQueuePolicy::RetryAction::Schedule) {
queued.retry_attempts = retry.retry_attempts;
@@ -3005,6 +3066,69 @@ void MQTTBridge::processPacketQueue() {
// Publishing
// ---------------------------------------------------------------------------
// Which slots will actually take this payload type, resolved in two stages so
// the cheap test runs first: the filter/enabled check costs a bit test, while
// the topic build costs an snprintf. Both are far cheaper than the ~2 KB JSON
// serialisation they gate, so this must complete before a caller builds a
// message — a slot that cannot form a topic (MeshRank raw, or a meshcore
// preset with no IATA set) would otherwise pay for a document nobody receives.
//
// A slot's connection state is deliberately not consulted: a configured but
// disconnected broker is still a target, so the packet stays on the queue for
// the existing bounded retry rather than being dropped as complete.
uint8_t MQTTBridge::eligiblePacketSlots(uint8_t packet_type, MQTTMessageType type) {
static_assert(RUNTIME_MQTT_SLOTS <= 8, "eligible slot mask must fit in uint8_t");
// Per-slot filters cover packet traffic only. Status and neighbors are
// documented as never filtered, so refuse those publication types here
// rather than silently applying a packet mask if a caller is added later.
if (!_obs || (type != MSG_PACKETS && type != MSG_RAW)) return 0;
uint8_t eligible_slots = 0;
char topic[128];
for (int i = 0; i < RUNTIME_MQTT_SLOTS; ++i) {
const bool slot_enabled = _slots[i].enabled && _slots[i].client != nullptr;
// Load once so a live CLI/WebConfig update cannot split this packet's
// decision across two different masks.
const uint16_t filter_mask = _obs->mqtt_slot_packet_filter[i];
if (!MQTTPacketFilter::slotCandidate(slot_enabled, filter_mask, packet_type)) continue;
const bool topic_supported = buildTopicForSlot(i, type, topic, sizeof(topic));
if (MQTTPacketFilter::slotEligible(slot_enabled, topic_supported,
filter_mask, packet_type)) {
eligible_slots |= static_cast<uint8_t>(1u << i);
}
}
return eligible_slots;
}
// Conservative pre-queue gate: the OR of every configured slot's mask. Runs on
// Core 1 in the radio callback, so it must stay cheap — a packet no broker
// wants is rejected before it is copied into the queue at all. `filtered`
// separates "a broker is configured but none wants this type" (worth counting)
// from "no broker configured at all" (the pre-existing silent drop).
bool MQTTBridge::shouldQueuePacketType(uint8_t packet_type, bool& filtered) {
filtered = false;
if (!_obs) return false;
uint16_t masks[RUNTIME_MQTT_SLOTS];
bool enabled[RUNTIME_MQTT_SLOTS];
bool any_enabled = false;
for (int i = 0; i < RUNTIME_MQTT_SLOTS; ++i) {
masks[i] = _obs->mqtt_slot_packet_filter[i];
enabled[i] = _slots[i].enabled && _slots[i].client != nullptr;
any_enabled = any_enabled || enabled[i];
}
if (!any_enabled) return false;
if (MQTTPacketFilter::allows(
MQTTPacketFilter::enabledUnion(masks, enabled, RUNTIME_MQTT_SLOTS),
packet_type)) {
return true;
}
filtered = true;
return false;
}
bool MQTTBridge::publishStatus() {
if (!_cached_has_connected_slots) {
return false;
@@ -3104,10 +3228,19 @@ bool MQTTBridge::publishStatus() {
}
bool MQTTBridge::publishPacket(mesh::Packet* packet, bool is_tx,
bool& has_eligible_target,
const uint8_t* raw_data, int raw_len,
float snr, float rssi) {
has_eligible_target = false;
if (!packet) return false;
const uint8_t packet_type = packet->getPayloadType();
const uint8_t eligible_slots = eligiblePacketSlots(packet_type, MSG_PACKETS);
has_eligible_target = eligible_slots != 0;
// Filtered out everywhere, or no slot can form a packets topic: intentionally
// complete, and nothing below (JSON build included) is worth paying for.
if (!has_eligible_target) return false;
refreshOriginFromPrefs();
// Memory pressure check: Skip publishes when there's not enough contiguous
@@ -3213,7 +3346,11 @@ bool MQTTBridge::publishPacket(mesh::Packet* packet, bool is_tx,
bool published = false;
char topic[128];
for (int i = 0; i < RUNTIME_MQTT_SLOTS; i++) {
if (_slots[i].enabled && _slots[i].client && _slots[i].connected) {
// Eligibility already proved this slot's topic builds; rebuilding it is
// one snprintf, cheaper than carrying a RUNTIME_MQTT_SLOTS x 128 topic
// cache on the MQTT task's 8 KB stack alongside the JSON buffer.
if ((eligible_slots & static_cast<uint8_t>(1u << i)) != 0 &&
_slots[i].enabled && _slots[i].client && _slots[i].connected) {
if (buildTopicForSlot(i, MSG_PACKETS, topic, sizeof(topic))) {
if (publishToSlot(i, topic, active_buffer, false)) {
published = true;
@@ -3223,7 +3360,6 @@ bool MQTTBridge::publishPacket(mesh::Packet* packet, bool is_tx,
}
return published;
} else {
uint8_t packet_type = packet->getPayloadType();
if (packet_type == 4 || packet_type == 9) {
MQTT_DEBUG_PRINTLN("Failed to build packet JSON for type=%d (len=%d), packet not published", packet_type, len);
}
@@ -3231,9 +3367,17 @@ bool MQTTBridge::publishPacket(mesh::Packet* packet, bool is_tx,
return false;
}
bool MQTTBridge::publishRaw(mesh::Packet* packet) {
bool MQTTBridge::publishRaw(mesh::Packet* packet, bool& has_eligible_target) {
has_eligible_target = false;
if (!packet) return false;
const uint8_t packet_type = packet->getPayloadType();
const uint8_t eligible_slots = eligiblePacketSlots(packet_type, MSG_RAW);
has_eligible_target = eligible_slots != 0;
// Filtered out everywhere, or no slot has a raw topic (MeshRank is
// packets-only): intentionally complete, and no JSON is built.
if (!has_eligible_target) return false;
refreshOriginFromPrefs();
#if defined(BOARD_HAS_PSRAM)
@@ -3264,7 +3408,8 @@ bool MQTTBridge::publishRaw(mesh::Packet* packet) {
bool published = false;
char topic[128];
for (int i = 0; i < RUNTIME_MQTT_SLOTS; i++) {
if (_slots[i].enabled && _slots[i].client && _slots[i].connected) {
if ((eligible_slots & static_cast<uint8_t>(1u << i)) != 0 &&
_slots[i].enabled && _slots[i].client && _slots[i].connected) {
if (buildTopicForSlot(i, MSG_RAW, topic, sizeof(topic))) {
if (publishToSlot(i, topic, active_buffer, false)) {
published = true;
+12 -2
View File
@@ -8,6 +8,7 @@
#include <WiFiUdp.h>
#include <Timezone.h>
#include "helpers/JWTHelper.h"
#include "helpers/MQTTPacketFilter.h"
#include "helpers/MQTTPresets.h"
#include "helpers/MQTTLifecycle.h"
#include <atomic>
@@ -325,6 +326,10 @@ private:
unsigned long _last_memory_check;
bool _memory_pressure = false; // Cached max-alloc verdict; re-sampled at most once per interval in publishPacket() so the heap walk isn't paid per-packet under pressure
int _skipped_publishes; // Exposed via SNMP; count of publishes skipped when max_alloc is too low
// Packets rejected by the per-slot filters before reaching the queue. Written
// on Core 1 (radio callbacks), read on Core 0 for `mqtt.stats`; a torn read of
// a diagnostic counter is harmless, so no atomic is warranted.
unsigned long _filtered_packets = 0;
// Status publish retry tracking
unsigned long _last_status_retry; // Track last retry attempt (separate from successful publish)
@@ -369,6 +374,8 @@ private:
enum MQTTMessageType { MSG_STATUS, MSG_PACKETS, MSG_RAW, MSG_NEIGHBORS };
bool buildTopicForSlot(int index, MQTTMessageType type, char* topic_buf, size_t buf_size);
bool substituteTopicTemplate(const char* tmpl, MQTTMessageType type, int slot_index, char* buf, size_t buf_size);
uint8_t eligiblePacketSlots(uint8_t packet_type, MQTTMessageType type);
bool shouldQueuePacketType(uint8_t packet_type, bool& filtered);
// Internal methods - slot management
// Lifetime model (Phase 1 of MQTT memory-defrag):
@@ -404,10 +411,10 @@ private:
void mqttTaskLoop(); // Main loop for MQTT task
void initializeWiFiInTask(); // WiFi initialization moved to task
#endif
bool publishPacket(mesh::Packet* packet, bool is_tx,
bool publishPacket(mesh::Packet* packet, bool is_tx, bool& has_eligible_target,
const uint8_t* raw_data = nullptr, int raw_len = 0,
float snr = 0.0f, float rssi = 0.0f);
bool publishRaw(mesh::Packet* packet);
bool publishRaw(mesh::Packet* packet, bool& has_eligible_target);
#if defined(WITH_MQTT_NEIGHBORS)
// Publishes the pending _neighbors_json_buffer to every connected slot's
// neighbors topic. Runs on the MQTT task (Core 0) only.
@@ -591,6 +598,9 @@ public:
const char* state; // "inactive" | "wait" | "ok" | "fail" | "disc"
unsigned long publish_ok;
unsigned long publish_err;
// Raw packet-type allowlist. Kept as the mask rather than canonical text so
// the 1 KB stats JSON stays compact; the portal renders it.
uint16_t filter_mask;
};
static bool getSlotStatusSnapshot(int slot_index, SlotStatusSnapshot* out);
/** True when WiFi is set and at least one MQTT slot can run (preset + custom host if needed). */
+32 -1
View File
@@ -11,6 +11,7 @@
#include <esp_heap_caps.h>
#include <helpers/CommonCLI.h>
#include <helpers/MQTTPacketFilter.h>
#include <helpers/MQTTPresets.h>
#include <helpers/WebConfigKeys.h>
#include <helpers/bridges/MQTTBridge.h>
@@ -618,6 +619,15 @@ void WebConfigServer::handleConfigGet(AsyncWebServerRequest* req) {
s["token"] = _obs->mqtt_slot_token[i][0] ? SECRET_SENTINEL : "";
s["topic"] = (const char*)_obs->mqtt_slot_topic[i];
s["audience"] = (const char*)_obs->mqtt_slot_audience[i];
char filter_text[MQTTPacketFilter::kFilterTextSize];
if (MQTTPacketFilter::format(_obs->mqtt_slot_packet_filter[i],
filter_text, sizeof(filter_text))) {
// Mutable char input is copied into the ArduinoJson document; the
// stack buffer is reused on the next slot.
s["filter"] = filter_text;
} else {
s["filter"] = "all";
}
}
}
@@ -736,11 +746,32 @@ void WebConfigServer::handleConfigPost(AsyncWebServerRequest* req) {
// top-level `password` command, so it persists exactly as the CLI does.
int pos = admin_pwd ? snprintf(e.cmd, sizeof(e.cmd), "password ")
: snprintf(e.cmd, sizeof(e.cmd), "set %s ", key);
for (const char* p = val; *p && pos < (int)sizeof(e.cmd) - 1; p++) {
bool overflow = false;
for (const char* p = val; *p; p++) {
if (*p == '\r' || *p == '\n') continue;
if (pos >= (int)sizeof(e.cmd) - 1) { overflow = true; break; }
e.cmd[pos++] = *p;
}
e.cmd[pos] = 0;
// A value that doesn't fit used to be truncated here and applied anyway.
// For length-checked keys the CLI would reject the remainder, but a key
// whose grammar stays valid when clipped (mqttN.filter: "advert,2" cut to
// "advert") would silently persist a *different* setting and still answer
// OK. Refuse the batch instead — the caller can shorten and retry.
if (overflow) {
// Name the key, like the "bad key" rejection above: a 20-field batch is
// rejected whole, so without it the operator has nothing to correct.
char safe_key[33];
strncpy(safe_key, key, sizeof(safe_key) - 1);
safe_key[sizeof(safe_key) - 1] = 0;
StaticJsonDocument<128> ed;
ed["error"] = "value too long";
ed["key"] = safe_key;
String out;
serializeJson(ed, out);
req->send(400, "application/json", out);
return;
}
count++;
}
// Phase 2: the count is now known, so the remaining NoChanges/Accept arms
+2 -1
View File
@@ -30,8 +30,9 @@ does not reflect the GoogleTest count — run the built binary directly
| `test_mqtt_topic_router` | `src/helpers/MQTTTopicRouter.h` | complete preset/custom topic-routing contract; MeshRank packets-only behavior; required identifiers; invalid inputs/slots; exact buffer boundaries |
| `test_mqtt_connection_policy` | `src/helpers/MQTTConnectionPolicy.h` | reconnect guard/backoff/stagger and breaker transitions; stable reset; JWT lifetime/renewal policy; exact timing boundaries and 32-bit `millis()` rollover |
| `test_mqtt_packet_queue_policy` | `src/helpers/MQTTPacketQueuePolicy.h` | queue-full eviction; stale-disconnect flush; adaptive drain limits; bounded QoS0 retries; exact timing boundaries and 32-bit `millis()` rollover |
| `test_mqtt_packet_filter` | `src/helpers/MQTTPacketFilter.h` | per-slot 0-15 allowlist parsing/formatting, numeric and named spellings; exact bounds; membership; candidate/eligible split and retry-completion policy; pre-queue union gate; default-mask detection |
| `test_mqtt_runtime_buffer_lifecycle` | `src/helpers/MQTTRuntimeBufferLifecycle.h` | idempotent allocation/release; partial-allocation degradation; retry of only missing buffers |
| `test_mqtt_prefs_codec` | `src/helpers/MQTTPrefsStorage.h`, `src/helpers/MQTTPrefsCodec.h` | binary pre-slot/3-slot/6-slot migration fixtures; v1 header integrity; downgrade preservation |
| `test_mqtt_prefs_codec` | `src/helpers/MQTTPrefsStorage.h`, `src/helpers/MQTTPrefsCodec.h` | binary pre-slot/3-slot/6-slot migration fixtures; v1 header integrity; downgrade preservation; shortest-payload write policy (default filters stay downgrade-readable) |
| `test_mqtt_prefs_atomic_store` | `src/helpers/MQTTPrefsAtomicStore.h` | transactional MQTT writes and legacy `/node_prefs` handoff; exact short-write detection; begin/finish/rename failure cleanup; original-file preservation |
| `test_mqtt_payload_builder` | `src/helpers/MQTTPayloadBuilder.cpp` | status/packet/raw JSON contracts; optional fields; escaping; RX metrics and path; score handling; exact buffer bounds; maximum representative payloads |
| `test_utils` | `src/Utils.cpp` | `Utils::toHex` (upstream) |
@@ -0,0 +1,303 @@
#include <gtest/gtest.h>
#include "helpers/MQTTPacketFilter.h"
namespace Filter = MQTTPacketFilter;
TEST(MQTTPacketFilter, EmptyAndAllMeanEveryType) {
for (const char* value : {"", " ", "\tall\r\n"}) {
uint16_t mask = 0;
ASSERT_TRUE(Filter::parse(value, &mask)) << value;
EXPECT_EQ(Filter::kAllPacketTypes, mask);
}
}
TEST(MQTTPacketFilter, NoneMeansNoTypes) {
uint16_t mask = Filter::kAllPacketTypes;
ASSERT_TRUE(Filter::parse(" none ", &mask));
EXPECT_EQ(0u, mask);
}
TEST(MQTTPacketFilter, ParsesDecimalCsvWithWhitespaceAndDuplicates) {
uint16_t mask = 0;
ASSERT_TRUE(Filter::parse(" 2, 4,\t15,2 ", &mask));
EXPECT_EQ(static_cast<uint16_t>((1u << 2) | (1u << 4) | (1u << 15)), mask);
}
TEST(MQTTPacketFilter, FullNumericListCanonicalizesToAll) {
uint16_t mask = 0;
ASSERT_TRUE(Filter::parse("0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15", &mask));
EXPECT_EQ(Filter::kAllPacketTypes, mask);
char output[Filter::kFilterTextSize];
ASSERT_TRUE(Filter::format(mask, output, sizeof(output)));
EXPECT_STREQ("all", output);
}
TEST(MQTTPacketFilter, ParsesPayloadTypeNames) {
uint16_t mask = 0;
ASSERT_TRUE(Filter::parse("advert", &mask));
EXPECT_EQ(static_cast<uint16_t>(1u << 4), mask);
ASSERT_TRUE(Filter::parse(" txt_msg , grp_txt ", &mask));
EXPECT_EQ(static_cast<uint16_t>((1u << 2) | (1u << 5)), mask);
// Names and numbers are interchangeable within one list.
ASSERT_TRUE(Filter::parse("advert,2,raw_custom", &mask));
EXPECT_EQ(static_cast<uint16_t>((1u << 4) | (1u << 2) | (1u << 15)), mask);
// Output stays numeric regardless of how the value was spelled.
char output[Filter::kFilterTextSize];
ASSERT_TRUE(Filter::format(mask, output, sizeof(output)));
EXPECT_STREQ("2,4,15", output);
}
TEST(MQTTPacketFilter, EveryNamedTypeResolvesToItsPacketHeaderValue) {
// Pins the table against src/Packet.h's PAYLOAD_TYPE_* values.
const struct { const char* name; uint8_t type; } expected[] = {
{"req", 0}, {"response", 1}, {"txt_msg", 2}, {"ack", 3}, {"advert", 4},
{"grp_txt", 5}, {"grp_data", 6}, {"anon_req", 7}, {"path", 8},
{"trace", 9}, {"multipart", 10}, {"control", 11}, {"raw_custom", 15},
};
for (const auto& e : expected) {
uint16_t mask = 0;
ASSERT_TRUE(Filter::parse(e.name, &mask)) << e.name;
EXPECT_EQ(static_cast<uint16_t>(1u << e.type), mask) << e.name;
}
// Reserved types have no name and stay reachable by number only.
size_t count = 0;
const Filter::NamedPacketType* names = Filter::namedPacketTypes(&count);
EXPECT_EQ(sizeof(expected) / sizeof(expected[0]), count);
for (size_t i = 0; i < count; ++i) {
EXPECT_TRUE(names[i].type < 12 || names[i].type == 15) << names[i].name;
}
}
TEST(MQTTPacketFilter, RejectsMalformedOrOutOfRangeValuesWithoutChangingOutput) {
const char* invalid[] = {
",", "1,", ",1", "1,,2", "1, ,2", "-1", "+1", "0x2", "16",
"999999999999999999999", "ALL", "All", "all,2", "none,2", "2-4", "2x",
// Names are lowercase, exact, and are not whole-value keywords.
"ADVERT", "Advert", "advert_", "adver", "advertx", "reserved", "12_reserved",
"2,all", "advert,none", "advert advert", "advert;2",
};
for (const char* value : invalid) {
uint16_t mask = 0x1234;
EXPECT_FALSE(Filter::parse(value, &mask)) << value;
EXPECT_EQ(0x1234, mask) << value;
}
EXPECT_FALSE(Filter::parse(nullptr, nullptr));
}
TEST(MQTTPacketFilter, FormatsSubsetsInAscendingOrder) {
const uint16_t mask = static_cast<uint16_t>(
(1u << 15) | (1u << 2) | (1u << 10) | (1u << 0));
char output[Filter::kFilterTextSize];
ASSERT_TRUE(Filter::format(mask, output, sizeof(output)));
EXPECT_STREQ("0,2,10,15", output);
}
TEST(MQTTPacketFilter, FormatsNoneAndHonorsExactBufferBoundaries) {
char none[5];
ASSERT_TRUE(Filter::format(0, none, sizeof(none)));
EXPECT_STREQ("none", none);
char too_small[4] = {'x', 'x', 'x', '\0'};
EXPECT_FALSE(Filter::format(0, too_small, sizeof(too_small)));
EXPECT_STREQ("", too_small);
char all_types[Filter::kFilterTextSize];
const uint16_t subset = static_cast<uint16_t>(Filter::kAllPacketTypes & ~(1u << 14));
ASSERT_TRUE(Filter::format(subset, all_types, sizeof(all_types)));
EXPECT_STREQ("0,1,2,3,4,5,6,7,8,9,10,11,12,13,15", all_types);
char one_short[34];
EXPECT_FALSE(Filter::format(subset, one_short, sizeof(one_short)));
EXPECT_STREQ("", one_short);
}
TEST(MQTTPacketFilter, MembershipIsBoundedToFourBitTypes) {
EXPECT_TRUE(Filter::allows(static_cast<uint16_t>(1u << 0), 0));
EXPECT_TRUE(Filter::allows(static_cast<uint16_t>(1u << 15), 15));
EXPECT_FALSE(Filter::allows(Filter::kAllPacketTypes, 16));
EXPECT_FALSE(Filter::allows(0, 0));
}
TEST(MQTTPacketFilter, EligibilityIgnoresConnectionButRequiresTopicAndFilter) {
const uint16_t only_advert = static_cast<uint16_t>(1u << 4);
EXPECT_TRUE(Filter::slotEligible(true, true, only_advert, 4));
EXPECT_FALSE(Filter::slotEligible(false, true, only_advert, 4));
EXPECT_FALSE(Filter::slotEligible(true, false, only_advert, 4));
EXPECT_FALSE(Filter::slotEligible(true, true, only_advert, 2));
}
TEST(MQTTPacketFilter, AllFilteredOrTopicIncompatibleTargetsCompleteIntentionally) {
const uint16_t only_text = static_cast<uint16_t>(1u << 2);
const bool filtered = Filter::slotEligible(true, true, only_text, 4);
const bool topic_incompatible =
Filter::slotEligible(true, false, Filter::kAllPacketTypes, 4);
EXPECT_FALSE(filtered || topic_incompatible);
EXPECT_TRUE(Filter::publishComplete(filtered || topic_incompatible, false));
}
TEST(MQTTPacketFilter, EligibleDisconnectedTargetStillRequiresRetry) {
const uint16_t only_advert = static_cast<uint16_t>(1u << 4);
// Connection state is intentionally absent from slotEligible(): the first
// slot remains a target while disconnected. A second connected slot whose
// filter rejects the advert cannot turn that into intentional completion.
const bool disconnected_eligible =
Filter::slotEligible(true, true, only_advert, 4);
const bool connected_filtered =
Filter::slotEligible(true, true, static_cast<uint16_t>(1u << 2), 4);
ASSERT_TRUE(disconnected_eligible);
ASSERT_FALSE(connected_filtered);
EXPECT_FALSE(Filter::publishComplete(
disconnected_eligible || connected_filtered, false));
}
TEST(MQTTPacketFilter, PublishFailureRetriesAndAnySuccessCompletes) {
EXPECT_FALSE(Filter::publishComplete(true, false));
EXPECT_TRUE(Filter::publishComplete(true, true));
EXPECT_TRUE(Filter::publishComplete(false, true)); // defensive: success wins
}
TEST(MQTTPacketFilter, UnsupportedRawPathCannotHideEligiblePacketFailure) {
const bool packet_eligible = true;
const bool packet_published = false;
const bool raw_eligible = false; // e.g. MeshRank has no raw topic
const bool raw_published = false;
EXPECT_FALSE(Filter::publishComplete(
packet_eligible || raw_eligible,
packet_published || raw_published));
EXPECT_TRUE(Filter::publishComplete(false, false));
}
TEST(MQTTPacketFilter, PacketAndRawCanShareTheSameMaskDecision) {
const uint16_t text_and_advert = static_cast<uint16_t>((1u << 2) | (1u << 4));
for (uint8_t type = 0; type <= 15; ++type) {
const bool expected = type == 2 || type == 4;
EXPECT_EQ(expected, Filter::allows(text_and_advert, type)) << unsigned(type);
}
}
TEST(MQTTPacketFilter, CandidateIsTheTopicFreePartOfEligibility) {
const uint16_t only_advert = static_cast<uint16_t>(1u << 4);
EXPECT_TRUE(Filter::slotCandidate(true, only_advert, 4));
EXPECT_FALSE(Filter::slotCandidate(false, only_advert, 4));
EXPECT_FALSE(Filter::slotCandidate(true, only_advert, 2));
// A candidate is exactly an eligible slot minus the topic-support term. The
// bridge evaluates the cheap half first and only builds a topic for slots
// that pass it, so the two must stay in this exact relationship.
for (int enabled = 0; enabled <= 1; ++enabled) {
for (int topic = 0; topic <= 1; ++topic) {
for (uint8_t type = 0; type <= 15; ++type) {
EXPECT_EQ(Filter::slotCandidate(enabled != 0, only_advert, type) && topic != 0,
Filter::slotEligible(enabled != 0, topic != 0, only_advert, type));
}
}
}
}
// The gate has to resolve topic support before anything expensive runs: a
// connected slot that cannot form a topic (MeshRank raw, or a meshcore preset
// with no IATA) must reach "intentionally complete" without a serialisation.
TEST(MQTTPacketFilter, TopicIncompatibleTargetsAreCompleteWithoutPublishing) {
const uint16_t all = Filter::kAllPacketTypes;
for (uint8_t type = 0; type <= 15; ++type) {
const bool eligible = Filter::slotEligible(true, false, all, type);
EXPECT_FALSE(eligible) << unsigned(type);
EXPECT_TRUE(Filter::publishComplete(eligible, false)) << unsigned(type);
}
// With topic support restored the same slot becomes a retry target again.
EXPECT_TRUE(Filter::slotEligible(true, true, all, 4));
EXPECT_FALSE(Filter::publishComplete(Filter::slotEligible(true, true, all, 4), false));
}
TEST(MQTTPacketFilter, EnabledUnionIgnoresDisabledSlotsAndGatesTheQueue) {
const uint16_t masks[3] = {
static_cast<uint16_t>(1u << 2), // text only
static_cast<uint16_t>(1u << 4), // adverts only
Filter::kAllPacketTypes, // everything, but disabled below
};
const bool enabled[3] = {true, true, false};
const uint16_t combined = Filter::enabledUnion(masks, enabled, 3);
EXPECT_EQ(static_cast<uint16_t>((1u << 2) | (1u << 4)), combined);
EXPECT_TRUE(Filter::allows(combined, 2));
EXPECT_TRUE(Filter::allows(combined, 4));
EXPECT_FALSE(Filter::allows(combined, 3)); // no enabled slot wants ACKs
// A null `enabled` array counts every slot; no slots means no destination.
EXPECT_EQ(Filter::kAllPacketTypes, Filter::enabledUnion(masks, nullptr, 3));
EXPECT_EQ(0u, Filter::enabledUnion(masks, enabled, 0));
EXPECT_EQ(0u, Filter::enabledUnion(nullptr, enabled, 3));
// The union must never reject a type any single slot would have published.
for (uint8_t type = 0; type <= 15; ++type) {
const bool any_slot_allows =
(enabled[0] && Filter::allows(masks[0], type)) ||
(enabled[1] && Filter::allows(masks[1], type)) ||
(enabled[2] && Filter::allows(masks[2], type));
EXPECT_EQ(any_slot_allows, Filter::allows(combined, type)) << unsigned(type);
}
}
TEST(MQTTPacketFilter, AllMasksDefaultDrivesTheShortPrefsPayload) {
uint16_t masks[6];
for (int i = 0; i < 6; ++i) masks[i] = Filter::kAllPacketTypes;
EXPECT_TRUE(Filter::allMasksDefault(masks, 6));
EXPECT_TRUE(Filter::allMasksDefault(nullptr, 6));
EXPECT_TRUE(Filter::allMasksDefault(masks, 0));
// Any slot off the default — including "none" — opts into the longer payload.
for (int i = 0; i < 6; ++i) {
masks[i] = 0;
EXPECT_FALSE(Filter::allMasksDefault(masks, 6)) << i;
masks[i] = static_cast<uint16_t>(Filter::kAllPacketTypes & ~1u);
EXPECT_FALSE(Filter::allMasksDefault(masks, 6)) << i;
masks[i] = Filter::kAllPacketTypes;
}
EXPECT_TRUE(Filter::allMasksDefault(masks, 6));
}
// countTypes() backs the diag line's fallback when the exact list won't fit,
// so it has to agree with what format() would have emitted.
TEST(MQTTPacketFilter, CountTypesMatchesTheFormattedList) {
EXPECT_EQ(0, Filter::countTypes(0));
EXPECT_EQ(16, Filter::countTypes(Filter::kAllPacketTypes));
EXPECT_EQ(1, Filter::countTypes(static_cast<uint16_t>(1u << 15)));
for (uint32_t mask = 0; mask <= 0xFFFFu; ++mask) {
uint8_t expected = 0;
for (uint8_t type = 0; type <= 15; ++type) {
if ((mask & (1u << type)) != 0) expected++;
}
ASSERT_EQ(expected, Filter::countTypes(static_cast<uint16_t>(mask))) << mask;
}
}
// The diag fallback exists because a clipped list parses as a real, different
// allowlist. Pin that hazard so the fallback is never "simplified" away.
TEST(MQTTPacketFilter, ClippedListStillParsesAsADifferentAllowlist) {
const uint16_t configured = static_cast<uint16_t>(Filter::kAllPacketTypes & ~1u);
char text[Filter::kFilterTextSize];
ASSERT_TRUE(Filter::format(configured, text, sizeof(text)));
ASSERT_STREQ("1,2,3,4,5,6,7,8,9,10,11,12,13,14,15", text);
// Drop the trailing "15" the way a bounded reply buffer would.
char clipped[Filter::kFilterTextSize];
memcpy(clipped, text, sizeof(clipped));
clipped[strlen("1,2,3,4,5,6,7,8,9,10,11,12,13,14,")] = '\0';
uint16_t misread = 0;
ASSERT_FALSE(Filter::parse(clipped, &misread)); // trailing comma is rejected
clipped[strlen("1,2,3,4,5,6,7,8,9,10,11,12,13,14")] = '\0';
ASSERT_TRUE(Filter::parse(clipped, &misread)); // but one char earlier reads clean
EXPECT_NE(configured, misread) << "a clipped list must not be mistaken for the real one";
EXPECT_EQ(14, Filter::countTypes(misread));
EXPECT_EQ(15, Filter::countTypes(configured));
}
int main(int argc, char** argv) {
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
@@ -5,6 +5,7 @@
#define WITH_MQTT_BRIDGE 1
#include "helpers/MQTTPrefsCodec.h"
#include "helpers/MQTTPacketFilter.h"
namespace Codec = MQTTPrefsCodec;
@@ -20,6 +21,7 @@ MQTTPrefs defaults() {
prefs.wifi_power_save = 1;
for (int i = 0; i < MQTT_PREFS_SLOT_COUNT; ++i) {
strncpy(prefs.mqtt_slot_preset[i], "none", sizeof(prefs.mqtt_slot_preset[i]) - 1);
prefs.mqtt_slot_packet_filter[i] = MQTTPacketFilter::kAllPacketTypes;
}
strncpy(prefs.snmp_community, "public", sizeof(prefs.snmp_community) - 1);
prefs.radio_watchdog_minutes = 5;
@@ -218,6 +220,10 @@ TEST(MQTTPrefsCodec, MigratesAllLegacySixSlotPrefixesWithoutClobberingDefaults)
} else {
EXPECT_STREQ("time.example", prefs.mqtt_ntp_server);
}
for (int slot = 0; slot < MQTT_PREFS_SLOT_COUNT; ++slot) {
EXPECT_EQ(MQTTPacketFilter::kAllPacketTypes,
prefs.mqtt_slot_packet_filter[slot]) << size << ":" << slot;
}
}
}
@@ -228,6 +234,9 @@ TEST(MQTTPrefsCodec, CurrentVersionedPayloadRoundTripsExactly) {
strncpy(source.alert_region, "PNW", sizeof(source.alert_region) - 1);
source.mqtt_neighbors_enabled = 1;
source.mqtt_neighbors_interval = MQTT_NEIGHBORS_MAX_INTERVAL_MS;
for (int i = 0; i < MQTT_PREFS_SLOT_COUNT; ++i) {
source.mqtt_slot_packet_filter[i] = static_cast<uint16_t>(1u << i);
}
std::vector<uint8_t> bytes(Codec::kEncodedSize);
ASSERT_EQ(Codec::kEncodedSize, Codec::encode(source, bytes.data(), bytes.size()));
@@ -240,6 +249,114 @@ TEST(MQTTPrefsCodec, CurrentVersionedPayloadRoundTripsExactly) {
EXPECT_EQ(0, memcmp(&source, &loaded, sizeof(source)));
}
// /mqtt_prefs also carries the WiFi credentials, so a payload older firmware
// rejects strands a downgraded node with no network and no way to save. The
// filter tail is only written once it actually holds something.
TEST(MQTTPrefsCodec, DefaultFiltersKeepTheDowngradeReadablePayloadLength) {
MQTTPrefs source = defaults();
strncpy(source.wifi_ssid, "home-net", sizeof(source.wifi_ssid) - 1);
strncpy(source.mqtt_slot_preset[0], "analyzer-us", sizeof(source.mqtt_slot_preset[0]) - 1);
ASSERT_EQ(Codec::kV1PreFilterPayloadSize, Codec::payloadLenFor(source));
std::vector<uint8_t> bytes(Codec::kEncodedSize, 0xEE);
const size_t written = Codec::encode(source, bytes.data(), bytes.size());
ASSERT_EQ(sizeof(MQTTPrefsHeader) + Codec::kV1PreFilterPayloadSize, written);
EXPECT_LT(written, Codec::kEncodedSize);
MQTTPrefsHeader header;
memcpy(&header, bytes.data(), sizeof(header));
EXPECT_EQ(Codec::kV1PreFilterPayloadSize, header.payload_len);
// Nothing past the boundary was touched, so the file really is the short one.
for (size_t i = written; i < bytes.size(); ++i) {
EXPECT_EQ(0xEE, bytes[i]) << i;
}
bytes.resize(written);
const Codec::DecodePlan plan = classify(bytes);
ASSERT_EQ(Codec::Source::Current, plan.source);
ASSERT_EQ(Codec::kV1PreFilterPayloadSize, plan.payload_len);
ASSERT_FALSE(plan.preserve_file);
MQTTPrefs loaded = defaults();
memcpy(&loaded, bytes.data() + sizeof(MQTTPrefsHeader), plan.payload_len);
EXPECT_EQ(0, memcmp(&source, &loaded, sizeof(source)));
}
TEST(MQTTPrefsCodec, AnyNonDefaultFilterOptsIntoTheLongerPayload) {
MQTTPrefs source = defaults();
strncpy(source.wifi_ssid, "home-net", sizeof(source.wifi_ssid) - 1);
// "none" is as much a real setting as a subset, and must survive a reload.
for (int slot = 0; slot < MQTT_PREFS_SLOT_COUNT; ++slot) {
for (uint16_t mask : {static_cast<uint16_t>(0),
static_cast<uint16_t>(1u << 4),
static_cast<uint16_t>(MQTTPacketFilter::kAllPacketTypes & ~1u)}) {
source.mqtt_slot_packet_filter[slot] = mask;
ASSERT_EQ(Codec::kV1BaselinePayloadSize, Codec::payloadLenFor(source)) << slot;
std::vector<uint8_t> bytes(Codec::kEncodedSize);
ASSERT_EQ(Codec::kEncodedSize, Codec::encode(source, bytes.data(), bytes.size()));
const Codec::DecodePlan plan = classify(bytes);
ASSERT_EQ(Codec::kV1BaselinePayloadSize, plan.payload_len);
MQTTPrefs loaded = defaults();
memcpy(&loaded, bytes.data() + sizeof(MQTTPrefsHeader), plan.payload_len);
EXPECT_EQ(mask, loaded.mqtt_slot_packet_filter[slot]) << slot;
EXPECT_EQ(0, memcmp(&source, &loaded, sizeof(source)));
}
source.mqtt_slot_packet_filter[slot] = MQTTPacketFilter::kAllPacketTypes;
}
// Clearing the last non-default filter returns the node to the short payload.
EXPECT_EQ(Codec::kV1PreFilterPayloadSize, Codec::payloadLenFor(source));
}
TEST(MQTTPrefsCodec, EncodeRefusesAnOutputBufferShorterThanItsChosenPayload) {
MQTTPrefs shortest = defaults();
const size_t short_size = sizeof(MQTTPrefsHeader) + Codec::kV1PreFilterPayloadSize;
std::vector<uint8_t> bytes(Codec::kEncodedSize);
EXPECT_EQ(0u, Codec::encode(shortest, bytes.data(), short_size - 1));
EXPECT_EQ(short_size, Codec::encode(shortest, bytes.data(), short_size));
EXPECT_EQ(0u, Codec::encode(shortest, nullptr, bytes.size()));
// A buffer that fits the short payload is not enough for the long one.
MQTTPrefs longest = defaults();
longest.mqtt_slot_packet_filter[0] = 0;
EXPECT_EQ(0u, Codec::encode(longest, bytes.data(), short_size));
EXPECT_EQ(Codec::kEncodedSize, Codec::encode(longest, bytes.data(), bytes.size()));
}
TEST(MQTTPrefsCodec, PreFilterV1PayloadDefaultsEverySlotToAllTypes) {
MQTTPrefs source = defaults();
strncpy(source.mqtt_origin, "pre-filter-node", sizeof(source.mqtt_origin) - 1);
source.mqtt_neighbors_enabled = 1;
source.mqtt_neighbors_interval = MQTT_NEIGHBORS_MAX_INTERVAL_MS;
for (int i = 0; i < MQTT_PREFS_SLOT_COUNT; ++i) {
source.mqtt_slot_packet_filter[i] = 0;
}
std::vector<uint8_t> bytes(sizeof(MQTTPrefsHeader) + Codec::kV1PreFilterPayloadSize, 0);
writeHeader(&bytes, MQTT_PREFS_VERSION,
static_cast<uint16_t>(Codec::kV1PreFilterPayloadSize));
memcpy(bytes.data() + sizeof(MQTTPrefsHeader), &source, Codec::kV1PreFilterPayloadSize);
const Codec::DecodePlan plan = classify(bytes);
ASSERT_EQ(Codec::Source::Current, plan.source);
ASSERT_EQ(Codec::kV1PreFilterPayloadSize, plan.payload_len);
ASSERT_TRUE(plan.observer_fields_present);
ASSERT_FALSE(plan.preserve_file);
MQTTPrefs loaded = defaults();
memcpy(&loaded, bytes.data() + sizeof(MQTTPrefsHeader), plan.payload_len);
EXPECT_STREQ("pre-filter-node", loaded.mqtt_origin);
EXPECT_EQ(1u, loaded.mqtt_neighbors_enabled);
EXPECT_EQ(MQTT_NEIGHBORS_MAX_INTERVAL_MS, loaded.mqtt_neighbors_interval);
for (int i = 0; i < MQTT_PREFS_SLOT_COUNT; ++i) {
EXPECT_EQ(MQTTPacketFilter::kAllPacketTypes,
loaded.mqtt_slot_packet_filter[i]) << i;
}
}
TEST(MQTTPrefsCodec, CompatibleShortV1PayloadPreservesDefaultsBeyondObserverBoundary) {
MQTTPrefs source = defaults();
strncpy(source.mqtt_origin, "short-v1-node", sizeof(source.mqtt_origin) - 1);
@@ -304,6 +421,10 @@ TEST(MQTTPrefsCodec, PreNeighborsV1PayloadLoadsObserverFieldsAndDefaultsNeighbor
// Interval begins at 2860 (beyond the read) -> keeps the caller's default.
EXPECT_EQ(0u, loaded.mqtt_neighbors_enabled);
EXPECT_EQ(MQTT_NEIGHBORS_DEFAULT_INTERVAL_MS, loaded.mqtt_neighbors_interval);
for (int i = 0; i < MQTT_PREFS_SLOT_COUNT; ++i) {
EXPECT_EQ(MQTTPacketFilter::kAllPacketTypes,
loaded.mqtt_slot_packet_filter[i]) << i;
}
}
TEST(MQTTPrefsCodec, CorruptOrShortVersionedInputsArePreserved) {
@@ -378,18 +499,83 @@ TEST(MQTTPrefsCodec, UnsupportedHeaderlessSizesArePreserved) {
}
}
TEST(MQTTPrefsCodec, NewerAndSameVersionExtendedPayloadsAreHeld) {
// A different version tag means the layout may have changed shape, so it is
// still refused outright. That refusal is what makes reading longer same-version
// payloads safe, so the two belong in one test.
TEST(MQTTPrefsCodec, ADifferentVersionTagIsStillRefusedAndHeld) {
std::vector<uint8_t> newer(sizeof(MQTTPrefsHeader), 0);
writeHeader(&newer, MQTT_PREFS_VERSION + 1, 0);
Codec::DecodePlan plan = classify(newer);
const Codec::DecodePlan plan = classify(newer);
EXPECT_EQ(Codec::Source::UnsupportedVersion, plan.source);
EXPECT_TRUE(plan.preserve_file);
}
std::vector<uint8_t> extended(sizeof(MQTTPrefsHeader) + Codec::kV1BaselinePayloadSize + 1, 0);
writeHeader(&extended, MQTT_PREFS_VERSION,
static_cast<uint16_t>(Codec::kV1BaselinePayloadSize + 1));
plan = classify(extended);
EXPECT_EQ(Codec::Source::UnsupportedVersion, plan.source);
// The downgrade contract: a v1 payload longer than this build's baseline was
// appended to by a later build, so the baseline prefix is present verbatim.
// Read it and drop the tail — never refuse the file, which would cost the
// operator WiFi and every broker slot to save settings they don't understand.
TEST(MQTTPrefsCodec, LongerSameVersionPayloadLoadsTheBaselineAndIgnoresTheTail) {
MQTTPrefs source = defaults();
strncpy(source.mqtt_origin, "future-node", sizeof(source.mqtt_origin) - 1);
strncpy(source.wifi_ssid, "field-ssid", sizeof(source.wifi_ssid) - 1);
strncpy(source.wifi_password, "field-secret", sizeof(source.wifi_password) - 1);
strncpy(source.mqtt_iata, "SEA", sizeof(source.mqtt_iata) - 1);
strncpy(source.mqtt_slot_preset[0], "meshrank", sizeof(source.mqtt_slot_preset[0]) - 1);
source.mqtt_neighbors_enabled = 1;
for (int i = 0; i < MQTT_PREFS_SLOT_COUNT; ++i) {
source.mqtt_slot_packet_filter[i] = static_cast<uint16_t>(1u << i);
}
// Baseline image plus a 40-byte tail of fields this build has never heard of.
const size_t kTail = 40;
const size_t payload_len = Codec::kV1BaselinePayloadSize + kTail;
std::vector<uint8_t> bytes(sizeof(MQTTPrefsHeader) + payload_len, 0xA5);
writeHeader(&bytes, MQTT_PREFS_VERSION, static_cast<uint16_t>(payload_len));
memcpy(bytes.data() + sizeof(MQTTPrefsHeader), &source, sizeof(source));
const Codec::DecodePlan plan = classify(bytes);
ASSERT_EQ(Codec::Source::Current, plan.source);
EXPECT_FALSE(plan.preserve_file) << "refusing the file would strand the node";
EXPECT_TRUE(plan.observer_fields_present);
EXPECT_FALSE(plan.rewrite_legacy);
ASSERT_EQ(Codec::kV1BaselinePayloadSize, plan.payload_len);
// Reading plan.payload_len bytes recovers this build's whole struct exactly,
// and cannot run past it into the unknown tail.
MQTTPrefs loaded = defaults();
memcpy(&loaded, bytes.data() + sizeof(MQTTPrefsHeader), plan.payload_len);
EXPECT_EQ(0, memcmp(&source, &loaded, sizeof(source)));
EXPECT_STREQ("future-node", loaded.mqtt_origin);
EXPECT_STREQ("field-ssid", loaded.wifi_ssid);
EXPECT_STREQ("field-secret", loaded.wifi_password);
EXPECT_STREQ("meshrank", loaded.mqtt_slot_preset[0]);
EXPECT_EQ(1u, loaded.mqtt_neighbors_enabled);
EXPECT_EQ(1u, loaded.mqtt_slot_packet_filter[0]);
}
// The rule has to hold for any appended size, including a single byte and a
// tail far larger than the baseline.
TEST(MQTTPrefsCodec, EveryLongerSameVersionLengthReadsTheBaseline) {
for (const size_t tail : {size_t(1), size_t(2), size_t(12), size_t(64),
size_t(512), size_t(4096)}) {
const size_t payload_len = Codec::kV1BaselinePayloadSize + tail;
std::vector<uint8_t> bytes(sizeof(MQTTPrefsHeader) + payload_len, 0);
writeHeader(&bytes, MQTT_PREFS_VERSION, static_cast<uint16_t>(payload_len));
const Codec::DecodePlan plan = classify(bytes);
EXPECT_EQ(Codec::Source::Current, plan.source) << tail;
EXPECT_FALSE(plan.preserve_file) << tail;
EXPECT_EQ(Codec::kV1BaselinePayloadSize, plan.payload_len) << tail;
}
}
// A length that merely *declares* a longer payload without the bytes to back it
// is still corrupt — the header/file-size agreement check must run first.
TEST(MQTTPrefsCodec, LongerDeclaredLengthWithoutTheBytesIsStillCorrupt) {
std::vector<uint8_t> bytes(sizeof(MQTTPrefsHeader) + Codec::kV1BaselinePayloadSize, 0);
writeHeader(&bytes, MQTT_PREFS_VERSION,
static_cast<uint16_t>(Codec::kV1BaselinePayloadSize + 16));
const Codec::DecodePlan plan = classify(bytes);
EXPECT_EQ(Codec::Source::Corrupt, plan.source);
EXPECT_TRUE(plan.preserve_file);
}
@@ -22,6 +22,7 @@ TEST(WebConfigKeys, AllowsPerSlotKeys) {
EXPECT_TRUE(wcIsAllowedSetKey("mqtt1.preset"));
EXPECT_TRUE(wcIsAllowedSetKey("mqtt1.server"));
EXPECT_TRUE(wcIsAllowedSetKey("mqtt1.token"));
EXPECT_TRUE(wcIsAllowedSetKey("mqtt2.filter"));
EXPECT_TRUE(wcIsAllowedSetKey("mqtt6.audience")); // MAX_MQTT_SLOTS == 6
}
@@ -102,6 +103,7 @@ TEST(WebConfigKeys, NonSecretKeysNotFlagged) {
EXPECT_FALSE(wcIsSecretKey("wifi.ssid"));
EXPECT_FALSE(wcIsSecretKey("mqtt1.username")); // username is not masked
EXPECT_FALSE(wcIsSecretKey("mqtt1.server"));
EXPECT_FALSE(wcIsSecretKey("mqtt1.filter"));
EXPECT_FALSE(wcIsSecretKey("mqtt.origin"));
EXPECT_FALSE(wcIsSecretKey("name"));
EXPECT_FALSE(wcIsSecretKey(""));
+137 -4
View File
@@ -41,6 +41,18 @@ input[type=text],input[type=password],input[type=number],input[type=email],selec
border:1px solid var(--in-line);border-radius:8px;outline:none} /* >=16px: stops iOS zoom-on-focus (sideways-scroll in captive sheet) */
input:focus,select:focus{border-color:var(--acc);box-shadow:0 0 0 2px color-mix(in srgb,var(--acc) 25%,transparent)}
input.dirty,select.dirty{border-color:var(--warn)}
/* packet-type allowlist: auto-fit so six slots stay readable on a phone */
.pkg{display:grid;grid-template-columns:repeat(auto-fill,minmax(122px,1fr));gap:2px 8px;
border:1px solid var(--in-line);border-radius:8px;padding:8px;background:var(--in-bg)}
.f.dirty .pkg{border-color:var(--warn)}
.pk{display:flex;align-items:center;gap:6px;font-size:12.5px;font-weight:400;margin:0;
min-height:26px;cursor:pointer}
.pk input{width:15px;height:15px;margin:0;flex:none}
.pk span{overflow:hidden;text-overflow:ellipsis;white-space:nowrap}
.pka{display:flex;align-items:center;gap:10px;margin-top:5px;font-size:11.5px}
.lnk{background:none;border:none;padding:0;font:inherit;font-size:11.5px;color:var(--acc);
cursor:pointer;text-decoration:underline}
.pfsum{color:var(--mut);margin-left:auto}
.row{display:flex;gap:10px}.row>.f{flex:1;min-width:0}
.sw{display:flex;align-items:center;justify-content:space-between;gap:10px;padding:9px 0}
.sw span b{display:block;font-size:13.5px;font-weight:600}
@@ -468,7 +480,7 @@ function api(path,opts){
if(!r.ok&&r.status!==202){
// Carry the HTTP status and any batch reqid so callers can tell a
// definite rejection (400/409/413) from an ambiguous network failure.
var e=new Error(j.error||("HTTP "+r.status));e.status=r.status;e.reqid=j.reqid;throw e;
var e=new Error(j.error||("HTTP "+r.status));e.status=r.status;e.reqid=j.reqid;e.key=j.key;throw e;
}
return j;
});
@@ -579,6 +591,7 @@ function loadConfig(){
$$("[data-rg]").forEach(function(el){el.value=String(c.radio[el.dataset.rg]);el.classList.remove("dirty")});
st.orig.radio=radioCombo();afHint();
for(var i=1;i<=st.nslots;i++){refreshSlotFields($("#app-slots"),i);refreshSlotFields($("#wz-slots"),i)}
syncPacketFilters();
updateSaveBar();
});
}
@@ -627,11 +640,99 @@ function revertAll(){loadConfig().then(function(){toast("Reverted")})}
// maxlength per destination MQTTPrefs buffer minus the NUL (host[64],
// username[32], password[64], token[48], topic[96], audience[64]). Bounds the
// common browser case; the backend still owns authoritative length limits.
// mqttN.filter is deliberately absent: it is a bitmask, not a text buffer, and
// its accepted input (names, spaces) is longer than its canonical output — a
// maxlength here would silently truncate a valid list into a different one.
var SLOT_MAXLEN={token:47,username:31,password:63,server:63,topic:95,audience:63};
// Payload types, mirroring PAYLOAD_TYPE_* in src/Packet.h. 12-14 are reserved
// upstream with no name, so they are offered by number only.
var PKT_TYPES=[[0,"REQ"],[1,"RESPONSE"],[2,"TXT_MSG"],[3,"ACK"],[4,"ADVERT"],
[5,"GRP_TXT"],[6,"GRP_DATA"],[7,"ANON_REQ"],[8,"PATH"],[9,"TRACE"],
[10,"MULTIPART"],[11,"CONTROL"],[12,"reserved"],[13,"reserved"],[14,"reserved"],
[15,"RAW_CUSTOM"]];
// Name -> type, for the names MQTTPacketFilter::resolveToken actually accepts.
// "reserved" is a shared display label for 12-14, not a spelling the firmware
// resolves, so it is excluded — otherwise "reserved" would silently mean 14.
// Null-prototype so a token like "constructor" can't match an inherited key.
var PKT_NAMED=Object.create(null);
PKT_TYPES.forEach(function(p){if(p[1]!=="reserved")PKT_NAMED[p[1].toLowerCase()]=p[0]});
// Mirrors MQTTPacketFilter::parse/format. The text form must round-trip the
// firmware's canonical output exactly, or an unedited slot reads as dirty.
function filterMask(text){
var t=String(text==null?"":text).trim();
// Empty is "all" on purpose — it is what the firmware stores for a missing
// or cleared value, so an absent `filter` in /api/config reads as unfiltered
// rather than as a slot that uploads nothing.
if(t===""||t==="all")return 0xFFFF;
if(t==="none")return 0;
var mask=0,ok=true;
t.split(",").forEach(function(part){
var tok=part.trim(),n=-1;
if(/^[0-9]+$/.test(tok))n=parseInt(tok,10);
else if(tok.toLowerCase() in PKT_NAMED)n=PKT_NAMED[tok.toLowerCase()];
if(n<0||n>15){ok=false;return}
mask|=(1<<n);
});
// The device only ever emits canonical text; anything else means a value we
// did not write, so show it as unfiltered rather than inventing a subset.
return ok?mask:0xFFFF;
}
function filterText(mask){
if(mask===0xFFFF)return"all";
if(mask===0)return"none";
var out=[];
for(var i=0;i<16;i++)if(mask&(1<<i))out.push(i);
return out.join(",");
}
function slotFieldHtml(i,f,type,label,hint){
var ml=SLOT_MAXLEN[f]?' maxlength="'+SLOT_MAXLEN[f]+'"':'';
return'<div class="f sf-'+f+'"><label>'+label+'</label><input type="'+type+'" data-k="mqtt'+i+'.'+f+'"'+ml+' autocapitalize="off" autocorrect="off" autocomplete="off">'+(hint?'<div class="hint">'+hint+'</div>':'')+'</div>';
}
// Checkboxes rather than a text field: the accepted grammar is wider than the
// canonical form, so free text invites a value that saves as something else.
// The hidden input stays the single source of truth for the generic dirty /
// save / result-chip machinery, which is keyed on [data-k].
function packetFilterHtml(i){
var boxes=PKT_TYPES.map(function(p){
return'<label class="pk"><input type="checkbox" data-pf="'+i+'" value="'+p[0]+'"> <span>'+p[0]+' '+p[1]+'</span></label>';
}).join("");
return'<div class="f sf-filter"><label>Packet types</label>'
+'<input type="hidden" data-k="mqtt'+i+'.filter">'
+'<div class="pkg">'+boxes+'</div>'
+'<div class="pka"><button type="button" class="lnk" data-pfset="'+i+':all">All</button>'
+'<button type="button" class="lnk" data-pfset="'+i+':none">None</button>'
+'<span class="pfsum"></span></div>'
+'<div class="hint">Which payload types are uploaded to this broker. Applies to both packet and raw uploads; status and neighbors are never filtered. Clearing every box uploads nothing.</div></div>';
}
// Reflect each hidden filter value into its checkboxes and summary. Cheap
// enough to re-run wholesale after any load or edit.
function syncPacketFilters(){
$$('[data-k$=".filter"]').forEach(function(h){
var i=h.dataset.k.match(/^mqtt([1-9])\./)[1];
var card=h.closest(".slot");if(!card)return;
var mask=filterMask(h.value);
card.querySelectorAll('[data-pf="'+i+'"]').forEach(function(cb){
cb.checked=(mask&(1<<parseInt(cb.value,10)))!==0;
});
var sum=card.querySelector(".pfsum");
if(sum)sum.textContent=mask===0xFFFF?"all types":(mask===0?"nothing uploaded":filterText(mask));
var wrap=h.closest(".f");
if(wrap)wrap.classList.toggle("dirty",h.dataset.k in st.dirty);
});
}
// Recompute one slot's canonical value from its boxes and push it through the
// same markDirty path a typed field would use.
function packetFilterChanged(i,card){
var mask=0;
card.querySelectorAll('[data-pf="'+i+'"]').forEach(function(cb){
if(cb.checked)mask|=(1<<parseInt(cb.value,10));
});
var k="mqtt"+i+".filter",v=filterText(mask);
$$('[data-k="'+k+'"]').forEach(function(h){h.value=v});
markDirty(k,v);
syncPacketFilters();
}
function buildSlots(sel){
var host=$(sel);if(!host)return;
var html="";
@@ -649,6 +750,7 @@ function buildSlots(sel){
+slotFieldHtml(i,"port","number","Port","")
+slotFieldHtml(i,"topic","text","Topic template","Placeholders: {iata} {device} {type} {token}")
+slotFieldHtml(i,"audience","text","JWT audience","Optional — blank for user/pass auth")
+packetFilterHtml(i)
+'</div>';
}
host.innerHTML=html;
@@ -662,7 +764,7 @@ function refreshSlotFields(host,i){
var needs=name==="custom"?"custom":(def?def.needs:"none");
var showMap={token:needs==="token",username:needs==="userpass"||needs==="custom",
password:needs==="userpass"||needs==="password"||needs==="custom",server:needs==="custom",
port:needs==="custom",topic:needs==="custom",audience:needs==="custom"};
port:needs==="custom",topic:needs==="custom",audience:needs==="custom",filter:name!=="none"};
Object.keys(showMap).forEach(function(f){
var w=card.querySelector(".sf-"+f);if(w)w.classList.toggle("hide",!showMap[f]);
});
@@ -671,8 +773,21 @@ document.addEventListener("change",function(ev){
var el=ev.target,m;
if(el.dataset.k&&(m=el.dataset.k.match(/^mqtt([1-9])\.preset$/))){
["#app-slots","#wz-slots"].forEach(function(sel){refreshSlotFields($(sel),+m[1])});
}else if(el.dataset.pf){
var card=el.closest(".slot");
if(card)packetFilterChanged(el.dataset.pf,card);
}
});
document.addEventListener("click",function(ev){
var b=ev.target.closest?ev.target.closest("[data-pfset]"):null;
if(!b)return;
var parts=b.dataset.pfset.split(":"),i=parts[0],all=parts[1]==="all";
// Write through the same path a checkbox edit takes, so both the app and
// wizard copies of this slot stay in step.
var card=b.closest(".slot");if(!card)return;
card.querySelectorAll('[data-pf="'+i+'"]').forEach(function(cb){cb.checked=all});
packetFilterChanged(i,card);
});
/* ---------- save / results ---------- */
function chipFor(el,ok,msg){
@@ -701,7 +816,19 @@ function saveChanges(extra,cb){
// result to poll for. Report it instead of chasing a phantom save.
if(e.status===400||e.status===413){
$("#save-btn").disabled=false;
if(cb)cb(null,false);else toast(e.message||"Save rejected");
// A whole-batch rejection names the offending key; chip it so the
// operator doesn't have to guess which of N edited fields was at fault.
// Every copy, not querySelector: the wizard's slot cards precede the
// app's in document order, so one lookup always lands on the hidden one.
// Comparing the property also keeps a device-echoed key — "bad key"
// returns whatever was sent — out of the selector grammar, where a quote
// would throw SyntaxError and swallow the rest of this handler.
if(e.key){
$$("[data-k]").forEach(function(el){
if(el.dataset.k===e.key)chipFor(el,false,e.message||"rejected");
});
}
if(cb)cb(null,false);else toast((e.message||"Save rejected")+(e.key?": "+e.key:""));
return;
}
// 409 busy: a batch is in flight. If it's ours (same reqid — our POST
@@ -825,6 +952,7 @@ function loadConfigSoft(){ // re-sync accepted values without clobbering chips/r
el.value=radioParts[idx];el.classList.toggle("dirty",keepRadio);
});
for(var i=1;i<=st.nslots;i++){refreshSlotFields($("#app-slots"),i);refreshSlotFields($("#wz-slots"),i)}
syncPacketFilters();
afHint();
updateSaveBar();
}).catch(function(){});
@@ -879,7 +1007,12 @@ function pollStats(){
spark($("#spark-heap"),st.hist.heap);spark($("#spark-noise"),st.hist.noise);
var sl=s.slots||[];
$("#stat-slots").innerHTML=sl.length?sl.map(function(x){
return'<div class="kv"><i>Slot '+x.n+" · "+esc(x.name)+'</i><b><span class="pill '+esc(x.state)+'">'+esc(x.state)+'</span> '+x.ok+' ok / '+x.err+' err</b></div>';
// "filt" is sent only when the slot is not on the all-types default. A
// healthy slot that has stopped publishing is otherwise indistinguishable
// from an idle one.
var f=(x.filt==null)?"":'<div class="kv"><i></i><b style="font-weight:400;color:var(--mut)">filter: '
+esc(x.filt===0?"none (nothing uploaded)":filterText(x.filt))+'</b></div>';
return'<div class="kv"><i>Slot '+x.n+" · "+esc(x.name)+'</i><b><span class="pill '+esc(x.state)+'">'+esc(x.state)+'</span> '+x.ok+' ok / '+x.err+' err</b></div>'+f;
}).join(""):"No servers configured.";
}).catch(function(){}).then(function(){
// Runs whether the poll resolved or rejected: clear the guard, then schedule