MeshCore/docs/management_reports.md

<!-- meshcore-hosted-doc-link:start -->
<p class="meshcore-hosted-doc-link"><a href="https://mikecarper.github.io/MeshCore/management_reports/">View this page on MeshCore Docs</a>.</p>
<!-- meshcore-hosted-doc-link:end -->

# Management reports (MGR1/MGR2)

Available on repeater (including observer), room-server and sensor firmware.
Exception: the constrained `wio-e5-mini_repeater` omits management reporting
and shared `data.tx` routing to fit its existing application region. Normal
authenticated administration, statistics, live USB/MQTT logging and the
host-service bridge remain available.
Companions/terminal-chat nodes and KISS modems do not originate these reports.
Off by default. A management password and explicit enable are both required.
No radio settings, existing preferences layout, or OTA authorization policy is
changed by enabling reporting. This protocol does **not** authorize updates.

Current firmware emits `MGR2`, including a public USB logging/recovery snapshot
and its latest watchdog event. Updated decoders still accept released `MGR1`;
older decoder copies need an update to read MGR2. The earlier USB-only MGR2
development layout was never released and was replaced before release.
Reporting USB status does not enable logging or change retry policy.

## CLI

Run through the existing local CLI or an authenticated administrator session:

```
get data.tx
set data.tx path 1:12ab77
set data.tx region auto
set mgmt.password <12-to-96-byte password>
set mgmt.direct 5
set mgmt.flood 21
set mgmt.enabled on
get mgmt
set mgmt.enabled off
```

Use a long randomly generated password. There is no password getter. Firmware
stores the derived 32-byte key, not the plaintext password. That stored key is
equivalent authority to decrypt reports and must also be protected. Passwords
entered into terminal programs may still be recorded by those programs.
The recognized firmware password command wipes its value from the input buffer
on success and error, including unavailable management or allocation failure.

`data.tx` is the single shared route for management reports, telemetry history,
and future scheduled data producers. Its fresh-install defaults are `path=direct`
(zero hops) and `region=auto`; configuring it never enables a producer. Paths
use `1:`, `2:` or `3:` followed by complete hop hashes without separators, or
the comma-separated form accepted by `set outpath`. Empty hops, including a
trailing comma, are rejected. `none` removes the path.
`get/set mgmt.path` remain compatibility aliases for `get/set data.tx path`.
Alias expansion that exceeds the command buffer is rejected, not truncated.

`region=auto` uses the radio's configured default region when it is usable;
otherwise it resolves the unique deepest flood-enabled entry in the region
hierarchy. If equally deep candidates make that choice ambiguous, it resolves
to nothing and no fallback is transmitted. `default` requires and follows
`region default`; a region name pins that named scope. `none` disables scoped fallback. The
resolved transport key is looked up when a report starts, so edits to the region
definition take effect without rewriting `/data_tx`.

The direct path leads to the receiver/uplink's vicinity; this broadcast-radio
datagram has no private destination identity. The radio ID in the payload
identifies the reporter.

Direct and flood schedules are independently configurable and may each be
turned off. Fresh settings are `direct=5d` and `flood=21d`; global reporting is
still off until `mgmt.enabled on`. Direct accepts 5-90 days and requires the
shared path. Flood accepts 21-90 days and requires a resolvable shared region.
At least one route must remain active while reporting is enabled. The legacy
`set mgmt.interval N` shorthand enables both, setting direct to `N` and flood to
`max(21,N)`. No transmission is sent merely by configuring a password. Initial
reporting waits for the configured schedules. Invalid settings leave saved
configuration, countdowns, and any report pages already in progress unchanged.
Day counts must be unsigned decimal integers; signs and trailing text are rejected.
Reports have deterministic
per-radio/per-sequence jitter of up to an hour; pages are spaced at least a
minute apart. Watchdog events appear in the next scheduled snapshot, not as
immediate radio alerts. Multiple events between snapshots can replace the
latest-event record; this is not a complete off-node event history.

The radio cannot know that an observer uploaded a packet to MQTT, so the flood
schedule is deliberately independent of direct transmission. A region-scoped
`TRANSPORT_FLOOD` is sent at its independently configured interval. An
unresolved or ambiguous data region blocks that transmission
rather than sending an unscoped flood. If direct and flood become due together,
the flood is sent and replaces the redundant direct copy.
Ordinary reports are never retried in a tight loop; a partial report gives up
after an hour. Existing relay filters, hop limits and duty constraints still apply.

Schedule state is atomically reserved **before** transmission, and checkpointed
hourly. Timers use elapsed powered-on time rather than the RTC: clock corrections
cannot create floods, and reboot does not clear the budget. Downtime is not
credited; each reboot can delay a report by up to an additional hour. This is a
deliberately conservative tradeoff for nodes with unreliable clocks. Off/on and
password changes do not reset the flood limit. Corrupt/unreadable state or failed
writes stop reporting; `get mgmt` shows `FAULT(no TX)` until storage is repaired
and the radio restarted. `/management` and the shared `/data_tx` are versioned,
CRC-protected, and replaced transactionally.

On upgrade, an existing saved `data.tx` route takes precedence over the legacy
management-only path. Otherwise that path must be saved successfully to
`/data_tx` before reporting resumes. A failed migration leaves the old
`/management` file intact for retry after storage is repaired and the radio
restarted.

Weekly history is collected once a minute while enabled. Hour-bucket extrema
cover 7 days to 7 days + 1 hour (conservative boundary bucket). The first report
after enable/reboot is marked partial where appropriate. Since-report extrema
reset after all pages have been queued, retaining readings taken since the
snapshot for the next report. These statistics are not durable;
reboot loses the history, not the flood countdown. Temperature is MCU temperature,
not ambient. Reporting does not wake GPS, start Wi-Fi, or initialize external OTA
media. Unknown capabilities/readiness are explicitly distinguishable from false.
History/snapshot working memory is allocated only when reporting is enabled and
is bounded to 1.5 KiB, plus a small configuration object and temporary stack use.

## Routing and MQTT

Both direct/path and flood reports use **`PAYLOAD_TYPE_GRP_DATA` (`0x06`)**.
The route bits independently select direct or flood. `MGR1`/`MGR2` are application
extensions with **literally plaintext public fields**, not a call to the ordinary
encrypted `createGroupDatagram()` builder. A fixed public marker is not an owner
or password-derived channel ID. There is no outer channel encryption.

The body is padded with zeroes to a group-compatible length `3 + 16*n` (maximum
179 bytes). Existing repeaters in the checked upstream implementation route group
data without requiring a successful channel decryption. This fork recognizes the
management envelope before ordinary channel processing. Reception of a management
packet never exempts it from forwarding policy. Some third-party firmware may
apply additional channel/layout policies; interoperability with every fork is
not guaranteed. Packet logging/uplinks can capture it without the password.

`RAW_CUSTOM` (`0x0F`) is **not used for reports** because stock upstream does
not flood-route it.

The observer's existing MQTT `PACKET` JSON supplies the complete frame in `raw`.
No broker configuration or password changes are necessary to capture a report.
The decoder understands all four route forms, 1-3-byte hashes, scope transport
codes, and duplicate copies heard by several uplinks:

```
python -m pip install -r tools/management/requirements.txt
python tools/management/report.py --mqtt capture.jsonl
```

The password is prompted, not supplied as a process argument. Input may be JSONL
or a JSON array of MQTT messages. Alternatively omit `--mqtt` for a JSON array
of canonical payload hex strings. `--match-admin FULL_PUBLIC_KEY` matches a known
administrator against the encrypted fingerprints in either input mode. MQTT
captures match each radio's fingerprints separately. Invalid files or keys are
rejected before the password prompt; an unavailable password input gives a
command-line error instead of a traceback.
This is an offline capture decoder, not a broker subscriber or downlink service.

For a browser-local decoder that also decrypts and authenticates ACL entries,
use the [Management report decoder](management_decoder.md). It does not upload
the captured packet or password.

## Canonical MGR2 payload (little endian)

| Offset | Bytes | Field |
|---:|---:|---|
| 0 | 4 | `MGR2` |
| 4 | 16 | First 16 bytes of reporter public key |
| 20 | 4 | Persisted report sequence (never wraps; exhaustion stops TX) |
| 24 | 4 | RTC Unix timestamp; advisory, may be wrong |
| 28 | 4 | Firmware major/minor/patch/pre packed as mOTA version |
| 32 | 4 | Bootloader packed version, or unknown |
| 36 | 4 | EndF target ID |
| 40 | 8 | Complete mOTA delta-base body hash |
| 48 | 4 | EndF image length |
| 52 | 4 | Staging capacity; planning still checks actual package geometry |
| 56 | 4 | OTA capability bits |
| 60 | 2 | Uptime hours, saturated at 65535 |
| 62 | 4 | Weekly minimum mV, minimum C, maximum C |
| 66 | 4 | Since-report extrema in the same format |
| 70 | 1 | History coverage hours, capped at 168 |
| 71 | 1 | Interval days for this report's direct or flood schedule |
| 72 | 1 | Role: 1 repeater, 2 room server, 3 sensor |
| 73 | 1 | Compiled/detected capability bits |
| 74 | 1 | Active status bits |
| 75 | 1 | Mask of status bits whose state is known |
| 76 | 2 | Validity/partial-history flags |
| 78 | 1 | Zero-based page index |
| 79 | 1 | Total pages (1-9) |
| 80 | 1 | Unique ACL count (0-36) |
| 81 | 1 | First ACL index in this page |
| 82 | 1 | ACL entries on this page (0-4) |
| 83 | 2 | USB status flags |
| 85 | 1 | Packed USB recovery stage and retry backoff step |
| 86 | 4 | USB full-reboot retry interval seconds |
| 90 | 4 | USB inactive seconds |
| 94 | 4 | USB continuous Auto qualification seconds |
| 98 | 1 | Latest watchdog reason/action/durability code |
| 99 | 4 | Latest watchdog event RTC Unix epoch; advisory, zero if unavailable |
| 103 | 4 | Uptime seconds at that event's boot |
| 107 | 4 | Latest watchdog event sequence, saturated at 4294967295 |
| 111 | 13 * count | AES-SIV encrypted ACL entries |
| after ACL | 16 | Full AES-SIV authentication tag |
| after tag | 0-15 | Zero padding to group-compatible length; not part of canonical payload |

MGR2 retains all 36 ACL entries, using four per page and up to nine pages. The
largest canonical page is 179 bytes and already group-compatible;
the radio payload budget is unchanged. The USB snapshot is frozen with the
other public fields at report start and repeated on each page. It is not a
live host-health monitor.

USB flag bits 0-10 are, in order: supported, logging enabled, watchdog enabled,
host connected, reader connected, stalled, recovering, recovery deferred,
persistence ready, watchdog Auto, and logger active. Bits 11-15 must be zero.
Byte 85 bits 0-1 hold stage: 0 idle, 1 soft-recovery wait, 2 re-enumeration
wait, and 3 reboot pending. Bits 2-5 hold the saved backoff step; bits 6-7
must be zero. Backoff is not a count of successful recoveries. The three
timers are unsigned seconds; local recovery/reboot counters are not sent.
The retry field is the saved backoff interval, not a countdown to an action;
inactive age is reported separately. USB-only cleanup can precede that interval.
Unsupported USB observation/recovery is explicitly represented by a false
supported bit, not omitted or mistaken for a connected healthy reader.
USB status describes the radio's observed transport and policy, not proof
that the host process, MQTT uplink, SSH service, or Pi is healthy.
Host/reader connection alone does not mean a logging client is running.
The logger-active bit means recent USB stats polling within a 15-minute lease
while logging is enabled and the observed USB host/reader link is not stalled,
separate from USB attachment or an open reader. The existing five-minute stats
polls renew this lease; no new client-registration or heartbeat command is
required. This does not identify the client, prove logs reached MQTT or disk,
or show that the Pi remains otherwise healthy.

Missing watchdog state on new or upgraded installations defaults to Auto.
Auto promotes to saved On after 14 continuous healthy logging-USB days with
renewed stats polling; its qualification age resets on reboot, detach, stall,
stats-lease expiry, or USB logging off.
Explicit On and Off remain available. Nonlogging/no-reader nodes remain
Auto without watchdog board reboots. The Auto flag and age report this
qualification separately from an enabled On policy.
Use the node's `get usb.watchdog` and `set usb.watchdog off|on|auto` CLI to
inspect or change policy; receiving a management snapshot changes nothing.

The 13-byte latest-event block is all zero when no event is available. Otherwise
byte 98 bits 0-3 are the reason mask: host absent, reader absent, TX stalled,
and confirmed stats client inactive. Bits 4-6 hold the action: 1 software
recovery attempted, 2 re-enumeration attempted, 3 reboot requested, or 4 reboot
cancelled after a final health/ownership check. Bit 7 means the event was saved
and verified on durable storage. Sequence is nonzero for a present event.
Physical faults may combine reason bits; stats-only expiry cannot authorize a
board reboot. Epoch zero is allowed when the clock source is unavailable.

Event time is the node's advisory clock, not verified UTC. Default clocks can
start at a plausible fixed date. Clock corrections do not rewrite old events;
recorded uptime and sequence help review them independently. A saved reboot
request is intent, not proof of a completed reset. The latest event survives
normal reboot and mode changes, but is not a log of hardware hangs or sudden
power failures. See [USB logging watchdog](usb_logging_watchdog.md) for storage
failure, cancellation, and review limitations.

Legacy MGR1 retains the same offsets through byte 82, followed immediately
by encrypted ACL entries at byte 83 and the authentication tag. It carries
six entries per page and up to six pages, with no USB block. Its absent USB
status is decoded as unavailable, not as all switches off.

Each private entry is a per-radio 12-byte keyed fingerprint followed by flags:
bit 0 administrator, bit 1 trusted OTA signer. The firmware combines permissions
for the same fingerprint before paging, so every transmitted entry is unique.
Decoders reject authenticated duplicate fingerprints within or across pages;
repeated identical packet copies from multiple uplinks are still deduplicated.
An oversized ACL fails closed rather than silently truncating. The reported
allowlist includes all current full administrators and all four possible trusted
OTA signing keys; region/filter managers and ordinary clients are excluded.

Feature bits: 0 Wi-Fi, 1 GPS, 2 NTP, 3 USB data, 4 LoRa OTA. Wi-Fi active means
connected, GPS means receiver enabled (not necessarily a fix), NTP means an
actual accepted NTP response this boot, USB means observable native USB data
connection (not power or an unobservable external UART bridge). OTA capability
means compiled support; active means an established usable apply/store path.

Validity bits: 0 firmware version, 1 bootloader version, 2 EndF/base identity,
3 staging capacity, 4 partial week, 5 partial since-report period, 6 MCU temperature.
Voltage is unsigned millivolts, zero missing. Temperatures: zero missing,
1-251 represent -50..200 C, 252 below range, 253 above range, 254-255 reserved.
OTA bits: 0 protocol compiled, 1 transfer DEFLATE, 2 2-KiB app transfer blocks;
bits 8-23 are apply codec bits (full/sequential/in-place). Transfer DEFLATE is
not compressed bootloader apply. There is **no manifest ID**. Exact old binaries
are still needed on the computer to generate a differential update; a hash alone
cannot reconstruct them. Unknown metadata must not be treated as OTA readiness.

## Cryptography

Password root: `SHA256("#" || literal UTF-8 password)` (hashtag-style derivation).
Subkeys: `HMAC-SHA256(root, ASCII_domain || radio_id16)`.
Domains: `MeshCore-MGR1-SIV`, `MeshCore-MGR1-ACL`.
Fingerprint: `HMAC-SHA256(ACL_subkey, radio_id16 || full_admin_key32)[0:12]`.
The same owner has different fingerprints on different radios. A collector
needs a candidate administrator's full key to identify a fingerprint.

Encryption is [RFC 5297 AES-SIV-CMAC-256](https://www.rfc-editor.org/rfc/rfc5297),
using the existing rweather AES primitive. The single associated-data string is
the complete clear header: 111 bytes for MGR2, including USB status and its event,
or 83 bytes for MGR1. Both retain the existing derivation domains; the authenticated
magic and layout distinguish their wire versions. Ciphertext is exactly the ACL byte length.
Full 16-byte tag, no truncation. This deterministic misuse-resistant mode avoids
reliance on the firmware's noncryptographic general-purpose RNG or RTC nonces.
An identical restored snapshot may repeat ciphertext, but does not expose XORs
of different ACL plaintexts as nonce-reused stream encryption would.

All public fields are readable without a password but are authenticated only to
password holders. A shared password authenticates knowledge of that password,
**not** a unique individual radio identity; another holder can forge reports.
There is no per-device signature. Collectors must enforce their own persisted
sequence/replay policy and explicitly handle factory resets/restored backups.
The decoder verifies snapshot consistency but does not maintain a database.
This fast password derivation permits offline guessing, so twelve characters
is only a minimum length, not a guarantee of password strength. Packet timing,
public metadata and entry/page counts remain visible.