397 lines
20 KiB
Markdown
397 lines
20 KiB
Markdown
---
|
|
layout: default
|
|
title: Private fleet management
|
|
---
|
|
<!-- meshcore-hosted-doc-link:start -->
|
|
<p class="meshcore-hosted-doc-link"><a href="https://mikecarper.github.io/MeshCore/fleet_management/">View this page on MeshCore Docs</a>.</p>
|
|
<!-- meshcore-hosted-doc-link:end -->
|
|
|
|
|
|
# Private fleet management
|
|
|
|
Fleet control lets one publisher send filtering, radio schedules, secondary-radio
|
|
changes, and clock commands to
|
|
all enrolled infrastructure nodes, regions, a geographic radius, one node, or a list of targets over a
|
|
private LoRa channel.
|
|
It is disabled until configured. It is supported on repeaters and room servers;
|
|
fixed-size STM32 images omit it because of their flash limit.
|
|
Both the publishing Companion and receivers need firmware built with fleet-control
|
|
support; the existing app can use its local Command Line screen.
|
|
|
|
## Read-only receivers
|
|
|
|
Commands use two separate credentials:
|
|
|
|
- A random **128-bit channel key** encrypts the traffic. Receivers need this key
|
|
to read the private channel.
|
|
- The publisher's **Ed25519 public key** verifies commands. Only the publisher
|
|
holds the corresponding private key and can authorize changes.
|
|
|
|
Giving a repeater the channel key and publisher public key does **not** give it
|
|
the publisher's signing key. An ordinary channel message, a copied display name,
|
|
or possession of the channel key cannot authorize a fleet command. Each command
|
|
signature also binds the channel, target, sequence, expiry, and exact command.
|
|
|
|
This makes management commands read-only for receivers. It does not turn ordinary
|
|
MeshCore chat into a signed, read-only announcement channel: stock apps do not
|
|
verify signatures on ordinary channel text. Fleet acknowledgements are convenient
|
|
channel text, not authenticated inventory evidence. Use existing individually
|
|
authenticated administration for sensitive confirmation.
|
|
|
|
## Enroll nodes
|
|
|
|
Use an existing authenticated admin connection or the local console. Generate a
|
|
random private channel key; for example, `openssl rand -hex 16`. Do not use Public
|
|
or a predictable hashtag key. Add that same key as a private channel on the
|
|
publishing Companion, and note its channel index.
|
|
|
|
Get the publishing Companion's full public identity key from its device information
|
|
or existing identity command. On every managed repeater or room server, run:
|
|
|
|
```text
|
|
set fleet.controller <publisher-public-key-64-hex>
|
|
set fleet.channel <private-channel-key-32-hex>
|
|
get fleet.channel
|
|
get fleet.controller
|
|
get fleet.stats
|
|
```
|
|
|
|
`get fleet.channel` reports enrollment, storage health, a channel hash, and the
|
|
last accepted sequence. It does not reveal the channel key. `set fleet.channel
|
|
off` disables reception; `set fleet.controller off` revokes the publisher. Key
|
|
changes use the existing individually authenticated admin path, never fleet
|
|
commands.
|
|
`get fleet.stats` includes `parts`, the number of fragments currently buffered
|
|
across the two bounded, incomplete two-packet command slots.
|
|
|
|
Nodes need a clock set to current UTC. Commands expire after ten minutes and may
|
|
be at most one minute ahead of a receiver's clock. A bad clock fails closed;
|
|
correct it through the normal administration path.
|
|
|
|
## Send commands from the app
|
|
|
|
On the publishing Companion, use the existing local Command Line screen. The
|
|
Companion signs with its own identity. No publisher private key is copied to a
|
|
repeater, and no app protocol change is required.
|
|
|
|
Examples use private channel index `3`:
|
|
|
|
```text
|
|
fleet send 3 get radio2
|
|
fleet send 3 set tempradio2 910.5,500,8,5,rxtx,120
|
|
fleet send 3 set radio2.cross auto
|
|
fleet send 3 set flood.rule.4 type=grp_txt hops=6+ drop
|
|
fleet send 3 del flood.rule.4
|
|
```
|
|
|
|
Targets are optional: omitting the target sends to **all enrolled nodes** on the
|
|
private fleet channel. Explicit `all` still works, for example
|
|
`fleet send 3 all get radio2`.
|
|
|
|
To address particular nodes, insert **8 or 12 hexadecimal characters from the
|
|
start of a node's public key**, or its **complete 64-character public key**, before
|
|
the command. Separate multiple targets with semicolons, without spaces. Lengths can
|
|
be mixed:
|
|
|
|
```text
|
|
fleet send 3 1257aee5 get radio2
|
|
fleet send 3 1257aee5a754;9abc0123 get radio2
|
|
fleet send 3 1257aee5;9abc01234567 set radio2.cross auto
|
|
```
|
|
|
|
These are example public-key prefixes; substitute your nodes' actual keys.
|
|
**A prefix addresses every enrolled node whose public key starts with it.**
|
|
Use all 64 characters when you need one exact identity, especially in a large
|
|
fleet. Names and short LoRa path IDs are not accepted as target identities.
|
|
`all` must appear alone. A node that matches several entries still executes the
|
|
command only once, and the signature protects the entire list.
|
|
|
|
### Target existing regions
|
|
|
|
Fleet commands read each receiver's **existing region map**. Two selectors are
|
|
available, and they can be mixed with public-key targets:
|
|
|
|
| Selector | Which receivers execute |
|
|
| --- | --- |
|
|
| `sea` or `region:sea` | Receivers with an exact `sea` entry in their configured region list |
|
|
| `home:sea` | Receivers whose home region is `sea`, or whose home has `sea` as a parent or ancestor |
|
|
|
|
```text
|
|
fleet send 3 sea get radio2
|
|
fleet send 3 sea;pdx clock sync
|
|
fleet send 3 region:sea set radio2.cross auto
|
|
fleet send 3 home:sea get tempradio2
|
|
fleet send 3 home:sea;home:pdx clock sync
|
|
fleet send 3 sea;home:pdx;1257aee5 get radio2
|
|
```
|
|
|
|
The semicolon-separated list is an **OR**: a receiver that matches any entry executes
|
|
once, even if it matches several entries. Names match exactly and are case
|
|
sensitive; `sea` does not match `seattle`. One leading `#` is an alias, as in the
|
|
existing region commands. `$` remains significant for private region names.
|
|
Names may contain up to 30 printable ASCII bytes using region-name characters.
|
|
Use `region:` or `home:` explicitly for names with 8 or 12 characters, names
|
|
made entirely of hexadecimal characters, or reserved command words such as
|
|
`get`, `set`, `del`, `time` and `clock`. Bare `all` still means the entire fleet;
|
|
`region:all` selects a region actually named `all`.
|
|
|
|
Inspect the existing configuration with `region` and `region home`. To choose a
|
|
home from regions already defined on that receiver, use:
|
|
|
|
```text
|
|
region home sea
|
|
region save
|
|
```
|
|
|
|
List matching includes a configured region even when forwarding for it is
|
|
denied. Home matching follows the selected home's ancestry; a wildcard or unset
|
|
home matches no named home target. The default transmit scope and the incoming
|
|
packet's scope do not determine target membership. Region selection is checked
|
|
against the live map when the complete command is received.
|
|
|
|
Targets select which nodes execute. Delivery still follows the Companion's
|
|
channel scope and the intervening forwarding rules. Unknown names match no
|
|
receivers and never become a whole-fleet request.
|
|
|
|
### Target a GPS center and radius
|
|
|
|
Use `gps:latitude,longitude:radius-km` to select receivers within a circle. For
|
|
example, this selects nodes within **25 km** of the given Seattle coordinates:
|
|
|
|
```text
|
|
fleet send 3 gps:47.6062,-122.3321:25 get radio2
|
|
fleet send 3 gps:47.6062,-122.3321:25 set radio2.cross auto
|
|
fleet send 3 sea;home:pdx;gps:47.6062,-122.3321:25;1257aee5 clock sync
|
|
```
|
|
|
|
GPS, region, home and key selectors can share the same semicolon-separated list.
|
|
They use the same OR rule: any matching entry selects the node, and the command
|
|
executes once. The center, radius and complete target list are signed.
|
|
The comma is only for the coordinate pair; `;` is the only target-list separator.
|
|
|
|
Coordinates use ordinary decimal degrees, with latitude from -90 to 90 and
|
|
longitude from -180 to 180, and at most six decimal places. Radius is in
|
|
kilometers, from **0.001 to 20050**, with at most three decimal places. For
|
|
example, `0.5` means 500 meters. Scientific notation, excess decimal places,
|
|
missing fields and out-of-range values are rejected without sending. Distance
|
|
uses the great-circle calculation, including across the date line and near the
|
|
poles; a node on the radius boundary is included.
|
|
|
|
Each receiver prefers its configured `lat` and `lon`. A stationary radio can
|
|
have a fixed position even without GPS hardware; configure it through its
|
|
normal administration connection:
|
|
|
|
```text
|
|
set lat 47.6062
|
|
set lon -122.3321
|
|
get lat
|
|
get lon
|
|
```
|
|
|
|
When both configured coordinates are zero, the receiver can use an existing
|
|
valid GPS cache no older than 12 hours. Matching never wakes GPS, takes a UART
|
|
or requests telemetry. A missing, expired, invalid or default `(0,0)` position
|
|
matches no GPS selector; it can still match another entry in the list. Invalid
|
|
nonzero configured coordinates are rejected rather than falling back to GPS.
|
|
For a stationary radio that lets GPS sleep for longer than 12 hours, configure
|
|
its fixed coordinates.
|
|
|
|
Private fleet matching uses this position even when location is hidden from
|
|
public adverts. It reads the position when the complete command arrives,
|
|
including after the second fragment of a two-packet command, so a moving node
|
|
is selected using its position at completion.
|
|
|
|
On air, full keys use a 128-bit SHA-256 digest; prefixes use their original
|
|
public-key bytes. Whole-fleet commands use the compact FMC2 format with no target
|
|
records, whether the CLI target is omitted or explicitly `all`. This reduces the
|
|
header from 29 to 14 bytes. A single complete key uses the smaller fixed-target
|
|
FMC1 format. Region, home and GPS selectors use separate signed record types in FMC2.
|
|
Separate fleets use separate channel keys.
|
|
|
|
Only local commands on the publishing Companion can originate fleet sends. A
|
|
remote CLI command cannot turn that Companion into a multicast signing proxy.
|
|
An `OK` means the Companion queued the command, not that every receiver applied
|
|
it. Wait for acknowledgements and confirm the result on important nodes.
|
|
|
|
Use at most one new command per second. The publishing Companion remembers its
|
|
last queued sequence for the current boot; its clock supplies the next sequence.
|
|
Each receiving node reserves the Unix-second sequence in durable storage
|
|
**before** dispatch. That receiver never executes the same sequence twice,
|
|
including after reboot or a lost reply. Retry an uncertain command with a fresh
|
|
sequence; the operation should be chosen to be idempotent.
|
|
|
|
## Supported controls
|
|
|
|
The Companion automatically uses **one or two LoRa packets**, depending on the
|
|
signed command's size. Whole-fleet commands fit **87 ASCII bytes** in one packet
|
|
or **230 bytes** across two. Targets consume some of that allowance:
|
|
|
|
| Targets | One packet: command bytes | Two packets: command bytes |
|
|
| --- | ---: | ---: |
|
|
| Omitted or `all` | 87 | 230 |
|
|
| One 8-character public-key prefix | 82 | 225 |
|
|
| One 12-character public-key prefix | 80 | 223 |
|
|
| One complete 64-character public key | 72 | 215 |
|
|
| One three-character region or home name, such as `sea` | 82 | 225 |
|
|
| Two three-character region or home names, such as `sea;pdx` | 77 | 220 |
|
|
| One GPS center and radius | 74 | 217 |
|
|
| Two 8-character public-key prefixes | 77 | 220 |
|
|
| Two complete public keys | 53 | 196 |
|
|
|
|
Each prefix in a list uses 5 or 7 bytes; each complete key in a list uses 17.
|
|
Each region or home selector uses two bytes plus its canonical name length.
|
|
Each GPS selector uses 13 bytes: its type, latitude and longitude in millionths
|
|
of a degree, and radius in meters.
|
|
A single complete key uses the smaller fixed-target format with a 29-byte header,
|
|
so its limits come from packet space. Lists are capped at 17 entries and 86 bytes
|
|
of encoded target records; for example, at most six GPS selectors fit. Requests
|
|
exceeding their target-specific budget are rejected before signing or sending,
|
|
with no partial target list.
|
|
|
|
The existing app/console input limit also applies. The app's normal 176-byte
|
|
command frame fits **161 command bytes** after `fleet send 3 `; a correlated
|
|
request uses three more bytes, and explicit targets reduce the space further.
|
|
The shorter USB rescue console has an 80-byte input buffer. The 230-byte limit
|
|
is the fleet protocol's capacity, rather than a promise that every console can
|
|
enter that much text. Oversized or invalid lists are rejected as a whole,
|
|
without sending or truncating any targets.
|
|
|
|
### Two-packet delivery
|
|
|
|
Longer commands keep **one 64-byte signature over the entire command**, including
|
|
its channel, targets, sequence and expiry. The signed envelope is split into two
|
|
`FMP1` fragments. Each fragment has an 11-byte collection header and up to 154
|
|
envelope bytes, within the existing 165-byte group-data limit. Together they can
|
|
carry a 308-byte signed envelope. No third packet is used.
|
|
|
|
The Companion reserves both packets before sending. If either cannot be queued,
|
|
it cancels the queued part and any radio copy, reports an error, and leaves the
|
|
sequence available for a retry. Successful queueing still does not guarantee
|
|
delivery over LoRa.
|
|
|
|
A receiver accepts either arrival order and identical duplicates. It executes
|
|
and acknowledges **only after both parts arrive and the complete signature,
|
|
target, clock and replay checks pass**. Missing or altered parts cannot execute
|
|
a partial command. Two bounded assemblies are kept for up to five minutes
|
|
from their respective first parts; duplicates do not extend those deadlines.
|
|
Unverified fragment sequences cannot grant priority over another transfer. A
|
|
third transfer replaces the oldest collection. A complete, valid signed command
|
|
clears the partial collections, and durable replay checks remain authoritative.
|
|
Reboot or changing enrollment discards the partial transfers.
|
|
|
|
| Control | Fleet commands |
|
|
| --- | --- |
|
|
| Saved second profile | `get radio2`, `set radio2 <tuple|off>` |
|
|
| Temporary second profile | `get tempradio2`, `set tempradio2 <tuple|off>` |
|
|
| Scheduled primary profile | `get/set/del radioat`, `get/set/del tempradioat` |
|
|
| Scheduled second profile | `get/set/del radioat2`, `get/set/del tempradioat2` |
|
|
| UTC clock | `clock`, `clock sync`, `time <epoch-seconds>` |
|
|
| Profile diagnostics | `get radio2.status`, `get radio2.timing` |
|
|
| Crossover | `get radio2.cross`, `set radio2.cross <auto|on|off>` |
|
|
| Filter rules | `get/set/del flood.filter[.n]`, `flood.rule[.n]` |
|
|
| Path blacklist | `get/set/del flood.filter.blacklist[.n]` |
|
|
| Channel moderation | `get/set/del flood.moderation[.n]` |
|
|
| Scope policy | `get/set/del flood.channel.scope[.n]`, `flood.channel.scope.require[.n]` |
|
|
| Group-data forwarding | `get/set flood.channel.data`, `flood.channel.data.hops` |
|
|
| Flood hop limits | `get/set flood.max`, `flood.max.unscoped`, `flood.max.advert` |
|
|
|
|
Each role still checks whether it supports the requested setting. Immediate
|
|
primary radio changes, identity export, passwords, ACL edits, filesystem operations, firmware
|
|
installation, reboot, and unrestricted CLI are outside fleet capability.
|
|
|
|
## Schedule radio changes and set clocks
|
|
|
|
Use the normal schedule syntax, with UTC epoch seconds or `+N` whole minutes.
|
|
Primary schedules are supported by repeaters; a role without a primary scheduler
|
|
replies that the command is unsupported. Second-profile schedules use `rx` or
|
|
`rxtx` after the coding rate:
|
|
|
|
```text
|
|
fleet send 3 clock
|
|
fleet send 3 clock sync
|
|
fleet send 3 set radioat 910.5,500,8,5,+5
|
|
fleet send 3 set tempradioat 910.5,500,8,5,+5,+15
|
|
fleet send 3 set radioat2 910.5,500,8,5,rxtx,+5
|
|
fleet send 3 set tempradioat2 910.5,500,8,5,rxtx,+5,+15
|
|
fleet send 3 1257aee5 get tempradioat2 1
|
|
fleet send 3 1257aee5 del tempradioat2 all
|
|
```
|
|
|
|
Both relative endpoints use the receiver's clock at command acceptance. For a
|
|
coordinated fleet switch, use the same absolute UTC start/end timestamps on all
|
|
nodes. An optional final preamble value or `auto` uses the existing radio syntax.
|
|
Schedule capacity, radio-range validation, and overlap checks remain unchanged;
|
|
pending schedules still do not survive reboot.
|
|
|
|
`clock sync` advances a receiver to the publishing Companion's signed timestamp
|
|
plus one second. `time <epoch-seconds>` sets an explicit UTC timestamp. Both use
|
|
the existing forward-only clock rule. Initial setup or a clock far out of sync
|
|
still requires local or individually authenticated clock correction: fleet clock
|
|
commands retain the signature, expiry, replay, and maximum-clock-lead checks.
|
|
|
|
Radio mutations and schedule edits keep the old state until their exact acknowledgement
|
|
packets drain, with at least one successful physical transmission. Queue failure,
|
|
failed transmissions, or an acknowledgement timeout cancels the staged mutation.
|
|
Temporary-radio leases and scheduled windows retain their original monotonic end time; waiting for an
|
|
acknowledgement does not lengthen them.
|
|
|
|
Exact single-node acknowledgements use 0.5-1.5 seconds of random delay; prefix,
|
|
multiple-target, region, GPS and whole-fleet replies use 0.5-10 seconds, including
|
|
radio mutations. Queuing, airtime and forwarding can add delivery time.
|
|
This reduces simultaneous responses but is not a reliable
|
|
delivery protocol for an arbitrarily large fleet. Use individual requests for
|
|
confirmation and divide very large fleets into separately keyed channels.
|
|
|
|
## Filtering and recovery
|
|
|
|
Local fleet reception happens before a node's forwarding decision, so that node
|
|
can still process an authenticated command when a filter or `repeat off` blocks
|
|
forwarding. **Intermediate nodes can still block delivery to downstream nodes.**
|
|
Management traffic has no automatic exemption from forwarding filters. Preserve
|
|
an appropriate group-data/channel forwarding rule and an existing direct-admin
|
|
recovery path before changing network-wide filters. Primary-radio schedules can
|
|
move nodes off the normal network; use a temporary window or a planned recovery
|
|
path for experiments. A forwarding policy can also partition the fleet.
|
|
|
|
The receiver is allocated lazily when fleet settings are present or queried, with
|
|
one bounded mailbox and two 308-byte fragment assemblies, plus bounded metadata.
|
|
Cryptographic work and storage writes occur after the receive stack unwinds.
|
|
Signature-validation attempts are bounded after private-channel MAC verification,
|
|
complete framing, replay, target, time and command checks. Keyless matching-hash
|
|
junk, already-used requests, other nodes' requests and incomplete fragments do not
|
|
spend that signature budget. A corrupt or unreadable fleet store denies commands
|
|
rather than discarding its replay state.
|
|
|
|
## Ideas for packet-abuse containment
|
|
|
|
A packet-count chart can show a burst, but cannot establish intent, an authenticated
|
|
source, or the airtime consumed. A large Trace spike deserves investigation;
|
|
MeshCore already rejects flood-form Trace and Control packets. Valid direct Trace
|
|
requests need different treatment from flood forwarding rules.
|
|
|
|
Useful next additions, rather than features already implemented here:
|
|
|
|
- **Temporary containment with automatic rollback:** deploy a drop/rate policy
|
|
for a bounded period and restore the previous policy unless confirmed.
|
|
- **Direct-request rate limits:** separately bound Trace, anonymous requests,
|
|
discovery, login, and expensive response generation without breaking legitimate
|
|
management, ACKs, or ongoing OTA exchanges.
|
|
- **Policy preview and revisions:** validate complete rule bundles, detect board
|
|
limitations, apply atomically, and report the active revision before rollout.
|
|
- **Better evidence:** report per-type receive/drop counts, airtime, queue pressure,
|
|
invalid signatures, duplicate rate, and radio recoveries. Truncated path IDs
|
|
remain observations, not authenticated attacker identities.
|
|
- **Authenticated inventory:** stagger signed status replies with firmware version,
|
|
uptime/reset reason, battery, memory/storage headroom, and radio errors.
|
|
- **Maintenance windows:** coordinated secondary-profile experiments and OTA
|
|
readiness checks, keeping existing firmware-signing policy intact.
|
|
- **Per-controller roles and revocation:** multiple public verification keys with
|
|
separate filter-only/radio permissions and independent replay state.
|
|
|
|
Start with observation, then a narrow temporary rule, then verify neighboring
|
|
nodes still communicate. Software filtering can reduce forwarding and response
|
|
amplification; it cannot recover airtime already occupied by an RF transmitter.
|
|
|
|
See [flood filtering](flood_filtering.md), [radio profiles](radio_profiles.md), and
|
|
[ACL administration](cli_commands.md#acl).
|