Record completed LoRa OTA and source restoration results
This commit is contained in:
1 parent
e285214349
commit
d3661f66f0
1 file changed
+199
-25
@@ -4,25 +4,30 @@
|
||||
|
||||
# OTA hardware qualification, 20-21 October 2026 UTC
|
||||
|
||||
Status: IN PROGRESS. Results below describe completed measurements only.
|
||||
Status: COMPLETED for the listed OTA cases. Earlier failures and unresolved
|
||||
observations remain recorded below; this is not a claim that every board or
|
||||
transport has passed all release qualification.
|
||||
The operator authorized erasing these test nodes without backups. MeshPocket
|
||||
remains excluded. No bootloader or SoftDevice payload is part of this campaign.
|
||||
remains excluded. No nRF52 bootloader or SoftDevice payload is part of this
|
||||
campaign. Temporary ESP32 bridge role changes use the board's merged image,
|
||||
including its ESP32 bootloader and partitions, under the authorized erase.
|
||||
|
||||
Completed hardware OTA results use artifacts pinned to
|
||||
The initial hardware OTA results use artifacts pinned to
|
||||
`8b5660b00b88d8abaae094859bc1232f7ce886fd`.
|
||||
That commit changes documentation only relative to firmware source
|
||||
`0b207a1ec3a32eb0a8ad6b8643db211de4921cd4`. Separate OTA-check version strings
|
||||
produce distinct image hashes for controlled update tests. The common numeric
|
||||
version is `0x01120001`; same-version updates are explicitly authorized for
|
||||
these tests. They are not claimed as numeric version upgrades.
|
||||
Later source, CPU-sleep and one-hop results identify their own commits below.
|
||||
|
||||
| Device and role | Transport | Result |
|
||||
| --- | --- | --- |
|
||||
| Heltec V4.3 Full Companion | Production browser WiFi OTA | PASS: one upload installed the exact intended image; postboot identity, version, core, radio and original flags verified |
|
||||
| RAK4631 unified repeater | Bluetooth application DFU; QSPI full LoRa OTA | Transfers and eventual postchecks PASS; initial USB version response remains under investigation |
|
||||
| RAK4631 unified repeater | Bluetooth application DFU; QSPI full LoRa OTA; two-frequency bridged one-hop LoRa delta | Transfers and eventual postchecks PASS, including one-hop installation; initial USB version response remains under investigation |
|
||||
| Seeed XIAO nRF52 repeater | Bluetooth application DFU; external QSPI full LoRa OTA | Transfers and eventual postchecks PASS; initial USB version response remains under investigation |
|
||||
| RAK3401 unified repeater and gateway | Bluetooth application DFU; internal LoRa delta; gateway restoration | PASS: Bluetooth update and repeater service, internal LoRa delta apply, normal-radio restoration and gateway service handback |
|
||||
| MeshTower SD repeater | Signed SD-backed LoRa delta and full update | Delta transfer, signature verification and postboot checks PASS; full update pending |
|
||||
| MeshTower SD repeater | Signed SD-backed LoRa delta and full update | PASS: signed SD delta and full-image transfer, installation and LoRa postboot checks |
|
||||
|
||||
## V4 WiFi OTA
|
||||
|
||||
@@ -157,6 +162,59 @@ is not treated as missing command support or a first-attempt pass. The
|
||||
successful image, role, settings and core checks above describe the later
|
||||
independent verification.
|
||||
|
||||
## Two-frequency ESP-NOW bridged one-hop LoRa update
|
||||
|
||||
The physical one-hop test PASSED. The V4 source ran `2bd4cb4d1` and the
|
||||
RAK4631 received an application-only delta to `bd4524318`. Its setup,
|
||||
discovery, download, progress checks, install command, postboot proof and
|
||||
settings restoration all used LoRa. The receiving RAK's USB and Bluetooth
|
||||
interfaces were never opened for this case.
|
||||
|
||||
The measured path was V4 at 909.5 MHz to a V3 bridge, ESP-NOW to a XIAO
|
||||
ESP32 bridge, then LoRa at 908.5 MHz to the RAK4631. Both LoRa segments used
|
||||
500 kHz bandwidth, SF7 and CR5. The two bridges copied received packets before
|
||||
appending their own hop, with one-hop limits. Disabling the far bridge's
|
||||
ESP-NOW link made the target's full-key query time out; re-enabling and
|
||||
verifying the same configuration restored that query. This counterfactual
|
||||
confirms that the two-frequency bridge was required.
|
||||
|
||||
The ordinary file-data session populated the source catalog and its automatic
|
||||
discovery burst. No separate announcement command or control channel was
|
||||
used. The first asynchronous catalog query requested discovery; the next
|
||||
query found the intended manifest. Download completed **62/62 blocks**.
|
||||
The one install command received an on-air acknowledgement verifying the
|
||||
unsigned QSPI delta. Subsequent normal-channel LoRa reads proved exact
|
||||
installed body `D2073EC88FF4E459`, the full `dev-bd452431` version and zero
|
||||
core errors. Original target policies, receive power saving, output path and
|
||||
the source's saved contact path were restored. The package SHA256 is
|
||||
`f0a7319570394e23f06c9a901838be3cbaf5ae520d2743607828977e5fa3a0e9`.
|
||||
|
||||
A passive source USB capture wrote zero bytes to the radio and recorded
|
||||
47 matching update requests with one hop through the measured near bridge.
|
||||
This is source-side packet evidence, not an independent RF capture of the
|
||||
whole chain. The separate frequencies, bridge counterfactual and on-air
|
||||
install/postboot proofs establish the complete case. The integrated private
|
||||
receipt SHA256 is
|
||||
`c6d6b2e2d3aefec128303b0ef5e2d8a0be52c38f03bc17ecf3032f4118489cc6`.
|
||||
|
||||
Preparation failures remain recorded separately: a historical channel-busy
|
||||
flag prompted one documented LoRa restart, and several bridge receive-power
|
||||
configuration attempts returned a generic busy/unsupported rejection before
|
||||
a fresh configuration check and acknowledged attempt succeeded. No cause
|
||||
for those rejections or the historical busy flag is claimed fixed.
|
||||
|
||||
Both temporary ESP32 bridges were returned to Full Companion images built
|
||||
from `ebb59d482`. Exact runtime bodies were `E297CF6E342C8332` (V3) and
|
||||
`4EEB12667E5E554B` (XIAO ESP32), with matching Binary/CLI identity and zero
|
||||
core errors. The V3's first postflash Binary handshake timed out; a later
|
||||
independent read passed without a reset or another flash. Its role reply
|
||||
required a narrow correction to the bench parser's nested prompt handling.
|
||||
The XIAO's positive-readiness check also rejected a valid prompt-prefixed
|
||||
acknowledgement before sending any Binary request. After correcting that
|
||||
check, its first Binary handshake and complete CLI checks passed without
|
||||
another flash, reset or settings change. Original failed workflow receipts
|
||||
remain retained. These bench corrections are not claimed as firmware fixes.
|
||||
|
||||
## Findings and fixes
|
||||
|
||||
An authenticated nRF52 application handoff does not preserve a custom
|
||||
@@ -222,8 +280,9 @@ retries lost remote clock reads within the existing bounded operation budget.
|
||||
Validation passed 671 tests, including nine new clock-read regressions; one
|
||||
optional PyPI integration test was skipped. Canonical Full and Reduced
|
||||
RAK4631 and RAK3401 pairs, plus Full V3 and XIAO S3 WIO images, built
|
||||
successfully from this commit. These newly built images have not yet been
|
||||
flashed; their build success does not replace the earlier hardware results.
|
||||
successfully from this commit. The ESP32 images were subsequently installed
|
||||
as the temporary one-hop bridges. Build success alone does not qualify an
|
||||
update or replace the earlier hardware results.
|
||||
|
||||
Repeater Bluetooth provides an update service; Companion terminal/sync
|
||||
functionality is not claimed for repeater builds. The RAK4631 reports a 2 MiB
|
||||
@@ -254,8 +313,8 @@ During RAK4631 discovery, source receive power saving was on and discovery
|
||||
timed out. After disabling that setting under a restoration intent, a fresh
|
||||
advert was received. This sequence records an observation; it does not
|
||||
establish a receive-power-saving firmware defect. Source receive power
|
||||
saving remains off under the owned restoration intent while the remaining
|
||||
LoRa cases run; final source restoration is pending.
|
||||
saving stayed off under the owned restoration intent while the remaining
|
||||
LoRa cases ran. Exact final source restoration is recorded below.
|
||||
|
||||
## MeshTower signed SD updates and remaining failures
|
||||
|
||||
@@ -290,8 +349,8 @@ clock-read retry fix above addresses the host's handling of lost reads.
|
||||
A second full-image attempt at SF7 and 500 kHz bandwidth passed the full-key
|
||||
and status proofs, but all three `ota self` proof attempts failed. Safe
|
||||
temporary-lease expiry and return to the normal radio were verified. No
|
||||
firmware was transferred in either full-image attempt; full apply and
|
||||
postchecks remain pending.
|
||||
firmware was transferred in either of those full-image attempts; full apply
|
||||
and postchecks were still pending at that point.
|
||||
|
||||
A local USB snapshot afterward confirmed body `F1909AD4C46915FD`, 1133 seconds
|
||||
uptime, core error bitmask `0x02` and an empty queue. Radio diagnostics
|
||||
@@ -318,21 +377,136 @@ fresh identity/settings check followed by one new, acknowledged restoration
|
||||
attempt proved the original configuration. No particular runtime guard is
|
||||
identified as the cause of that rejection.
|
||||
|
||||
A later signed-full retry used the ordinary production file-data session and
|
||||
source catalog readiness, with every receiver command on LoRa. Its first
|
||||
attempt stopped before new data because an old failed receive session
|
||||
required an explicit replacement choice. The one-minute temporary-channel
|
||||
cleanup command lost its acknowledgement. The updater then proved identity
|
||||
on the temporary channel and restored the source, but did not prove that the
|
||||
receiver had returned to its normal channel. The next attempt could not
|
||||
reach the receiver on the normal channel. These are failed workflow records.
|
||||
|
||||
A bounded, LoRa-only recovery found the exact Tower still running body
|
||||
`F1909AD4C46915FD` on the temporary channel. One command restored its normal
|
||||
radio; its acknowledgement was lost and the command was not replayed.
|
||||
Subsequent authenticated normal-channel reads proved the full identity,
|
||||
unchanged body, inactive temporary mode, normal radio and 0 dBm TX. No target
|
||||
USB, Bluetooth, reset or flash operation was used. A historical core bitmask
|
||||
`0x06` was retained: channel-busy timeout and radio outside receive for over
|
||||
eight seconds. It is a bitmask, not a count of six errors. The recovery receipt
|
||||
SHA256 is
|
||||
`bc44880091e0024966fb6b2d9fd77b21556a9899e9c9995cf772dc14ae1bd57e`.
|
||||
|
||||
After that recovery, a fresh ordinary updater attempt passed its temporary
|
||||
channel rehearsal and natural return, explicitly replaced only the old failed
|
||||
receive session, and downloaded the signed full image **279/279 blocks in
|
||||
831 seconds**. The install acknowledgement verified a trusted signer and a
|
||||
full image on SD. Normal-channel LoRa postboot reads proved body
|
||||
`718365A6E08823C7`, bootloader apply OK and the full `dev-ebb59d48` version.
|
||||
The first matching body response arrived during the 60-second postboot probe.
|
||||
A lost optional statistics reply required a bounded getter retry. The
|
||||
570682-byte package SHA256 is
|
||||
`4f92525f7500ad74b7f1895a62c4626a02a049b1a3978906ae8eea8617393bde`.
|
||||
The production runner exited successfully; its private result SHA256 is
|
||||
`de1bb4c4fa7af7fe627c00d96c94d1f7244141556c29271754379ad8f7fd7ca9`.
|
||||
All receiver setup, download, install and postboot commands used LoRa; its USB
|
||||
and Bluetooth interfaces were never opened for this attempt. The source was
|
||||
fed through its ordinary file-data session, with no extra announcement or
|
||||
separate receiver control channel.
|
||||
|
||||
An independent, authenticated LoRa postcheck then verified the new full body
|
||||
and version, repeater role, normal radio/TX, inactive temporary mode, zero
|
||||
core error bitmask and an empty transmit queue. It removed only the owned
|
||||
fourth test signing key and verified that the original three key prefixes
|
||||
remained in their original order. The CLI exposes prefixes, so this is not
|
||||
an independent full-byte readback of those three keys. The postcheck repeated
|
||||
the identity, radio and core checks after removal and closed its controller.
|
||||
Lost getter/status replies used bounded read-only retries; all failed attempts
|
||||
remain recorded. Its private summary SHA256 is
|
||||
`02964ab9a2f511f96055ef6c99c59d32bb091d21dbf5c61dbe9de5ac00555659`.
|
||||
|
||||
The passive source-only capture recorded 405398 bytes and wrote zero USB
|
||||
bytes. Its SHA256 is
|
||||
`6558971dea7e9cdc221e3e196d34f5ae00427300932c766e3fb7798d3332d194`.
|
||||
The capture was stopped and its serial port closed before the source's
|
||||
original logging-off setting was restored and verified. This capture is
|
||||
source-side evidence; the receiver's installed-image and runtime proofs
|
||||
above came from LoRa.
|
||||
|
||||
The host cleanup defect is separate from the radio transfer: temporary-channel
|
||||
identity alone cannot establish that a lost relative-lease command executed.
|
||||
Cleanup must retain its recovery obligation until a full-key proof and
|
||||
inactive temporary mode are observed on the normal channel. The same
|
||||
requirement applies independently to controlled relay participants.
|
||||
|
||||
Commit [c5c67f8f7](https://github.com/mikecarper/MeshCore/commit/c5c67f8f7)
|
||||
fixes that host ownership defect. It persists a private recovery record for
|
||||
each full-key-bound target and controlled relay before arming, records a
|
||||
one-shot shortening attempt before sending, and requires each participant's
|
||||
full identity plus inactive temporary mode on the normal channel before
|
||||
retiring its obligation. A shared Full Companion source must also report its
|
||||
actual temporary mode inactive; its saved Binary radio settings alone are
|
||||
insufficient. Lost relative-lease acknowledgements are not replayed. Unresolved
|
||||
records survive interruption without passwords, and manual read-only recovery
|
||||
is documented in the [LoRa OTA automation guide](../lora_ota_automation.md).
|
||||
The regression cases cover delivered and undelivered lost acknowledgements,
|
||||
independent relay failures, mismatched keys, active temporary profiles,
|
||||
interruptions, existing recovery records, resolved-password redaction and the
|
||||
explicit no-install staging path. These are host changes; accepted unchanged
|
||||
firmware was not flashed again to validate them.
|
||||
|
||||
## Final source restoration and handback
|
||||
|
||||
After all transfers and the Tower signing-key cleanup, two fresh USB cleanup
|
||||
attempts received an explicit `Error: radio busy; retry` while restoring the
|
||||
source's original RXPS setting. Both attempts stopped and retained their raw
|
||||
failed records. This firmware rejects that command before updating or saving
|
||||
RXPS preferences; a missing or ambiguous reply would not permit the same
|
||||
continuation. Fresh read-only diagnostics showed normal receive operation,
|
||||
zero core errors, an empty queue and RXPS still off. They did not identify
|
||||
which radio reconfiguration guard rejected those particular commands.
|
||||
|
||||
A bounded continuation used the production updater's varying retry cadence
|
||||
for this exact explicit busy response, opening a fresh console after each
|
||||
rejection. Its second attempt received the exact positive acknowledgement
|
||||
and exact original configuration readback: on, level 8, preamble 16,
|
||||
39657 microseconds receive and 15975 microseconds sleep. It did not retry a
|
||||
lost or malformed acknowledgement, reset the radio or change firmware.
|
||||
The continuation receipt SHA256 is
|
||||
`4253c91dbf2eca91e7b2716367e27e95a7dd1b952008b9ef6bb036fd673d3c62`.
|
||||
|
||||
A separate final cleanup skipped the already restored RXPS setting, restored
|
||||
all five original saved flags, and verified normal radio/TX, idle detached OTA
|
||||
file service, original hop settings and strict USB Binary handback. It disabled
|
||||
the test WiFi runtime before restoring MQTT. The completed private summary
|
||||
SHA256 is
|
||||
`1c07fc8301f883eb3a33201924ca77ce4d2d02573b9a27a785f84547a06a8165`.
|
||||
The hash-bound completed receipt retired the host's RXPS restoration intent.
|
||||
Owned test tunnels, target leases and the cold-idle hold were closed; the
|
||||
root hardware session had no remaining FLOCK descriptors before it exited.
|
||||
The already restored RAK3401 gateway service was left running.
|
||||
|
||||
## Limits and completion
|
||||
|
||||
The MeshTower signed full-image LoRa case and final source cleanup remain
|
||||
pending. The live cold-idle RAK preflight check is complete. Newly requested
|
||||
two-frequency ESP-NOW bridged one-hop OTA testing remains pending. Latest
|
||||
RAK3401 CPU sleep, post-idle LoRa and gateway restoration passed. Receiver
|
||||
update setup, progress, installation
|
||||
and post-install verification in the new radio campaign must use LoRa;
|
||||
receiver USB commands cannot substitute for failed radio communication.
|
||||
The RAK3401's current-image pause snapshot recorded historical channel-busy
|
||||
bitmask `0x02`; it was retained without clearing statistics or claiming a
|
||||
CPU-sleep result from that historical snapshot. The first-command USB bootstrap observation,
|
||||
Tower SF5 preflight failure, later full-image self-proof failures and separate
|
||||
USB open failure remain unresolved. The Tower signed delta, RAK4631 and XIAO
|
||||
nRF52 full QSPI LoRa updates and their later postchecks are complete; the
|
||||
RAK3401 gateway handback is complete for the earlier pinned image.
|
||||
The listed physical OTA cases and their restoration are complete: V4 WiFi,
|
||||
RAK3401 internal delta, RAK4631 and XIAO nRF52 external QSPI full images,
|
||||
Tower signed SD delta/full, and the two-frequency ESP-NOW bridged one-hop
|
||||
update. Latest RAK3401 CPU sleep, post-idle LoRa and gateway handback passed.
|
||||
Each case records its exact firmware commit and body; these results do not
|
||||
qualify all 13 radios against every later commit.
|
||||
|
||||
Receiver setup, progress, installation and post-install verification in the
|
||||
new radio campaign used LoRa. Receiver USB commands did not substitute for
|
||||
failed radio communication. Earlier local-control diagnostics remain
|
||||
explicitly identified. The RAK3401's pre-fix historical channel-busy bitmask
|
||||
`0x02` was retained without clearing statistics or claiming a sleep result
|
||||
from that snapshot.
|
||||
|
||||
The first-command USB bootstrap observation, Tower SF5 preflight failure,
|
||||
earlier full-image self-proof failures, separate USB open failure and
|
||||
historical radio-busy flags remain unexplained. Later full-image transfer and
|
||||
postchecks passed under the stated conditions; this does not establish their
|
||||
underlying causes. Source restoration's explicit busy replies were handled
|
||||
with bounded retries; no radio guard defect is claimed fixed by that action.
|
||||
Private raw responses, keys, PINs and device identities stay outside the repo.
|
||||
Earlier accepted Companion tests are not repeated without a relevant change.
|
||||
Reference in new issue
Block a user