Commit Graph
116 Commits
Author SHA1 Message Date
mikecarper 4b3e419eb0 Reduce shared RAM usage without lowering firmware capacities
Specialize repeater ACL session state, compact recent and flood records, stream Merkle proofs, and release cold OTA buffers safely. Preserve saved formats, stack sizes, table capacities, and OTA source limits. Account for lazy buffers when active and add compatibility, ARM alignment, allocation failure, and queued-response tests.

Validated six representative board/role builds, 1736 native test cases, sanitizer fixtures, and strict documentation. Document the preexisting Wio-E5 Mini flash overflow separately.
2026-10-08 14:41:16 -07:00
mikecarper 76ddc06e56 Add opt-in combined RAK4631 Ethernet test image
Keep Full sensors, UART2, adaptive OTA storage and the 8 KiB loop stack. Arbitrate shared SPI and OTA workspace ownership, move network waits into a worker, and use bounded queued TCP operations. Add reproducible test configs, RAM checks and regression coverage. Validated the Full build, UF2 layout and 29 sanitizer runtime scenarios.
2026-10-08 13:53:31 -07:00
mikecarper 0e0faa33a4 Unify Tower repeater builds on SD-primary OTA storage
Keep the deployed SD target and layout, select qualified internal delta storage when no card is usable, and retain exact SD bootloader-update gates. Add storage, capability, transfer, recipe, and release-selection regression coverage.

Also preserve mandatory runtime packet logging in canonical XIAO images when verbose build debugging is disabled, fixing the four stopped release-matrix qualification failures.
2026-10-05 16:38:30 -07:00
mikecarper a0de63ab5e Avoid repeated missing SPIFFS scans during ESP32 startup 2026-10-05 14:12:44 -07:00
mikecarper fed810ff50 Cache verified nRF52 metadata for OTA status displays 2026-10-04 17:34:16 -07:00
mikecarper df5d009d3c Preserve USB mOTA ownership across deadlines and logging handoffs 2026-10-03 19:01:22 -07:00
mikecarper 6febd15ded Harden USB logging and persist watchdog events in management reports 2026-10-03 15:29:19 -07:00
mikecarper 5833ac2f1f Add compressed self-serve OTA and resilient native USB sleep
Use a bounded uzlib-derived block encoder with the existing strict tinf decoder, qualify infrastructure roles, budget temporary RAM, and cache running-image metadata. Add self-serve CLI and policy, USB host-loss sleep grace, hardware test harnesses, regression tests, and CI coverage.
2026-10-03 01:14:23 -07:00
mikecarper 29de7820e5 Integrate upstream fixes and paired nRF52 sensor OTA release profiles 2026-10-02 17:52:12 -07:00
mikecarper 8a69e73226 Reduce nRF52 infrastructure flash usage without trimming sensors
Keep Companion and Terminal Chat profiles unchanged; retain full-sensor and reduced-sensor OTA builds. Add compact crypto and OTA names, qualified display asset compression, font license packaging, and scope regression tests.
2026-10-02 15:52:23 -07:00
mikecarper 84429ae520 Use internal bootloader staging regardless of RAK app storage 2026-09-29 03:27:10 -07:00
mikecarper c3f37e129a Support compatible RAK bootloader updates with adaptive app storage 2026-09-29 03:08:53 -07:00
mikecarper ba2187db2c Revert adaptive RAK app-owned bootloader recovery after ACL test 2026-09-27 15:07:29 -07:00
mikecarper ec86d96039 Use application capabilities for adaptive RAK recovery status 2026-09-27 13:21:40 -07:00
mikecarper abaedd3894 Add application-owned RAK bootloader recovery over LoRa 2026-09-26 23:47:44 -07:00
mikecarper 9127f8fe25 Detect header W25Q16 on unified RAK firmware 2026-09-25 14:20:29 -07:00
mikecarper cecddf4bae Connect GPIO MISO during RAK NOR identification 2026-09-25 13:14:30 -07:00
mikecarper 284a67b9ec Drain companion queue on explicit OTA entry 2026-09-25 12:06:18 -07:00
mikecarper fca10471c3 Unify RAK repeater storage firmware and bootloader handoff 2026-09-25 10:35:04 -07:00
mikecarper 307e7bec02 Complete ESP32 partition expander and adaptive LoRa OTA pacing 2026-09-24 17:13:22 -07:00
mikecarper c0ef07a968 Add LoRa partition migration recipes for ESP32 infrastructure boards 2026-09-24 10:40:32 -07:00
mikecarper 7e1f87dfbc Fix nRF52 flash completion and preserve recoverable ExtraFS 2026-09-22 09:29:05 -07:00
mikecarper 90d8a7f3c4 fix(ota): recover corrupt companion settings stores 2026-09-21 18:35:57 -07:00
mikecarper 5fd9d039f5 Add one-way OTA target migration aliases 2026-09-19 08:30:52 -07:00
mikecarper ebee99af81 Consolidate companion components 2026-09-17 13:41:39 -07:00
mikecarper e2a1828987 Fix storage recovery and deferred runtime state
Recover usable ESP32 preferences and channels before permitting replacement; keep identity and ACL persistence transactional and complete.

Preserve MQTT commit boundaries, restore OTA identity and policy after deferred allocation, retain radio gain retries, and reject invalid flood limits. Add production-path fault regressions and CI coverage.
2026-09-15 16:46:17 -07:00
mikecarper 86defc840c Harden storage transactions and deferred radio settings 2026-09-15 16:01:46 -07:00
mikecarper 64ca313781 Improve Companion, repeater, and OTA reliability 2026-09-15 15:15:45 -07:00
mikecarper a88ced23b6 Fix OTA resume, CLI identity, radio persistence, and network sessions
Distinguish OTA read failures from missing blocks and select unique raw-flash checkpoints by MID and target. Keep deferred CLI requests and replies bound to immutable command identity and full authenticated keys.

Gate radio mutations on their exact reply delivery, recover coding-rate restoration, and roll back failed Companion preference writes. Preserve accepted low duty cycles across reboot.

Retain partial network writes, cancel displaced client operations, and reject malformed or stalled framed input. Add regression and CI coverage, and size-optimize the RAK4631 Ethernet build without dropping features.
2026-09-15 14:32:40 -07:00
mikecarper e4e0de8a79 Fix OTA resume and radio state recovery
Preserve resumed nRF52 flash pages and keep filtered OTA catalogs incomplete until full pages arrive. Bound stuck radio preparation, report companion setting failures, retain safe rollback, and codify conservative advert penalties with regression tests.
2026-09-15 12:17:54 -07:00
mikecarper 62ce8d3320 Fix OTA recovery, radio2 replies, and preference persistence
Keep each deferred OTA response tied to the request's radio profile and generation. Separate requests from different radio sessions, discard stale queued responses, and retain the existing companion and infrastructure reply policies.

Reject staged captures with invalid file framing before adopting their state. Recover fixed internal-flash trailer bytes after interrupted checkpoints and handle small ESP32 delta trailers that share metadata sectors. Fail closed when install preflight cannot read a valid header, so hardware and automatic-update policy checks cannot be skipped.

Replace Companion preferences through verified file transactions and quarantine writes after incomplete loads on all platforms. Initialize serializer runtime fields before preferences snapshots copy them.

Add production-code fault regressions to CI for deferred routing, store recovery, install preflight, and preferences persistence. Validation: 1,510 native tests, 60 host checks, and six firmware targets across ESP32, nRF52, STM32, and RP2040.
2026-09-15 10:45:04 -07:00
mikecarper 7c443e9f71 Harden radio recovery, OTA reconnects, and companion settings
Reject radio changes when receive mode cannot resume and restore the previous physical profile and power policy. Preserve selected folder OTA captures across disconnects and recoverable host I/O failures, while keeping corruption failures distinct. Validate OTA staging input before arithmetic or buffer writes and revoke published data before edits.

Commit companion channels through verified file replacement on ESP32, RP2040, and STM32, and quarantine incomplete startup reads. Recheck the last management connection during delayed wireless changes and prevent conflicting Bluetooth shutdowns. Detect complete OTA implementation sources rather than the shared inflater when configuring release builds.

Add fault-injection regressions and CI coverage for these paths, and document folder capture reconnect behavior.

Validation: 1,510 native tests; 55 host integration checks; build profile regressions; ESP32, nRF52, STM32, and RP2040 firmware builds.
2026-09-15 10:02:23 -07:00
mikecarper 6f99250388 Preserve OTA serving state across rejected images and source failures
Validate primary images before publishing their views, invalidate the shared source cache before reads that can partially overwrite it, and revoke self-serving references before replacing caller-owned buffers. Keep the CLI serving flag aligned with a surviving primary image after a rejected replacement.

Regression tests reproduce rejected replacements, partial source reads, and self-refresh allocation/read failures. Verify queued manifests, recovered payloads, Merkle proofs, and sanitizer-checked buffer lifetimes.
2026-09-15 09:27:37 -07:00
mikecarper dde87f1d15 Preserve OTA block progress and channel checks during radio recovery
Drain received packets in single-profile radio fault backoff as well as
dual-profile mode. Share the normal channel-clear and bounded busy policy
with driver retries so a failed send cannot retry blindly or bypass the
existing single-retry limit.

Retain an unadmitted OTA request flight and count v2 fallback evidence only
after a sparse retry enters the transmit queue. Local queue congestion no
longer disables v2 before a request was sent or discards received fragments.

Include primary airtime when temporary RX-only radio2 crosses requests onto
that channel, keeping response allowances conservative on slower profiles.

Add six native regressions, including five failing-before cases, and document
the queue-admission and crossover timing behavior.

Validation: 1,502 native tests, 29 integration checks, and five firmware
builds passed across ESP32-S3, nRF52, and STM32.
2026-09-15 07:54:16 -07:00
mikecarper 5d69aa461f Fix OTA retry accounting and radio delivery races
Keep catalog, manifest and leaf requests pending when the local transmit
queue rejects them instead of spending their network retry allowance.

Keep the packet selected before CAD so work becoming due during sensing
cannot bypass its own channel or pacing checks. Retire cancelled radio
retries before waiting for BUSY hardware or their retry deadline.

Wait for active transmission completion before applying firmware. Stop
admitting new transmissions after the apply drain deadline so busy queues
cannot postpone reboot indefinitely, and keep those deadlines armed across
millis rollover. Retain the existing persistence-flush backoff.

Add failing-before/fixed-after regression coverage for all four issues,
including the production OTA apply and TX guards in CI.

Validation: 1,496 native tests, 42 Python integration tests, and five
firmware builds passed across ESP32-S3, nRF52, and STM32 targets.
2026-09-15 06:45:28 -07:00
mikecarper b2f2b5bf02 Add persistent LoRa OTA speed control and harden timing
Add get/set ota.speed and OTA CLI aliases for factors from 0.05 to 3,
defaulting to the existing pace. Persist the setting independently of the
OTA workspace and apply it to OTA packet pacing, relays, discovery and
adverts while retaining the configured airtime budget.

Preserve physical proof/retry allowances, account for both participating
radio profiles, keep optional deadlines armed across millis rollover, and
exclude intentional pacing from observer stale-packet age. Keep local
verification at its normal rate and retain a valid saved speed if backup
recovery cannot finish. Add documentation and CI regression coverage.

Validation: 1,491 native tests and 26 CLI/integration tests passed. Built
ESP32 full repeater, ESP32 full Companion, nRF52 full Companion and STM32
USB Companion successfully, including their memory/size checks.
2026-09-15 06:16:29 -07:00
mikecarper a07f39c6dd Add per-profile TX routing and bridge filtering
Persist Companion TX preferences per contact and channel. Default repeater,
room, and sensor replies to both active TX profiles, with a reply-only force
option for an RX-only secondary profile. Track both reply copies before a
temporary-radio handoff and account for OTA copies under queue backpressure.

Add bridge/crossover filter modes and the third built-in wardriving filter,
plus compact filter CLI syntax, documentation, and CI coverage.

Fix quoted target names being interpreted as slot/key selectors, partial
recovery of invalid saved reply settings, filter suspension during tempradio2,
and unnecessary packet allocation while OTA traffic is throttled.

Validation: 1,477 native tests, 52 Python integration tests, 29 filter UI tests,
and five firmware builds covering ESP32, nRF52, and STM32 passed.
2026-09-15 01:58:43 -07:00
mikecarper 1a074116e8 Merge PR #7 with reviewed ExpressLRS power and memory fixes
Integrate ExpressLRS TX module support and its stacked ESP32 heap changes.
Use the approved Linkflow calibration (17-30 dBm), preserve the PA drive
and output path after radio recovery, and keep LoRa OTA enabled.

Preserve stored ACLs and filters on allocation failure, release owned
client/filter buffers, service heap OTA contexts on Companions, release
self-serving workspaces after TempRadio, and reset staged-resume state
when a context is released. Keep manual staging and active operations
alive. Account for all moved allocations in the runtime RAM gate.

Validation: 1,393 native cases; radio-power, heap-context, ACL persistence,
shared-queue transfer, display/inbox, radio receive, and memory regressions.
Firmware builds passed for Linkflow, Heltec V2 Companion, T-Beam MQTT
repeater, Heltec V4 R8 MQTT repeater, RAK4631 repeater, and Indicator Full.
Physical verification awaits access to the currently offline lab Pi.
2026-09-10 22:06:16 -07:00
MUSTARDTIGERFPV f0c4c6af00 Enable the on-demand mOTA context for every classic ESP32 build
Move OTA_HEAP_CONTEXT from two hand-edited envs to a pre-script gated on
build.mcu == "esp32", so every classic ESP32 image gets it and none can drift.
S2/S3/C-series and nRF52 keep the .bss singleton: they have no equivalent
static-DRAM ceiling, so there a guaranteed-present workspace is the better
trade for what is ultimately a recovery path. It has to be a build flag rather
than a header default, because OTA_HEAP_CONTEXT must hold the same value in
every translation unit that sees OtaContext.h - OtaContext.cpp included - and a
header test on CONFIG_IDF_TARGET_ESP32 would depend on include order relative
to sdkconfig.h. The script defers to OTA_SHARED_COMPANION_QUEUE where a Full
Companion recipe already owns the storage.

Fix a gap this exposes in the roles that now use it. Only CLI entry points
acquire the context, so once it was released, a repeater with the temporary
radio profile up would no longer serve or announce its own firmware - LoRa OTA
would look enabled and silently do nothing. ota_service_temp_radio_context()
holds the workspace for exactly the temp-radio window and hands it back
outside it, which is where the saving was coming from anyway. Wired into the
repeater, room server and sensor loops; the Companion keeps its own
acquire-on-host-demand policy, since its context is borrowed from the offline
message queue and must not be taken speculatively.

Static DRAM, classic ESP32 (bytes occupied of a 124,580 region):
  Heltec_v2_repeater                   108,332 -> 70,732
  Heltec_v2_room_server                 83,716 -> 68,300
  Heltec_v2_companion_radio_ble        123,236 -> 107,772  (was failing on main)
  Tbeam_SX1262_repeater                 86,932 -> 71,468
  Tbeam_SX1262_repeater_observer_mqtt  121,132 -> 83,532   (was failing)
2026-09-09 01:24:00 -07:00
MUSTARDTIGERFPV f6bc7a194a Free OtaContext-owned buffers when the context is torn down
OtaContext owns serve_buf, serve_self_leaves and serve_self_proof as raw
malloc'd pointers, and nothing freed them on teardown: self-serve only frees
them when it re-allocates, and reset_session() does not touch them. That was
harmless while the context was a permanent .bss singleton, but both dynamic
storage modes destroy it between operations - OTA_HEAP_CONTEXT deletes it,
and the Companion's borrowed queue runs ~OtaContext() in place. A self-serving
node reaches release with both self-serve buffers populated, so every
acquire/release cycle leaked them.

Give OtaContext a destructor that releases what it owns, and delete the copy
operations so the raw pointers cannot be double-freed.
2026-09-09 01:24:00 -07:00
MUSTARDTIGERFPV afa1036304 Move large repeater tables off classic ESP32 static DRAM
Classic ESP32 links into a dram0_0_seg of only 124,580 bytes: memory.ld
carves the BT controller's 0xdb5c (56,156) reservation off the 0x2c200
window before the application gets any. Three large objects dominated what
was left, and Tbeam_SX1262_repeater_observer_mqtt overran the build's 8 KiB
static reserve by 4,744 bytes.

Heap-allocate them instead, each with a defined behaviour when the
allocation fails:

- OtaContext: generalize the Companion's borrowed-storage path behind
  OTA_DYNAMIC_CONTEXT and add OTA_HEAP_CONTEXT, which allocates the context
  on first use and frees it once no transfer, apply or folder link needs it.
  A repeater is idle nearly all the time, so the workspace is usually absent.
  Exhaustion is reported to the caller and the operation declines.
- ClientACL: the client table becomes a pointer with a live capacity;
  capacity 0 refuses new clients rather than writing through a null table.
- MyMesh flood packet filters: likewise, with flood_packet_filter_slots as
  the live bound for every rule loop. At capacity 0 the node forwards
  unfiltered, and both save paths refuse to write so a stored ruleset is
  never replaced by an empty file.

Also override the Arduino SDK's weak btInUse() in simple_repeater so
initArduino() releases the BT controller memory to the heap. That grows the
runtime heap these tables now come from; it cannot recover the same
reservation from the linker's static window.

Static DRAM, classic ESP32:
  Heltec_v2_repeater                   108,332 -> 70,732
  Tbeam_SX1262_repeater_observer_mqtt  121,132 -> 83,532  (was failing)
2026-09-09 01:24:00 -07:00
mikecarper aab46c7c84 Use native display resolution and adaptive typography 2026-09-08 16:02:16 -07:00
mikecarper 00015b8333 Restore Wireless Paper Full capacity with shared mOTA queue storage 2026-09-08 08:09:42 -07:00
mikecarper 0d7ca2d343 Gate every firmware build on runtime RAM and share nRF52 mOTA storage 2026-09-08 02:09:26 -07:00
mikecarper 2ea493994e Share T096 Full mOTA memory with the offline queue 2026-09-08 01:15:19 -07:00
mikecarper 2d50bad70a Add retained-RAM staging for nRF52 mOTA 2026-09-05 12:08:21 -07:00
mikecarper b724d19d6f Protect ESP32 HWCDC console sessions 2026-09-04 17:18:54 -07:00
mikecarper e36b092aa3 Fix USB radio stalls and add durable replay timestamp recovery 2026-09-03 23:58:32 -07:00
mikecarper 5a96dc613b Avoid overflow in OTA self block count 2026-09-03 01:08:28 -07:00
mikecarper 8f8e8b411f Support 2 KiB DEFLATE OTA transport 2026-09-03 01:07:06 -07:00