diff --git a/.github/workflows/run-unit-tests.yml b/.github/workflows/run-unit-tests.yml index 07f2a229..6ea5e5f0 100644 --- a/.github/workflows/run-unit-tests.yml +++ b/.github/workflows/run-unit-tests.yml @@ -22,6 +22,11 @@ jobs: --firmware-version v1.17.1.10-halo-keymind-cascade-dev --radio-preset usa-cascade-fixed --profile cascade --require-ota + - name: Verify linked Companion crypto, retained Reader and ARM stack use + run: | + python3 -m pip install unicorn==2.1.4 + python3 -B scripts/check_nrf52_companion_crypto.py .pio/build/RAK_4631_companion_radio_full/firmware.elf + nrf52-sensor-smoke: runs-on: ubuntu-latest steps: diff --git a/docs/research/firmware_memory_budget.md b/docs/research/firmware_memory_budget.md index 4c6fa635..bb9bf0dc 100644 --- a/docs/research/firmware_memory_budget.md +++ b/docs/research/firmware_memory_budget.md @@ -315,6 +315,59 @@ refuses the loan; sync messages with an app first. USB/TCP source detach or disconnect returns all 256 slots and releases the ESP32 proof/leaf scratch buffers. The display and simultaneous USB, Bluetooth and WiFi remain enabled. +## nRF52 Companion flash savings + +The common nRF52 Companion policy now retains a 960-byte Ed25519 fixed-base +table in place of the original 30,720-byte table. It evaluates the same signed +radix-16 digits with constant-time table selection, using more doublings instead +of storing 32 shifted rows. Compact SHA-512 and scalar arithmetic keep the same +outputs and wire format. The larger RAM-built table used by infrastructure +profiles is not enabled for Companions. + +The policy also omits unused USB device MSC/HID/MIDI/vendor/video classes, +SparkFun's unused automatic NMEA cache and the invisible SSD1306 vendor splash. +For the exact hash-qualified CC310 archive, the domain wrapper retains P-256 +for BLE Secure Connections and removes unused legacy curves. CDC, DFU, USB host, +GPS navigation, all Companion fonts, Meshtext, contacts, channels, the Reader +corpus and the 8 KiB loop stack remain available. Explicit board/user overrides +and `MESH_NRF52_COMPANION_TRIM=0` retain their requested values. Other platforms +do not receive this Companion policy. + +The RAK4631 Full comparison uses the released 1.17.1.10 preview3 ELF at +`9ce2a076` and the same target with these changes and Reader chapter/range +support. Flash is initialized linked code/data; static RAM is `.data` plus +ordinary `.bss`, excluding separately reserved arenas. + +| Measurement | Preview3 | With Companion trims and Reader ranges | Difference | +| --- | ---: | ---: | ---: | +| Linked flash bytes | 710,404 | 652,908 | 57,496 fewer (56.1 KiB) | +| Static RAM bytes | 157,028 | 153,876 | 3,152 fewer (3.1 KiB) | +| Linked heap capacity bytes | 78,484 | 81,636 | 3,152 more | +| UF2 space below the application ceiling | 2,048 | 59,648 | 57,600 more | + +Physical RAK signing averaged 14.4 ms with the original table and 35.9 ms with +the smaller implementation. All 24 signatures per image, including SHA-512 +block-boundary message lengths, matched byte for byte and verified independently. +The linked ARM emulator measures 1,928 bytes in signing versus 1,528 before; +these are crypto function frames only, excluding callers, interrupts and other +tasks. The loop stack stays at 8 KiB. A full firmware RAM check still applies. + +CI runs `scripts/check_nrf52_companion_crypto.py` on the actual Full RAK ELF. +It checks the 960-byte table and retained Reader corpus, executes two RFC 8032 +vectors, and rejects crypto function stack use above 2 KiB. Host tests compare +1,791 scalars and 128 keypair/signature/shared-secret cases against the original +implementation, plus scalar boundary/random and sanitizer tests. Clean Full +builds also cover T1000-E and T096. These builds are synthetic coverage for +those boards; the physical Reader/signing tests use the RAK4631. + +Reader commands support `get reader 2` and `get reader 2:2-12`. Text terminals +stream at most 96 bytes per mesh loop, preserving progress under zero/short +writes. The TCP console uses one nonblocking socket-send attempt for the stream, +so a peer that stops reading cannot hold the mesh loop in WiFiClient retries. +App CLI replies remain bounded and supply a continuation command. +The physical RAK passed all 21 chapters, ranges, app-style continuation replies, +a paused USB reader exceeding the 4 KiB queue, and disconnect cancellation. + ## Release evidence and regression tests Full Companions without PSRAM also use diff --git a/docs/terminal_chat_cli.md b/docs/terminal_chat_cli.md index b71995e6..26537bfe 100644 --- a/docs/terminal_chat_cli.md +++ b/docs/terminal_chat_cli.md @@ -200,10 +200,21 @@ ESP32 and nRF52 Full Companions include an offline World English Bible lookup: ```text get reader 3:16 +get reader 2 +get reader 2:2-12 ``` -This returns a single complete verse from the embedded corpus, using plain ASCII punctuation -with unchanged wording and capitalization, through the local USB/TCP terminal. +These select a verse, an entire chapter, or an inclusive verse range from the +embedded corpus. Local USB/TCP/browser terminals stream the complete selection +in small chunks, keeping the radio loop running while the terminal drains. +A new command or disconnected session cancels unfinished output. Text uses +plain ASCII punctuation with unchanged wording and capitalization. + +In the app's USB/BLE CLI, each reply stays within the existing frame limit. +Follow the displayed `Next: get reader ...` command to continue a long verse +or advance through the selected chapter/range. The optional final number is +the page within the first verse of that command, for example `get reader 21:17 2`. +The continuation command retains the requested range's end. See [Reader lookup and compression](https://github.com/mikecarper/MeshCore/blob/keymindCascade/tools/bible/README.md) for the source, supported build profiles and memory costs. diff --git a/examples/companion_radio/MyMesh.cpp b/examples/companion_radio/MyMesh.cpp index 8538dc4a..74cc6936 100644 --- a/examples/companion_radio/MyMesh.cpp +++ b/examples/companion_radio/MyMesh.cpp @@ -8663,6 +8663,9 @@ Stream& MyMesh::terminalOutput() { } void MyMesh::resetTerminalSession() { +#if COMPANION_FEATURE_READER + mesh::cancelReaderCommand(); +#endif _terminal_recipient_set = false; memset(_terminal_recipient_key, 0, sizeof(_terminal_recipient_key)); clearTerminalLogin(); @@ -9332,13 +9335,21 @@ void MyMesh::sendTerminalRawTrace(const char* arguments) { target); } -void MyMesh::handleTerminalCommand(char* command) { +#if defined(WITH_WEBCONFIG) && COMPANION_FEATURE_READER +bool MyMesh::streamTerminalBusy() const { + return mesh::readerCommandActive(_terminal_output); +} +#endif + +void MyMesh::handleTerminalCommand(char* command, bool reader_prompt, + mesh::ReaderTryWrite reader_try_write) { while (*command == ' ' || *command == '\t') command++; if (*command == 0) return; mesh::cli::normalizeCommandVerb(command); #if COMPANION_FEATURE_READER - if (mesh::handleReaderCommand(command, terminalOutput())) return; + if (mesh::handleReaderCommand(command, terminalOutput(), reader_prompt, + reader_try_write)) return; #endif // Match the repeater ASCII reply envelope so unmodified serial packet @@ -9847,7 +9858,7 @@ void MyMesh::handleTerminalCommand(char* command) { terminalOutput().print(" get flash.health\r\n"); #endif #if COMPANION_FEATURE_READER - terminalOutput().print(" get reader : (World English Bible, offline)\r\n"); + terminalOutput().print(" get reader [:[-]] [page] (World English Bible, offline)\r\n"); #endif terminalOutput().print(" get pwrmgt.bootreason\r\n"); #if COMPANION_FEATURE_MEMORY_DIAGNOSTICS @@ -10001,6 +10012,9 @@ void MyMesh::handleTerminalCommand(char* command) { #endif void MyMesh::enterCLIRescue() { +#if COMPANION_FEATURE_READER + mesh::cancelReaderCommand(); +#endif _cli_rescue = true; cli_command[0] = 0; _cli_line_overflow = false; @@ -10608,7 +10622,7 @@ complete_line: reply_buf[0] = 0; #if COMPANION_FEATURE_READER if (mesh::handleReaderCommand(cli_command, output)) { - // USB rescue is a text stream and can print the complete verse. + // USB rescue streams chapters/ranges cooperatively like the terminal. } else #endif if (handleCommand(cli_command, 0, reply_buf)) { @@ -10935,6 +10949,9 @@ void MyMesh::loop() { serviceTerminalTrace(); #endif servicePendingSerialReply(); +#if COMPANION_FEATURE_READER + mesh::serviceReaderCommand(); +#endif serviceBinaryTraceReply(); serviceSigningSession(); if (!command_radio_apply_pending && saved_radio_apply_pending && !hasOutbound() @@ -11085,6 +11102,9 @@ bool MyMesh::canRecoverUsbLogging() const { } bool MyMesh::hasPendingWork() const { +#if COMPANION_FEATURE_READER + if (mesh::readerCommandActive()) return true; +#endif #if COMPANION_FEATURE_NOTIFICATIONS if (_notifications.active()) return true; #endif diff --git a/examples/companion_radio/MyMesh.h b/examples/companion_radio/MyMesh.h index b388887c..479a68e4 100644 --- a/examples/companion_radio/MyMesh.h +++ b/examples/companion_radio/MyMesh.h @@ -4,6 +4,7 @@ #include #include "AbstractUITask.h" #include "CompanionFeatures.h" +#include /*------------ Frame Protocol --------------*/ #define FIRMWARE_VER_CODE 14 @@ -255,7 +256,10 @@ public: bool ownsStreamTerminal(const Stream& output) const override { return isNetworkTerminalMode(output); } - void runStreamTerminal(char* command) override { handleTerminalCommand(command); } + void runStreamTerminal(char* command) override { handleTerminalCommand(command, false); } +#if COMPANION_FEATURE_READER + bool streamTerminalBusy() const override; +#endif void endStreamTerminal(Stream& output) override; #endif void rebootNow() override; @@ -301,7 +305,8 @@ public: bool isTerminalWaitingForInput() const { return _terminal_mode && _terminal_usb_silent; } - void handleTerminalCommand(char* command); + void handleTerminalCommand(char* command, bool reader_prompt = true, + mesh::ReaderTryWrite reader_try_write = nullptr); #if COMPANION_FEATURE_NETWORK_TERMINAL || defined(WITH_WEBCONFIG) bool enterNetworkTerminalMode(Stream& output); void exitNetworkTerminalMode(Stream& output); diff --git a/examples/companion_radio/main.cpp b/examples/companion_radio/main.cpp index 69d27e5d..52572b5e 100644 --- a/examples/companion_radio/main.cpp +++ b/examples/companion_radio/main.cpp @@ -6,6 +6,7 @@ #include #include #include "MyMesh.h" +#include #include "CompanionBluetooth.h" #include "CompanionWireless.h" #include "CompanionWiFi.h" @@ -989,6 +990,9 @@ static bool usb_terminal_host_reset_completion_pending = false; static uint32_t usb_terminal_host_reset_retry_at = 0; static void resetUsbTerminalHostSession() { +#if COMPANION_FEATURE_READER + mesh::cancelReaderCommand(&mesh::usbTerminalPort()); +#endif // Stop only work routed to the old USB host; a simultaneous BLE/WiFi // transaction is independent and must continue. Clear the route after // stopping its producer so no response can fall back to another transport. @@ -1230,7 +1234,10 @@ static void serviceUsbTerminal() { usb_logging_terminal_mode = false; } #endif - usbTerminalOutput().print("> "); +#if COMPANION_FEATURE_READER + if (!mesh::readerCommandActive(&usbTerminalOutput())) +#endif + usbTerminalOutput().print("> "); return; // service at most one command per mesh loop } @@ -2087,6 +2094,7 @@ void halt() { the same role CLI here as its USB terminal. Other OTA-enabled Companion builds expose local maintenance commands and `ota ...` commands. */ #if defined(ESP32) && defined(WIFI_SSID) && defined(ENABLE_OTA) + #include #include // mesh::ota::handle_ota_command(line, reply, board) #include #ifndef OTA_CONSOLE_TCP_PORT @@ -2097,6 +2105,17 @@ void halt() { static char ota_console_line[MAX_TRANS_UNIT * 2 + 32]; static size_t ota_console_len = 0; static bool ota_console_discard_line = false; +#if COMPANION_FEATURE_NETWORK_TERMINAL && COMPANION_FEATURE_READER + static size_t ota_console_try_reader_write(Stream& output, + const uint8_t* data, size_t size) { + // WiFiClient::write retries socket readiness with one-second waits. + // A paused chapter reader must instead leave its cursor for the next loop. + const int socket = static_cast(output).fd(); + if (socket < 0) return 0; + const int written = ::send(socket, data, size, MSG_DONTWAIT); + return written > 0 ? static_cast(written) : 0; + } +#endif #if COMPANION_FEATURE_NETWORK_TERMINAL && defined(ENABLE_USB_INTERFACE) static mesh::UsbTcpTerminalHandoff ota_console_usb_handoff; #endif @@ -2220,8 +2239,16 @@ void halt() { ota_console_discard_line = false; return; } +#if COMPANION_FEATURE_READER + the_mesh.handleTerminalCommand(ota_console_line, true, + ota_console_try_reader_write); +#else the_mesh.handleTerminalCommand(ota_console_line); - ota_console_client.print("> "); +#endif +#if COMPANION_FEATURE_READER + if (!mesh::readerCommandActive(&ota_console_client)) +#endif + ota_console_client.print("> "); #else char reply[160]; reply[0] = 0; if (!handleCompanionWirelessCommand(ota_console_line, reply, sizeof(reply), CompanionWirelessSource::Network) diff --git a/lib/ed25519/ge.c b/lib/ed25519/ge.c index 32de576b..64d94683 100644 --- a/lib/ed25519/ge.c +++ b/lib/ed25519/ge.c @@ -412,6 +412,28 @@ void ge_scalarmult_base(ge_p3 *h, const unsigned char *a) { /* each e[i] is between -8 and 8 */ ge_p3_0(h); +#if defined(ED25519_SMALL_BASE) && ED25519_SMALL_BASE + /* Keep only the existing 1B..8B row (960 bytes) in flash. Evaluate the + * same signed radix-16 digits from most to least significant instead of + * keeping 32 shifted rows (30 KiB). The selection still scans all eight + * entries with cmov; no loop count/address depends on a secret digit. + * Unlike the RAM-built compact option, this uses the ordinary scratch + * objects above and does not add a multiples[] table to the stack. + */ + for (i = 63; i >= 0; --i) { + ge_p3_dbl(&r, h); + ge_p1p1_to_p2(&s, &r); + ge_p2_dbl(&r, &s); + ge_p1p1_to_p2(&s, &r); + ge_p2_dbl(&r, &s); + ge_p1p1_to_p2(&s, &r); + ge_p2_dbl(&r, &s); + ge_p1p1_to_p3(h, &r); + select(&t, 0, e[i]); + ge_madd(&r, h, &t); + ge_p1p1_to_p3(h, &r); + } +#else for (i = 1; i < 64; i += 2) { select(&t, i / 2, e[i]); ge_madd(&r, h, &t); @@ -432,6 +454,7 @@ void ge_scalarmult_base(ge_p3 *h, const unsigned char *a) { ge_madd(&r, h, &t); ge_p1p1_to_p3(h, &r); } +#endif } #else /* diff --git a/lib/ed25519/precomp_data.h b/lib/ed25519/precomp_data.h index e5d44a43..42e1aa09 100644 --- a/lib/ed25519/precomp_data.h +++ b/lib/ed25519/precomp_data.h @@ -43,8 +43,11 @@ static const ge_precomp Bi[8] = { /* base[i][j] = (j+1)*256^i*B */ +#if defined(ED25519_SMALL_BASE) && ED25519_SMALL_BASE && defined(ED25519_COMPACT_BASE) +#error "Choose either the small flash table or the RAM-built compact table" +#endif #ifndef ED25519_COMPACT_BASE -static const ge_precomp base[32][8] = { +static const ge_precomp base[][8] = { { { { 25967493, -14356035, 29566456, 3660896, -12694345, 4014787, 27544626, -11754271, -6079156, 2047605 }, @@ -87,6 +90,7 @@ static const ge_precomp base[32][8] = { { 27431194, 8222322, 16448760, -3907995, -18707002, 11938355, -32961401, -2970515, 29551813, 10109425 }, }, }, +#if !defined(ED25519_SMALL_BASE) || !ED25519_SMALL_BASE { { { -13657040, -13155431, -31283750, 11777098, 21447386, 6519384, -2378284, -1627556, 10092783, -4764171 }, @@ -1389,5 +1393,6 @@ static const ge_precomp base[32][8] = { { -20430234, 14955537, -24126347, 8124619, -5369288, -5990470, 30468147, -13900640, 18423289, 4177476 }, }, }, +#endif }; #endif diff --git a/platformio.ini b/platformio.ini index 902e115e..8f209a78 100644 --- a/platformio.ini +++ b/platformio.ini @@ -170,7 +170,7 @@ extra_scripts = create-uf2.py pre:scripts/nrf52_internal_bootloader_link.py ; Nordic's builder otherwise overwrites the LTO-aware archive tools. pre:scripts/nrf52_lto_toolchain.py - ; Infrastructure-only trims; all Companion/Terminal Chat profiles excluded. + ; Role-scoped trims: Companion keeps bounded crypto stack, fonts and features. pre:scripts/nrf52_flash_trim.py post:scripts/check_firmware_ram.py post:scripts/check_nrf52_ble_dfu_handoff.py diff --git a/scripts/check_nrf52_companion_crypto.py b/scripts/check_nrf52_companion_crypto.py new file mode 100644 index 00000000..d83b4d53 --- /dev/null +++ b/scripts/check_nrf52_companion_crypto.py @@ -0,0 +1,119 @@ +#!/usr/bin/env python3 +"""Execute the linked nRF52 Companion crypto and bound its function stack. + +Requires unicorn==2.1.4. This runs the actual ARM instructions from a release +ELF, not a host rebuild. It measures crypto call frames only: it does not +include their firmware callers, interrupts, or SoftDevice tasks, and does not +replace physical timing/transport tests or the full firmware RAM budget. +""" + +import argparse +import hashlib +import json +import struct + +from firmware_elf import FirmwareElf +from unicorn import Uc, UC_ARCH_ARM, UC_MODE_THUMB, UC_MODE_MCLASS, UC_HOOK_CODE +from unicorn import arm_const as arm + + +# RFC 8032 section 7.1, TEST 1 and TEST 2. Public test seeds, never device keys. +VECTORS = ( + ( + "9d61b19deffd5a60ba844af492ec2cc44449c5697b326919703bac031cae7f60", + "d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a", + "", + "e5564300c360ac729086e2cc806e828a84877f1eb8e5d974d873e065224901555f" + "b8821590a33bacc61e39701cf9b46bd25bf5f0595bbe24655141438e7a100b", + ), + ( + "4ccd089b28ff96da9db6c346ec114e0f5b8a319f35aba624da8cf6ed4fb8a6fb", + "3d4017c3e843895a92b70aa74d1b7ebc9c982ccf2ec4968cc0cd55f12af4660c", + "72", + "92a009a9f0d4cab8720e820b5f642540a2b27b5416503f8fb3762223ebdb69da0" + "85ac1e43e15996e458f3613d0f11d8c387b2eaeb4302aeeb00d291612bb0c00", + ), +) + + +def check(path): + elf = FirmwareElf(path) + if elf.unpack("H", 18)[0] != 40: + raise ValueError("expected an ARM firmware ELF") + if elf.symbols.get("base", (0, 0))[1] != 960: + raise ValueError("Companion must retain the 960-byte Ed25519 base table") + reader = "_ZN4mesh5bible9generatedL10readerDataE" + if reader not in elf.symbols: + raise ValueError("Companion Reader corpus is missing") + uc = Uc(UC_ARCH_ARM, UC_MODE_THUMB | UC_MODE_MCLASS) + uc.mem_map(0, 0x100000) + uc.mem_map(0x20000000, 0x40000) + for section in elf.sections: + if section[1] != 8 and section[2] & 2 and section[5]: + uc.mem_write(section[3], elf.section_bytes(section)) + stack_top, stop = 0x2003F000, 0x1000 + uc.mem_write(stop, b"\x00\xbe") + state = {"min_sp": stack_top} + + def instruction(machine, _address, _size, _context): + # Instruction-level hook includes stack space in leaf functions that + # have no branch between reserving and releasing their stack frame. + state["min_sp"] = min(state["min_sp"], machine.reg_read(arm.UC_ARM_REG_SP)) + + uc.hook_add(UC_HOOK_CODE, instruction) + + def call(name, args): + uc.reg_write(arm.UC_ARM_REG_SP, stack_top) + uc.reg_write(arm.UC_ARM_REG_LR, stop | 1) + for register, value in zip((arm.UC_ARM_REG_R0, arm.UC_ARM_REG_R1, + arm.UC_ARM_REG_R2, arm.UC_ARM_REG_R3), args): + uc.reg_write(register, value) + for index, value in enumerate(args[4:]): + uc.mem_write(stack_top + index * 4, struct.pack(" 2048: + raise ValueError(f"{name} uses {used} stack bytes; maximum is 2048") + return used + + seed, public, private, signature, message = ( + 0x20030000, 0x20030040, 0x20030080, 0x20030100, 0x20030200) + rows = [] + for index, vector in enumerate(VECTORS, 1): + source, expected_public, text, expected_signature = map(bytes.fromhex, vector) + uc.mem_write(seed, source) + uc.mem_write(message, text or b"\0") + key_stack = call("ed25519_create_keypair", [public, private, seed]) + if bytes(uc.mem_read(public, 32)) != expected_public: + raise ValueError(f"RFC 8032 vector {index}: public key mismatch") + sign_stack = call("ed25519_sign", [signature, message, len(text), public, private]) + if bytes(uc.mem_read(signature, 64)) != expected_signature: + raise ValueError(f"RFC 8032 vector {index}: signature mismatch") + rows.append({"rfc8032_vector": index, "keypair_stack_bytes": key_stack, + "sign_stack_bytes": sign_stack}) + return { + "passed": True, "elf_sha256": hashlib.sha256(elf.data).hexdigest(), + "base_table_bytes": elf.symbols["base"][1], + "reader_data_bytes": elf.symbols[reader][1], "vectors": rows, + "stack_limit_bytes": 2048, + "scope": "Crypto function frames only; excludes caller/ISR/task stacks and physical timing.", + } + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("elf") + args = parser.parse_args() + try: + print(json.dumps(check(args.elf), indent=2)) + except (ValueError, OSError) as error: + parser.exit(1, f"FAILED: {error}\n") + + +if __name__ == "__main__": + main() diff --git a/scripts/nrf52_flash_trim.py b/scripts/nrf52_flash_trim.py index 46ff7511..a3c8da81 100644 --- a/scripts/nrf52_flash_trim.py +++ b/scripts/nrf52_flash_trim.py @@ -1,11 +1,13 @@ -"""Feature-preserving flash trim for nRF52 infrastructure, never Companions. +"""Role-specific feature-preserving flash trims for nRF52 firmware. Decide from the effective source filter as well as the environment/OTA alias and flags: build.sh can synthesize profiles over another PlatformIO recipe. Unknown roles fail closed. No shared framework or downloaded library is edited. CDC, USB host, sensor/GPS support, BLE P-256, and stored Ed25519 keys remain. -The compact Ed25519 implementation trades more signing time/stack for flash; -the nRF52 base keeps an 8 KiB loop stack for this and the existing OTA path. +Infrastructure uses RAM-built compact Ed25519 and display-font trims. +Companion uses a 960-byte Ed25519 flash table to keep signing stack bounded, +and retains all UI fonts and the Reader. The nRF52 base keeps an 8 KiB loop +stack for crypto and the existing OTA path. """ import fnmatch @@ -49,6 +51,20 @@ TRIM_DEFINES = { CC310_ARCHIVE_SHA256 = "97dc648d44520e47252f62c402a71f976e36d7b3aef99235bbdc4de79a928577" CC310_ARCHIVE = "src/cortex-m4/fpv4-sp-d16-hard/libnrf_cc310_0.9.13-no-interrupts.a" +# Companion retains all UI fonts and uses a small Ed25519 flash table instead +# of the infrastructure implementation's extra 1.8 KiB of signing stack. +# These options remove library code/data that its application never uses; +# CDC, DFU, BLE pairing, USB host and UBX navigation remain available. +COMPANION_TRIM_DEFINES = { + name: value for name, value in TRIM_DEFINES.items() + if name not in ("MESH_NRF52_FLASH_TRIM", "ED25519_COMPACT_BASE", + "ED25519_COMPACT_SHA512") +} +COMPANION_TRIM_DEFINES["MESH_NRF52_COMPANION_TRIM"] = 1 +COMPANION_TRIM_DEFINES["ED25519_SMALL_BASE"] = 1 +COMPANION_TRIM_DEFINES["ED25519_COMPACT_SHA512"] = 1 +COMPANION_TRIM_DEFINES["ED25519_COMPACT_SCALAR"] = 1 + def flag_defines(flags=(), cppdefines=()): """Read effective defines without importing PlatformIO/SCons for tests.""" @@ -133,6 +149,27 @@ def supported_cc310(framework_dir, mcu, defines): return archive.is_file() and hashlib.sha256(archive.read_bytes()).hexdigest() == CC310_ARCHIVE_SHA256 +def companion_trim_policy(platform, mcu, src_filter, flags=(), cppdefines=()): + """Qualify the actual Companion sources, independent of board/alias name.""" + if platform != "nordicnrf52" or not str(mcu).lower().startswith("nrf52"): + return {} + if not source_selected(src_filter, COMPANION_SOURCES[0]): + return {} + if any(source_selected(src_filter, source) for source in ROLE_SOURCES): + return {} + values = flag_defines(flags, cppdefines) + for name in ("MESH_NRF52_FLASH_TRIM", "MESH_NRF52_COMPANION_TRIM"): + if name in values and not _enabled(values[name]): + return {} + definitions = {name: value for name, value in COMPANION_TRIM_DEFINES.items() + if name not in values} + # The older compact option uses #ifdef, so even an explicit =0 selects + # that path. Preserve any caller choice instead of enabling both modes. + if values.get("ED25519_COMPACT_BASE") is not None: + definitions.pop("ED25519_SMALL_BASE", None) + return definitions + + def trim_tft_library(build_env, node): # PlatformIO invokes this on TFT_eSPI's library builder, which is already # a private clone. Do not add a global forced include to the project: its @@ -153,14 +190,21 @@ def install(build_env): cppdefines = build_env.get("CPPDEFINES", ()) definitions = trim_policy(platform.name, mcu, build_env["PIOENV"], build_env.get("SRC_FILTER", ()), flags, cppdefines) + infrastructure = bool(definitions) + if not infrastructure: + definitions = companion_trim_policy(platform.name, mcu, + build_env.get("SRC_FILTER", ()), flags, cppdefines) if not definitions: return build_env["MESH_NRF52_FLASH_TRIM_ACTIVE"] = True build_env.AppendUnique(CPPDEFINES=list(definitions.items())) - print("nRF52 infrastructure flash trim: compact crypto, unused USB device classes/GPS cache/vendor splash, compact OTA target names") + if infrastructure: + print("nRF52 infrastructure flash trim: compact crypto, unused USB device classes/GPS cache/vendor splash, compact OTA target names") + else: + print("nRF52 Companion flash trim: small crypto table, unused USB device classes/GPS cache/vendor splash; fonts and Reader retained") framework = platform.get_package_dir("framework-arduinoadafruitnrf52") values = flag_defines(flags, cppdefines) - if values.get("ST7789") is not None: + if infrastructure and values.get("ST7789") is not None: # Keep the font license with this qualified display build, including # cached DFU packages. The hook independently checks the effective # font override and excludes every Companion transport/profile. @@ -169,7 +213,7 @@ def install(build_env): font_license = importlib.util.module_from_spec(hook_spec) hook_spec.loader.exec_module(font_license) font_license.install(build_env) - if (values.get("DISPLAY_CLASS") == "ST7735Display" + if (infrastructure and values.get("DISPLAY_CLASS") == "ST7735Display" and source_selected(build_env.get("SRC_FILTER", ()), "helpers/ui/ST7735Display.cpp")): build_env.AddBuildMiddleware(trim_tft_library, "*TFT_eSPI*/*.cpp") if framework and supported_cc310(framework, mcu, values): @@ -177,7 +221,7 @@ def install(build_env): build_env.AppendUnique(CPPPATH=[str(Path(framework) / "libraries/Adafruit_nRFCrypto/src/nrf_cc310/include")]) # BuildSources is called on this selected nRF52 environment only. The # wrapper is not in the common source filter and cannot leak to other - # platforms or an excluded Companion role. + # platforms or an unqualified role. build_env.BuildSources("$BUILD_DIR/nrf52-flash-trim", "$PROJECT_DIR/src/helpers/nrf52", "+") diff --git a/src/helpers/CompanionReader.cpp b/src/helpers/CompanionReader.cpp index 9dad41a0..ddb97aeb 100644 --- a/src/helpers/CompanionReader.cpp +++ b/src/helpers/CompanionReader.cpp @@ -15,8 +15,14 @@ bible::LookupResult readReaderVerse(bible::Reference ref, char* scratch, namespace { constexpr size_t kReplyPageTextBytes = 100; +constexpr const char* kUsage = "ERROR: use get reader [:[-]] [page]"; -bible::ParseResult parseReaderRequest(const char* command, bible::Reference& ref, +struct Selection { + bible::Reference first; + uint8_t last; +}; + +bible::ParseResult parseReaderRequest(const char* command, Selection& selection, uint8_t& page) { page = 0; // No page argument keeps text terminals' full-verse behavior. if (!command) return bible::ParseResult::NoMatch; @@ -25,13 +31,23 @@ bible::ParseResult parseReaderRequest(const char* command, bible::Reference& ref bible::skipSpace(command); if (!bible::token(command, "reader")) return bible::ParseResult::NoMatch; bible::skipSpace(command); - if (!bible::number(command, ref.chapter) || *command != ':') { + bible::Reference& ref = selection.first; + if (!bible::number(command, ref.chapter) || ref.chapter > sizeof(bible::kChapterVerses)) { return bible::ParseResult::Invalid; } - ++command; - if (!bible::number(command, ref.verse)) return bible::ParseResult::Invalid; - uint16_t unused; - if (!bible::verseNumber(ref, unused)) return bible::ParseResult::Invalid; + ref.verse = 1; + selection.last = bible::kChapterVerses[ref.chapter - 1]; + if (*command == ':') { + ++command; + if (!bible::number(command, ref.verse)) return bible::ParseResult::Invalid; + selection.last = ref.verse; + if (*command == '-') { + ++command; + if (!bible::number(command, selection.last)) return bible::ParseResult::Invalid; + } + if (ref.verse > selection.last || selection.last > bible::kChapterVerses[ref.chapter - 1]) + return bible::ParseResult::Invalid; + } if (*command) { if (!bible::space(*command)) return bible::ParseResult::Invalid; bible::skipSpace(command); @@ -41,10 +57,10 @@ bible::ParseResult parseReaderRequest(const char* command, bible::Reference& ref return *command == 0 ? bible::ParseResult::Valid : bible::ParseResult::Invalid; } -size_t nextReplyPage(const char* text, size_t start, size_t length) { +size_t nextReplyPage(const char* text, size_t start, size_t length, size_t limit) { size_t count = length - start; - if (count > kReplyPageTextBytes) { - count = kReplyPageTextBytes; + if (count > limit) { + count = limit; // Keep the separating space in this page, so following every page // reconstructs the supplied verse exactly without lost or repeated words. for (size_t n = count; n > 0; --n) { @@ -57,19 +73,37 @@ size_t nextReplyPage(const char* text, size_t start, size_t length) { return start + count; } -void formatVerseReply(bible::Reference ref, uint8_t page, const char* text, +void nextCommand(char* out, size_t capacity, Selection selection, uint8_t page) { + if (selection.first.verse == selection.last) { + snprintf(out, capacity, "\nNext: get reader %u:%u %u", selection.first.chapter, + selection.first.verse, page); + } else { + snprintf(out, capacity, "\nNext: get reader %u:%u-%u %u", selection.first.chapter, + selection.first.verse, selection.last, page); + } +} + +void formatVerseReply(Selection selection, uint8_t page, const char* text, char* reply, size_t capacity) { + const bible::Reference ref = selection.first; + char next_command[48] = {}; + if (ref.verse < selection.last) { + Selection remaining = selection; + ++remaining.first.verse; + nextCommand(next_command, sizeof(next_command), remaining, 1); + } char header[40]; snprintf(header, sizeof(header), "Reader %u:%u (WEB)\n", static_cast(ref.chapter), static_cast(ref.verse)); const size_t length = strlen(text); const size_t header_len = strlen(header); - if (header_len + length < capacity) { + if (header_len + length + strlen(next_command) < capacity) { if (page != 1) { snprintf(reply, capacity, "ERROR: Reader page must be 1-1"); } else { memcpy(reply, header, header_len); - memcpy(reply + header_len, text, length + 1); + memcpy(reply + header_len, text, length); + strcpy(reply + header_len + length, next_command); } return; } @@ -77,7 +111,10 @@ void formatVerseReply(bible::Reference ref, uint8_t page, const char* text, unsigned parts = 0; size_t start = 0, end = 0; for (size_t offset = 0; offset < length;) { - const size_t next = nextReplyPage(text, offset, length); + // A range needs a longer continuation command. Keep the tagged 157-byte + // app reply budget, without changing single-verse page boundaries. + const size_t next = nextReplyPage(text, offset, length, + ref.verse < selection.last ? 80 : kReplyPageTextBytes); if (++parts == page) { start = offset; end = next; @@ -91,11 +128,8 @@ void formatVerseReply(bible::Reference ref, uint8_t page, const char* text, snprintf(header, sizeof(header), "Reader %u:%u [%u/%u]\n", static_cast(ref.chapter), static_cast(ref.verse), static_cast(page), parts); - char next_command[48] = {}; if (page < parts) { - snprintf(next_command, sizeof(next_command), "\nNext: get reader %u:%u %u", - static_cast(ref.chapter), static_cast(ref.verse), - static_cast(page + 1)); + nextCommand(next_command, sizeof(next_command), selection, page + 1); } const size_t prefix_len = strlen(header); const size_t text_len = end - start; @@ -111,13 +145,13 @@ void formatVerseReply(bible::Reference ref, uint8_t page, const char* text, // Only a recognized Reader request enters this 2 KiB decode frame. Normal // framed commands must not reserve it, including with link-time optimization. -__attribute__((noinline)) void replyVerse(bible::Reference ref, uint8_t page, +__attribute__((noinline)) void replyVerse(Selection selection, uint8_t page, char* reply, size_t capacity) { char scratch[bible::kBlockSize]; const char* text = nullptr; - const bible::LookupResult result = readReaderVerse(ref, scratch, sizeof(scratch), text); + const bible::LookupResult result = readReaderVerse(selection.first, scratch, sizeof(scratch), text); if (result == bible::LookupResult::Found) { - formatVerseReply(ref, page, text, reply, capacity); + formatVerseReply(selection, page, text, reply, capacity); } else if (result == bible::LookupResult::Missing) { snprintf(reply, capacity, "This verse is not present in the supplied translation."); } else { @@ -125,63 +159,127 @@ __attribute__((noinline)) void replyVerse(bible::Reference ref, uint8_t page, } } -__attribute__((noinline)) void printReplyPage(bible::Reference ref, uint8_t page, +__attribute__((noinline)) void printReplyPage(Selection selection, uint8_t page, Stream& output) { char reply[160]; - replyVerse(ref, page, reply, sizeof(reply)); + replyVerse(selection, page, reply, sizeof(reply)); output.print(" "); output.print(reply); output.print("\r\n"); } -// Keep the decode buffer out of the parser's frame, including under LTO: -// unrelated terminal commands must not reserve another 2 KiB of stack. -__attribute__((noinline)) void printVerse(bible::Reference ref, Stream& output) { +// Only a cursor is retained between loop calls: no chapter buffer or heap. +// A slow/unread USB endpoint must not overflow its fixed queue or hold the +// mesh loop waiting. A new command/session cancels the unfinished cursor. +struct PrintJob { + Stream* output = nullptr; + ReaderTryWrite try_write = nullptr; + Selection selection = {}; + uint16_t offset = 0; + uint8_t phase = 0; + bool prompt = false; +} print_job; + +bool writePart(const char* text) { + const size_t remaining = strlen(text) - print_job.offset; + const size_t count = remaining < 96 ? remaining : 96; + const uint8_t* data = reinterpret_cast(text + print_job.offset); + const size_t written = print_job.try_write + ? print_job.try_write(*print_job.output, data, count) + : print_job.output->write(data, count); + print_job.offset += written <= count ? written : count; + if (written < remaining) return false; + print_job.offset = 0; + return true; +} + +// Keep decode scratch off the ordinary command/loop frame, including LTO. +__attribute__((noinline)) bool writeVersePart() { char scratch[bible::kBlockSize]; const char* text = nullptr; - const bible::LookupResult result = readReaderVerse(ref, scratch, sizeof(scratch), text); - if (result == bible::LookupResult::Found) { - output.printf(" Reader %u:%u (%s)\r\n", static_cast(ref.chapter), - static_cast(ref.verse), bible::generated::readerCorpus.translation); - // Do not feed long text through Adafruit Print::printf's 256-byte scratch. - output.print(text); - output.print("\r\n"); - output.print(bible::generated::readerCorpus.attribution); - output.print("\r\n"); - } else if (result == bible::LookupResult::Missing) { - output.print(" This verse is not present in the supplied translation.\r\n"); - } else { - output.print(" ERROR: invalid compressed Reader data\r\n"); + const bible::LookupResult result = readReaderVerse( + print_job.selection.first, scratch, sizeof(scratch), text); + if (result == bible::LookupResult::Missing) + text = "This verse is not present in the supplied translation."; + else if (result != bible::LookupResult::Found) { + text = "ERROR: invalid compressed Reader data"; + print_job.selection.last = print_job.selection.first.verse; } + return writePart(text); } } // namespace -bool handleReaderCommand(const char* command, Stream& output) { - bible::Reference ref = {}; +void cancelReaderCommand(const Stream* output) { + if (!output || output == print_job.output) print_job.output = nullptr; +} + +bool readerCommandActive(const Stream* output) { + return print_job.output && (!output || output == print_job.output); +} + +void serviceReaderCommand() { + if (!print_job.output) return; + switch (print_job.phase) { + case 0: { + char header[48]; + snprintf(header, sizeof(header), " Reader %u:%u (%s)\r\n", + print_job.selection.first.chapter, print_job.selection.first.verse, + bible::generated::readerCorpus.translation); + if (writePart(header)) ++print_job.phase; + break; + } + case 1: + if (writeVersePart()) ++print_job.phase; + break; + case 2: + if (writePart("\r\n")) { + if (print_job.selection.first.verse < print_job.selection.last) { + ++print_job.selection.first.verse; + print_job.phase = 0; + } else ++print_job.phase; + } + break; + case 3: + if (writePart(bible::generated::readerCorpus.attribution)) ++print_job.phase; + break; + default: + if (writePart(print_job.prompt ? "\r\n> " : "\r\n")) cancelReaderCommand(); + break; + } +} + +bool handleReaderCommand(const char* command, Stream& output, bool prompt, + ReaderTryWrite try_write) { + cancelReaderCommand(); + Selection selection = {}; uint8_t page; - const bible::ParseResult parsed = parseReaderRequest(command, ref, page); + const bible::ParseResult parsed = parseReaderRequest(command, selection, page); if (parsed == bible::ParseResult::NoMatch) return false; if (parsed == bible::ParseResult::Invalid) { - output.print(" ERROR: use get reader : [page] (example: get reader 3:16)\r\n"); + output.print(" "); output.print(kUsage); output.print("\r\n"); } else if (page != 0) { - printReplyPage(ref, page, output); + printReplyPage(selection, page, output); } else { - printVerse(ref, output); + print_job.selection = selection; + print_job.offset = print_job.phase = 0; + print_job.prompt = prompt; + print_job.try_write = try_write; + print_job.output = &output; } return true; } bool handleReaderReply(const char* command, char* reply, size_t capacity) { if (reply == nullptr || capacity == 0) return false; - bible::Reference ref = {}; + Selection selection = {}; uint8_t page; - const bible::ParseResult parsed = parseReaderRequest(command, ref, page); + const bible::ParseResult parsed = parseReaderRequest(command, selection, page); if (parsed == bible::ParseResult::NoMatch) return false; if (parsed == bible::ParseResult::Invalid) { - snprintf(reply, capacity, "ERROR: use get reader : [page] (example: get reader 3:16)"); + snprintf(reply, capacity, "%s", kUsage); } else { - replyVerse(ref, page == 0 ? 1 : page, reply, capacity); + replyVerse(selection, page == 0 ? 1 : page, reply, capacity); } return true; } diff --git a/src/helpers/CompanionReader.h b/src/helpers/CompanionReader.h index cca90612..6adfe089 100644 --- a/src/helpers/CompanionReader.h +++ b/src/helpers/CompanionReader.h @@ -1,17 +1,29 @@ #pragma once #include "CompanionReaderConfig.h" +#include +#include +class Stream; +namespace mesh { +using ReaderTryWrite = size_t (*)(Stream&, const uint8_t*, size_t); +} #if COMPANION_FEATURE_READER #include "bible/ReaderLookup.h" -class Stream; namespace mesh { // Caller-owned scratch lets the terminal and screen share one flash corpus. bible::LookupResult readReaderVerse(bible::Reference ref, char* scratch, size_t capacity, const char*& text); -// Text terminals stream complete verses; an explicit page matches CLI replies. -bool handleReaderCommand(const char* command, Stream& output); +// Text terminals stream complete chapters/verse ranges cooperatively. Each +// service call writes at most 96 bytes, retaining progress after short writes. +// The owner must cancel before ending a session or destroying its Stream. +// try_write lets a transport bypass its ordinary blocking Stream::write. +bool handleReaderCommand(const char* command, Stream& output, bool prompt = false, + ReaderTryWrite try_write = nullptr); +void serviceReaderCommand(); +void cancelReaderCommand(const Stream* output = nullptr); +bool readerCommandActive(const Stream* output = nullptr); // Existing USB/BLE CLI frames contain one bounded reply. Short verses are -// complete; longer verses include the command for their next page. +// complete; longer verses/ranges include the command for their next page. bool handleReaderReply(const char* command, char* reply, size_t capacity); } #endif diff --git a/src/helpers/nrf52/Cc310DomainTrim.c b/src/helpers/nrf52/Cc310DomainTrim.c index 6ec083a1..f1ed0143 100644 --- a/src/helpers/nrf52/Cc310DomainTrim.c +++ b/src/helpers/nrf52/Cc310DomainTrim.c @@ -4,7 +4,8 @@ * API and are unaffected. Build-local linker wrapping is qualified by the * SDK archive hash in scripts/nrf52_flash_trim.py, never by MCU name alone. */ -#if defined(MESH_NRF52_FLASH_TRIM) && MESH_NRF52_FLASH_TRIM +#if (defined(MESH_NRF52_FLASH_TRIM) && MESH_NRF52_FLASH_TRIM) || \ + (defined(MESH_NRF52_COMPANION_TRIM) && MESH_NRF52_COMPANION_TRIM) #include #include "crys_ecpki_domain.h" diff --git a/test/fixtures/companion_reader/Arduino.h b/test/fixtures/companion_reader/Arduino.h index c9ea3ee1..376e39e6 100644 --- a/test/fixtures/companion_reader/Arduino.h +++ b/test/fixtures/companion_reader/Arduino.h @@ -1,5 +1,6 @@ #pragma once #include +#include #include #include #include @@ -8,7 +9,20 @@ class Stream { public: std::string text; - size_t print(const char* value) { text += value; return std::strlen(value); } + size_t max_write = 65535; + size_t credit = 65535; + size_t writes = 0; + size_t write(const uint8_t* value, size_t size) { + ++writes; + if (size > max_write) size = max_write; + if (size > credit) size = credit; + credit -= size; + text.append(reinterpret_cast(value), size); + return size; + } + size_t print(const char* value) { + return write(reinterpret_cast(value), std::strlen(value)); + } size_t printf(const char* format, ...) { char buf[256]; va_list args; diff --git a/test/fixtures/companion_reader/test_reader_lookup.cpp b/test/fixtures/companion_reader/test_reader_lookup.cpp index a2e307f6..84792736 100644 --- a/test/fixtures/companion_reader/test_reader_lookup.cpp +++ b/test/fixtures/companion_reader/test_reader_lookup.cpp @@ -8,6 +8,67 @@ using namespace mesh::bible; +static void drain(Stream& output) { + unsigned calls = 0; + while (mesh::readerCommandActive()) { + const size_t writes = output.writes, bytes = output.text.size(); + mesh::serviceReaderCommand(); + assert(output.writes <= writes + 1 && output.text.size() <= bytes + 96); + assert(++calls < 20000); + } +} + +static void checkSelection(unsigned chapter, unsigned first, unsigned last, bool whole) { + char command[64]; + if (whole) snprintf(command, sizeof(command), "get reader %u", chapter); + else snprintf(command, sizeof(command), "get reader %u:%u-%u", chapter, first, last); + std::string expected, terminal_expected; + for (unsigned verse = first; verse <= last; ++verse) { + char scratch[kBlockSize], header[48]; + const char* text; + assert(mesh::readReaderVerse({uint8_t(chapter), uint8_t(verse)}, scratch, + sizeof(scratch), text) == LookupResult::Found); + expected += text; + snprintf(header, sizeof(header), " Reader %u:%u (WEB)\r\n", chapter, verse); + terminal_expected += std::string(header) + text + "\r\n"; + } + terminal_expected += generated::readerCorpus.attribution; + terminal_expected += "\r\n> "; + Stream terminal; + terminal.max_write = 7; // Genuine short writes, including header/newline cuts. + assert(mesh::handleReaderCommand(command, terminal, true)); + assert(mesh::readerCommandActive(&terminal)); + for (unsigned tick = 0; mesh::readerCommandActive(); ++tick) { + assert(tick < 20000); + terminal.credit = tick % 3 == 0 ? 0 : 11; // Stops accepting bytes regularly. + const size_t before = terminal.text.size(); + mesh::serviceReaderCommand(); + assert(terminal.text.size() - before <= (tick % 3 == 0 ? 0 : 7)); + } + assert(terminal.text == terminal_expected); + for (const size_t capacity : {size_t(160), size_t(157)}) { + std::string next = command, reconstructed; + for (unsigned replies = 0; ; ++replies) { + assert(replies < 1000); + char reply[162]; + memset(reply, '!', sizeof(reply)); + assert(mesh::handleReaderReply(next.c_str(), reply, capacity)); + assert(reply[capacity] == '!' && strlen(reply) < capacity); + std::string page(reply); + assert(page.find("Reader ") == 0); + const size_t body = page.find('\n') + 1; + const size_t continuation = page.find("\nNext: "); + reconstructed += page.substr(body, continuation == std::string::npos + ? continuation : continuation - body); + if (continuation == std::string::npos) break; + const std::string command_next = page.substr(continuation + 7); + assert(command_next != next); + next = command_next; + } + assert(reconstructed == expected); + } +} + int main(int argc, char**) { Reference ref = {}; for (const char* cmd : {"get reader 3:16", " GET\treader 3:16 \t", "get reader 03:16"}) { @@ -39,6 +100,7 @@ int main(int argc, char**) { snprintf(cmd, sizeof(cmd), "get reader %u:%u", chapter, verse); Stream output; assert(mesh::handleReaderCommand(cmd, output)); + drain(output); const std::string expected = "\r\n" + std::string(text) + "\r\n"; assert(output.text.find(expected) != std::string::npos); for (const unsigned char* c = reinterpret_cast(text); *c; ++c) @@ -83,6 +145,47 @@ int main(int argc, char**) { } } assert(count == 879); + for (unsigned chapter = 1; chapter <= 21; ++chapter) + checkSelection(chapter, 1, kChapterVerses[chapter - 1], true); + checkSelection(2, 2, 12, false); + checkSelection(21, 16, 18, false); // Includes a long, multi-page verse. + checkSelection(6, 1, 71, false); // Larger than the real 4 KiB USB queue. + checkSelection(2, 25, 25, false); + Stream cancelled, unrelated; + assert(mesh::handleReaderCommand("get reader 6", cancelled)); + mesh::serviceReaderCommand(); + mesh::cancelReaderCommand(&unrelated); + assert(mesh::readerCommandActive(&cancelled)); + mesh::cancelReaderCommand(&cancelled); + const std::string before_cancel = cancelled.text; + mesh::serviceReaderCommand(); + assert(cancelled.text == before_cancel); + assert(mesh::handleReaderCommand("get reader 6", cancelled)); + assert(!mesh::handleReaderCommand("get name", unrelated)); + assert(!mesh::readerCommandActive()); + // TCP bypasses its blocking Stream::write using one socket attempt. Keep + // the original owner for session cancellation, and retry zero/short writes. + Stream socket_owner, socket_bytes, reference; + assert(mesh::handleReaderCommand("get reader 2:2-12", reference, true)); + drain(reference); + static Stream* callback_bytes = nullptr; + callback_bytes = &socket_bytes; + auto try_write = [](Stream& owner, const uint8_t* data, size_t size) -> size_t { + assert(owner.writes == 0); + return callback_bytes->write(data, size); + }; + assert(mesh::handleReaderCommand("get reader 2:2-12", socket_owner, true, try_write)); + assert(mesh::readerCommandActive(&socket_owner)); + for (unsigned tick = 0; mesh::readerCommandActive(); ++tick) { + assert(tick < 20000); + socket_bytes.max_write = 5; + socket_bytes.credit = tick % 3 == 0 ? 0 : 9; + mesh::serviceReaderCommand(); + } + assert(socket_owner.writes == 0 && socket_bytes.text == reference.text); + assert(mesh::handleReaderCommand("get reader 6", socket_owner, false, try_write)); + mesh::cancelReaderCommand(&socket_owner); + assert(!mesh::readerCommandActive()); const char* text; assert(lookup(corpus, {22, 1}, buf.scratch, kBlockSize, text) == LookupResult::Invalid); assert(lookup(corpus, {1, 1}, buf.scratch, kBlockSize - 1, text) == LookupResult::Corrupt); @@ -125,7 +228,11 @@ int main(int argc, char**) { assert(invalid.text.find("ERROR") != std::string::npos); for (const char* cmd : {"get reader", "get reader 3:16 0", "get reader 3:16 -1", "get reader 3:16 100", "get reader 3:16 99999999999", "get reader 3:16 1 junk", - "get reader 3:16 1:2", "get reader 3:16 1\x1b", "get reader 22:1"}) { + "get reader 3:16 1:2", "get reader 3:16 1\x1b", "get reader 22:1", + "get reader 0", "get reader 22", "get reader 2:12-2", "get reader 2:0-12", + "get reader 2:2-26", "get reader 2:2-99999999", "get reader 2:2-", + "get reader 2:2-12-14", "get reader 2:2-12junk", "get reader 2x", + "get reader 2:2--12", "get reader 2:2-3:12"}) { char reply[160]; assert(mesh::handleReaderReply(cmd, reply, sizeof(reply))); assert(std::string(reply).find("ERROR: use get reader") == 0); diff --git a/test/test_companion_reader.py b/test/test_companion_reader.py index a07faf88..50ac9347 100644 --- a/test/test_companion_reader.py +++ b/test/test_companion_reader.py @@ -111,6 +111,52 @@ class CompanionReaderTest(unittest.TestCase): self.assertNotIn("readerData", result) self.assertNotIn("handleReaderCommand", result) + def test_tcp_reader_write_returns_immediately_under_socket_backpressure(self): + compiler = shutil.which("g++") + if not compiler: + self.skipTest("host G++ required") + source = (ROOT / "examples/companion_radio/main.cpp").read_text() + callback = extract_braced(source, "static size_t ota_console_try_reader_write(") + harness = r''' +#include +#include +#include +#include +class WiFiClient : public Stream { + int socket_; +public: + explicit WiFiClient(int socket) : socket_(socket) {} + int fd() const { return socket_; } +}; +@CALLBACK@ +int main() { + alarm(3); // A blocking write to a full socket fails the test, never hangs CI. + int sockets[2]; + assert(socketpair(AF_UNIX, SOCK_STREAM, 0, sockets) == 0); + int capacity = 1024; + assert(setsockopt(sockets[0], SOL_SOCKET, SO_SNDBUF, &capacity, sizeof(capacity)) == 0); + uint8_t data[96] = {}; + while (send(sockets[0], data, sizeof(data), MSG_DONTWAIT) > 0) {} + WiFiClient client(sockets[0]), invalid(-1); + for (int i = 0; i < 100; ++i) + assert(ota_console_try_reader_write(client, data, sizeof(data)) == 0); + assert(client.writes == 0); // The blocking Stream path must never be used. + uint8_t drained[8192]; + assert(recv(sockets[1], drained, sizeof(drained), 0) > 0); + assert(ota_console_try_reader_write(client, data, sizeof(data)) > 0); + assert(ota_console_try_reader_write(invalid, data, sizeof(data)) == 0); + close(sockets[0]); + close(sockets[1]); +} +'''.replace("@CALLBACK@", callback) + with tempfile.TemporaryDirectory(prefix="mesh-reader-socket-") as directory: + cpp = Path(directory) / "socket.cpp" + binary = Path(directory) / "socket-test" + cpp.write_text(harness) + self.run_checked([compiler, "-std=c++17", "-Wall", "-Wextra", "-Werror", + "-I" + str(FIXTURE), str(cpp), "-o", str(binary)]) + self.run_checked([str(binary)]) + def test_terminal_dispatch_is_separate_from_bounded_api_replies(self): source = (ROOT / "examples/companion_radio/MyMesh.cpp").read_text(encoding="utf-8") start = source.index("void MyMesh::handleTerminalCommand(") @@ -119,7 +165,7 @@ class CompanionReaderTest(unittest.TestCase): self.assertEqual(source.count("mesh::handleReaderCommand("), 2) self.assertLess(terminal.index("mesh::handleReaderCommand("), terminal.index("char local_reply[160]")) - self.assertIn("get reader : (World English Bible, offline)", terminal) + self.assertIn("get reader [:[-]] [page] (World English Bible, offline)", terminal) self.assertEqual(terminal.count("#if COMPANION_FEATURE_READER"), 2) rescue = extract_braced(source, "void MyMesh::checkCLIRescueCmd()") self.assertLess(rescue.index("mesh::handleReaderCommand(cli_command, output)"), diff --git a/test/test_ed25519_compact.py b/test/test_ed25519_compact.py index 1f910ef0..9950368a 100644 --- a/test/test_ed25519_compact.py +++ b/test/test_ed25519_compact.py @@ -34,6 +34,69 @@ class SHA512Context(ctypes.Structure): class Ed25519CompactTest(unittest.TestCase): + def test_small_flash_table_matches_standard_for_scalars_keys_and_signatures(self): + # Exercise every bit/carry boundary and deterministic random scalars, + # rather than validating only the two RFC keypair examples. + with tempfile.TemporaryDirectory() as directory: + wrapper = Path(directory) / "wrapper.c" + wrapper.write_text('''#include "ge.h" +void public_from_scalar(unsigned char *out, const unsigned char *scalar) { + ge_p3 point; + ge_scalarmult_base(&point, scalar); + ge_p3_tobytes(out, &point); +} +''') + libraries = [] + for label, flags in ( + ("standard", []), + ("small_sha512", ["-DED25519_SMALL_BASE=1", "-DED25519_COMPACT_SHA512=1"]), + ("small_all", ["-DED25519_SMALL_BASE=1", "-DED25519_COMPACT_SHA512=1", + "-DED25519_COMPACT_SCALAR=1"]), + ): + path = Path(directory) / (label + ".so") + subprocess.run(["cc", "-std=c99", "-O2", "-Wall", "-Wextra", "-Werror", + "-shared", "-fPIC", "-DED25519_NO_SEED=1", *flags, + "-I" + str(ED25519), str(wrapper), + *(str(source) for source in SOURCES), + str(ED25519 / "key_exchange.c"), "-o", str(path)], check=True) + library = ctypes.CDLL(str(path)) + library.public_from_scalar.argtypes = [ctypes.c_void_p] * 2 + library.ed25519_create_keypair.argtypes = [ctypes.c_void_p] * 3 + library.ed25519_sign.argtypes = [ctypes.c_void_p, ctypes.c_void_p, + ctypes.c_size_t, ctypes.c_void_p, ctypes.c_void_p] + library.ed25519_key_exchange.argtypes = [ctypes.c_void_p] * 3 + libraries.append(library) + rng = random.Random(0xED960) + scalars = [0, (1 << 255) - 1] + scalars += [value for bit in range(255) + for value in ((1 << bit) - 1, 1 << bit, (1 << bit) + 1)] + scalars += [rng.getrandbits(255) for _ in range(1024)] + for scalar in scalars: + outputs = [] + for library in libraries: + out = ctypes.create_string_buffer(32) + library.public_from_scalar(out, scalar.to_bytes(32, "little")) + outputs.append(out.raw) + for output in outputs[1:]: + self.assertEqual(outputs[0], output, "scalar=%x" % scalar) + for iteration in range(128): + seed = rng.getrandbits(256).to_bytes(32, "little") + peer_seed = rng.getrandbits(256).to_bytes(32, "little") + length = (0, 1, 111, 112, 127, 128, 129, 255, 256, 4097)[iteration % 10] + message = bytes(rng.randrange(256) for _ in range(length)) + outputs = [] + for library in libraries: + pub, prv = ctypes.create_string_buffer(32), ctypes.create_string_buffer(64) + peer, peer_prv = ctypes.create_string_buffer(32), ctypes.create_string_buffer(64) + sig, secret = ctypes.create_string_buffer(64), ctypes.create_string_buffer(32) + library.ed25519_create_keypair(pub, prv, seed) + library.ed25519_create_keypair(peer, peer_prv, peer_seed) + library.ed25519_sign(sig, message, len(message), pub, prv) + library.ed25519_key_exchange(secret, peer, prv) + outputs.append((pub.raw, prv.raw, sig.raw, secret.raw)) + for output in outputs[1:]: + self.assertEqual(outputs[0], output) + def build_sha512_library(self, temp_dir, compact): library = Path(temp_dir) / f"sha512-{int(compact)}.so" flags = ["-DED25519_COMPACT_SHA512=1"] if compact else [] @@ -113,6 +176,10 @@ class Ed25519CompactTest(unittest.TestCase): for label, extra_flags in ( ("standard", []), ("compact", ["-DED25519_COMPACT_BASE=1"]), + ("small", ["-DED25519_SMALL_BASE=1"]), + ("small_sha512", ["-DED25519_SMALL_BASE=1", "-DED25519_COMPACT_SHA512=1"]), + ("small_all", ["-DED25519_SMALL_BASE=1", "-DED25519_COMPACT_SHA512=1", + "-DED25519_COMPACT_SCALAR=1"]), ("compact_sha512", ["-DED25519_COMPACT_SHA512=1"]), ("compact_base_sha512", ["-DED25519_COMPACT_BASE=1", "-DED25519_COMPACT_SHA512=1"]), diff --git a/test/test_nrf52_flash_trim.py b/test/test_nrf52_flash_trim.py index e078bf12..402f3e5d 100644 --- a/test/test_nrf52_flash_trim.py +++ b/test/test_nrf52_flash_trim.py @@ -160,6 +160,62 @@ class Nrf52FlashTrimTest(unittest.TestCase): str(fixture / "test_domain_trim.c"), "-o", str(executable)], check=True) subprocess.run([str(executable)], check=True) + def test_companion_keeps_small_crypto_stack_fonts_reader_and_usb_transports(self): + for platform, mcu, expected in (("nordicnrf52", "nrf52840", True), + ("nordicnrf52", "nrf52832", True), + ("espressif32", "esp32s3", False)): + with self.subTest(platform=platform, mcu=mcu): + defines = TRIM.companion_trim_policy(platform, mcu, + "+<*> +<../examples/companion_radio/*.cpp>") + self.assertEqual(bool(defines), expected) + if not expected: + continue + self.assertEqual(defines["MESH_NRF52_COMPANION_TRIM"], 1) + self.assertEqual(defines["ED25519_SMALL_BASE"], 1) + self.assertEqual(defines["ED25519_COMPACT_SHA512"], 1) + self.assertEqual(defines["ED25519_COMPACT_SCALAR"], 1) + self.assertEqual(defines["CFG_TUD_MSC"], 0) + self.assertEqual(defines["CFG_TUD_HID"], 0) + for name in ("MESH_NRF52_FLASH_TRIM", "ED25519_COMPACT_BASE", + "CFG_TUD_CDC", "CFG_TUD_DFU", + "CFG_TUD_DFU_RUNTIME", "COMPANION_FEATURE_READER"): + self.assertNotIn(name, defines) + self.assertFalse(any(name.startswith("CFG_TUH") for name in defines)) + + def test_companion_role_requires_selected_source_and_honors_explicit_options(self): + for sources in ("+<*>", "+<../examples/simple_secure_chat>", + "+<../examples/companion_radio> -<../examples/companion_radio>", + "+<../examples/companion_radio> +<../examples/simple_repeater>"): + self.assertEqual(TRIM.companion_trim_policy("nordicnrf52", "nrf52840", sources), {}) + for flags in ("-DMESH_NRF52_COMPANION_TRIM=0", "-UMESH_NRF52_COMPANION_TRIM", + "-DMESH_NRF52_FLASH_TRIM=0"): + self.assertEqual(TRIM.companion_trim_policy("nordicnrf52", "nrf52840", + "+<../examples/companion_radio>", flags), {}) + result = TRIM.companion_trim_policy("nordicnrf52", "nrf52840", + "+<../examples/companion_radio>", "-DCFG_TUD_MSC=1 -USSD1306_NO_SPLASH") + self.assertNotIn("CFG_TUD_MSC", result) + self.assertNotIn("SSD1306_NO_SPLASH", result) + result = TRIM.companion_trim_policy("nordicnrf52", "nrf52840", + "+<../examples/companion_radio>", + "-DED25519_SMALL_BASE=0 -UED25519_COMPACT_SHA512 -UED25519_COMPACT_SCALAR") + for name in ("ED25519_SMALL_BASE", "ED25519_COMPACT_SHA512", "ED25519_COMPACT_SCALAR"): + self.assertNotIn(name, result) + for value in (0, 1): + result = TRIM.companion_trim_policy("nordicnrf52", "nrf52840", + "+<../examples/companion_radio>", "-DED25519_COMPACT_BASE=%d" % value) + self.assertNotIn("ED25519_SMALL_BASE", result) + + @unittest.skipUnless(shutil.which("cc"), "a native C compiler is required") + def test_companion_uses_same_p256_domain_wrapper(self): + fixture = ROOT / "test/fixtures/nrf52_flash_trim" + with tempfile.TemporaryDirectory() as temp_dir: + executable = Path(temp_dir) / "cc310-companion-test" + subprocess.run(["cc", "-std=c99", "-Wall", "-Wextra", "-Werror", + "-DMESH_NRF52_COMPANION_TRIM=1", "-I" + str(fixture), + str(ROOT / "src/helpers/nrf52/Cc310DomainTrim.c"), + str(fixture / "test_domain_trim.c"), "-o", str(executable)], check=True) + subprocess.run([str(executable)], check=True) + if __name__ == "__main__": unittest.main() diff --git a/tools/bible/README.md b/tools/bible/README.md index 1e8ecc1e..9e3b97a3 100644 --- a/tools/bible/README.md +++ b/tools/bible/README.md @@ -5,13 +5,23 @@ include an embedded 879-verse corpus from the World English Bible: ```text get reader 3:16 +get reader 2 +get reader 2:2-12 ``` -Book/verb case and surrounding spaces are ignored. One chapter:verse reference -is required; ranges, whole chapters and other books are not supported. ASCII -verse text is printed in full, including verses longer than the ordinary -160-byte CLI response. This is a local terminal command, not a LoRa request or -an extension of the framed Companion API. No WiFi, phone, API key, filesystem +Verb case and surrounding spaces are ignored. A chapter number selects the +whole chapter; `chapter:first-last` selects an inclusive range within it. +Other books and ranges crossing chapter boundaries are not supported. +USB/TCP/browser text terminals print each selected verse in full. The output +advances by at most 96 bytes per mesh loop and resumes after short or zero-byte +writes, so a chapter larger than the USB queue does not get truncated. A new +terminal command or a session disconnect cancels unfinished output. + +The app's local USB/BLE CLI uses the existing bounded reply frames. Long verses +and selections include a `Next: get reader ...` continuation, preserving the +last verse of a range. An optional page number refers to the first verse in +that command: `get reader 21:17 2`. No radio request can access the reader. +No WiFi, phone, API key, filesystem space or bootloader update is needed. Reduced/ordinary Companion and other roles omit the corpus. `COMPANION_FEATURE_READER=0` excludes it from a Full build. diff --git a/variants/rak4631/platformio.ini b/variants/rak4631/platformio.ini index aaee20df..24d7bd8c 100644 --- a/variants/rak4631/platformio.ini +++ b/variants/rak4631/platformio.ini @@ -416,12 +416,6 @@ lib_deps = extends = env:RAK_4631_companion_radio_usb build_flags = ${env:RAK_4631_companion_radio_usb.build_flags} - ; Meshtext needs flash headroom without changing the ExtraFS boundary or - ; removing GPS. These omit unused automatic NMEA caches/debug strings and - ; the vendor bitmap that SSD1306Display clears before its first transfer. - -D SFE_UBLOX_DISABLE_AUTO_NMEA=1 - -D SFE_UBLOX_REDUCED_PROG_MEM=1 - -D SSD1306_NO_SPLASH=1 -D ETHERNET_ENABLED=1 -D ETHERNET_USE_RAK13800 -D ETHERNET_CLASS=RAK13800EthernetInterface