Consolidate companion components

This commit is contained in:
mikecarper committed 2026-09-17 13:41:39 -07:00
1 parent aab8f5038a
commit ebee99af81
32 files changed
+286 -287

No files matched your search

+25 -26
View File
@@ -1,10 +1,10 @@
# Offline John lookup
# Offline Reader lookup
ESP32 and nRF52 **Full Companion** USB terminals (and the ESP32 TCP terminal)
include the 879 verses of John from the World English Bible:
include an embedded 879-verse corpus from the World English Bible:
```text
get John 3:16
get reader 3:16
```
Book/verb case and surrounding spaces are ignored. One chapter:verse reference
@@ -13,14 +13,14 @@ verse text is printed in full, including verses longer than the ordinary
160-byte CLI response. This is a local terminal command, not a LoRa request or
an extension of the framed Companion API. No WiFi, phone, API key, filesystem
space or bootloader update is needed. Reduced/ordinary Companion and other
roles omit the corpus. `COMPANION_FEATURE_JOHN=0` excludes it from a Full build.
roles omit the corpus. `COMPANION_FEATURE_READER=0` excludes it from a Full build.
## On-device reader
## On-device Reader
Full Companions using the `ui-new` display (including the V4) also offer a
reader from the radio-settings page:
Reader from the radio-settings page:
- Long press opens John 1:1 initially, or resumes the saved position.
- Long press opens at the first reference, or resumes the saved position.
- Single click advances one screenful, then to the next verse.
- Double click goes back one screenful; at a verse boundary it shows the
previous verse's last screenful.
@@ -50,8 +50,7 @@ Squeezed Regular 6 on panels at least 128x64 (or 64x128 when rotated), and
5px Picopixel on smaller panels. The reference and part counter retain their
normal header font, stacking on narrow rotated screens to avoid overlap. On
the V4's 128x64 display, five complete 8-pixel-spaced body rows fit above the
navigation hint, and John
3:16 fits on one screen. Wrapping and pagination use the selected font's
navigation hint, and reference 3:16 fits on one screen. Wrapping and pagination use the selected font's
actual character widths and line height. Any panel with either dimension at
least 160px uses its normal font, including rotated 80x160 panels. Larger display classes, and builds with
`UI_SMALL_MESSAGE_FONT=0`, retain the regular body font. Existing bookmarks
@@ -70,18 +69,18 @@ It is checkpointed after two seconds without navigation and flushed on exit,
screen sleep, or UI shutdown. Writes occur only for changed positions; failures
remain pending for retry. Abrupt power loss during the two-second debounce may
lose that last movement. Returning to the very first screen clears the bookmark
instead of saving a John 1:1 record. Opening and closing there creates no file.
instead of saving a first-reference record. Opening and closing there creates no file.
The bookmark uses `/john.pos` on the primary filesystem, with a verified `.tmp`
The bookmark uses `/reader.pos` on the primary filesystem, with a verified `.tmp`
replacement and `.bak` fallback for interrupted saves. It does not change the
radio-preference layout, contacts, channels, or keys. A malformed bookmark
falls back to the previous valid record or John 1:1; no filesystem is formatted.
falls back to the previous valid record or the first reference; no filesystem is formatted.
The reader allocates only a small cursor object on first use, reusing the
same flash corpus and on-demand 2 KiB decode scratch as the terminal.
## Source and reproduction
`john-web.json` contains only John, copied from eBible.org's verse-per-line
`reader-web.json` contains the embedded corpus, copied from eBible.org's verse-per-line
**engwebp** source (World English Bible, American English Protestant edition,
a subset of the World English Bible Updated). The JSON records the archive URL,
source date and SHA-256 hashes. The verse-per-line export omits footnotes and
@@ -102,14 +101,14 @@ moving upstream text. See the [official download inventory](https://ebible.org/f
Regenerate or verify the firmware header:
```sh
python tools/bible/pack_john.py
python tools/bible/pack_john.py --check
python tools/bible/pack_john.py --compare
python test/test_companion_john.py
python tools/bible/pack_reader.py
python tools/bible/pack_reader.py --check
python tools/bible/pack_reader.py --compare
python test/test_companion_reader.py
```
To deliberately update the source, download `engwebp_vpl.zip` from the recorded
URL, then run `python tools/bible/import_web_john.py /path/to/engwebp_vpl.zip`.
URL, then run `python tools/bible/import_web_reader.py /path/to/engwebp_vpl.zip`.
Review the plaintext changes and regenerate the header. This importer does
not download content automatically or add other books to the firmware.
@@ -155,7 +154,7 @@ on RAK3401 and 4.7 KiB on V4, including the main-loop/UI frames but before
framework/task and interrupt overhead. The ESP32-S3 V4 terminal's corresponding
decode path is approximately 4.6 KiB.
This is a compiler-derived estimate, not a hardware stack high-water reading.
Ordinary non-John commands never enter the decode-buffer function. Its object
Ordinary non-reader commands never enter the decode-buffer function. Its object
file has zero `.data`/`.bss`; the screen reader adds only its small UI state.
Tests compare all 879 decoded verses and terminal responses with an independently
@@ -185,7 +184,7 @@ The small-font C++11 reader fixture also passes AddressSanitizer and
UndefinedBehaviorSanitizer, the 22 existing nRF52 ExtraFS contract tests pass,
and the separate shared-font check matches all 95 printable ASCII glyphs
against Adafruit Picopixel. All 13 memory-policy tests pass in WSL. The following
USA Cascade Full builds include the local ASCII John reader changes on top of
USA Cascade Full builds include the local ASCII Reader changes on top of
`03e42830`; both pass firmware-size, runtime-RAM and capability checks:
| Build | Flash bytes | Static RAM bytes | Runtime RAM available / required |
@@ -205,7 +204,7 @@ readings; those earlier builds were not flashed to hardware.
### Adaptive 6px/5px update, 2026-09-08
The reader now shares `SmallMessageText` with messages on base `8f5614c9`.
All ten John host tests pass on both Windows and Linux. The actual reader
All ten reader host tests pass on both Windows and Linux. The actual reader
also passes AddressSanitizer and UndefinedBehaviorSanitizer across all ten
screen geometries above, including resizing and bookmark migration. The
separate shared-font test checks all 95 glyphs of both fonts against their
@@ -214,7 +213,7 @@ upstream fixtures; the four T096 footer/profile checks also pass.
The T096 Full FEM-on USA Cascade build uses 573,384 flash bytes and 135,452
static RAM bytes, with 100,060 runtime bytes available against 75,298 required.
Its ten capability markers pass. The application-only UF2 was installed on
the T096, then the full running version and John 1:1, 3:16 and 21:25 were
the T096, then the full running version and references 1:1, 3:16 and 21:25 were
verified over USB. The API snapshots confirm unchanged identity, contacts,
channels, self settings, auto-add, tuning and custom variables. No bootloader
or filesystem was erased. Physical on-screen readability and button behavior
@@ -225,7 +224,7 @@ with 263,936 internal runtime bytes available against 175,872 required. All 13
capability markers pass. The nonmerged application was installed in both
existing V4 app slots; their hashes were verified, and the bootloader, partition
table, NVS, OTA selector, SPIFFS and coredump regions compare byte-for-byte with
the pre-update full-flash backup. The full running version and John 1:1, 3:16,
the pre-update full-flash backup. The full running version and references 1:1, 3:16,
21:17 (the longest verse) and 21:25 were verified over USB. Pre/post API snapshots
match for identity, contacts, channels, self settings, auto-add, tuning, custom
variables, Bluetooth name and default flood scope.
@@ -235,7 +234,7 @@ same 135,452 static RAM bytes and runtime budget as the scaled build. The actual
driver header/drawing-method fixture passes in native and legacy modes, with
both constructor variants; it checks complete pixel coverage and all 95 native
6px glyphs. Linux additionally runs that fixture with address/undefined-behavior
sanitizers. The John tests include 160x80 navigation and bookmark migration.
sanitizers. The reader tests include 160x80 navigation and bookmark migration.
The adaptive reader's compiler-reported process frame is 2,504 bytes on T096
and 2,512 bytes on V4; these are not total stack high-water measurements.
@@ -254,7 +253,7 @@ the compact-panel footer policy. Normal-font line heights drive message and
radio spacing; Indicator render profiles keep their existing geometry.
Validation: 35 Python-host tests (including actual-driver recording fixtures,
all 879 John verses across 17 geometries, shared fonts, wake and pairing),
all 879 corpus verses across 17 geometries, shared fonts, wake and pairing),
30 native dashboard tests, and nine message-history/layout tests pass. Linux
pixel/transfer/font fixtures use address/undefined-behavior sanitizers. The
T096 normal-font routing check was rerun after its final assertion was added.
@@ -263,7 +262,7 @@ The T096 Full FEM-on USA Cascade build on `0058f721` plus these local changes
uses 573,480 flash bytes and 135,452 static RAM bytes. Runtime capacity is
100,060 bytes against 75,298 required; all ten capability checks pass. Its
application-only UF2 was installed, and USB reports
`v1.17.1.5-native-normal-0058f721`. John 1:1, 3:16 and 21:25 work. API snapshots
`v1.17.1.5-native-normal-0058f721`. References 1:1, 3:16 and 21:25 work. API snapshots
match for identity/private key, contacts, channels, self settings, auto-add,
tuning and custom variables. The bootloader and storage were not flashed.
Other panel families were host-tested, not physically flashed or visually
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Import only John from eBible.org's public-domain engwebp VPL download."""
"""Import the source for the offline Reader from eBible.org's engwebp VPL download."""
import argparse
import hashlib
import json
@@ -7,7 +7,7 @@ from pathlib import Path
import re
import zipfile
from pack_john import SOURCE, pack, references
from pack_reader import SOURCE, pack, references
SOURCE_URL = "https://ebible.org/Scriptures/engwebp_vpl.zip"
MEMBER = "engwebp_vpl.txt"
@@ -24,10 +24,10 @@ def import_archive(path):
continue
match = re.fullmatch(r"JOH ([0-9]+:[0-9]+) (.+)", line)
if not match or match[1] in verses:
raise ValueError("invalid or duplicate John verse in source")
raise ValueError("invalid or duplicate Reader verse in source")
verses[match[1]] = match[2]
if set(verses) != set(references()):
raise ValueError("the source must contain exactly the 879 verses of John")
raise ValueError("the source must contain exactly the 879 Reader verses")
document = {
"translation": "WEB",
"edition": "World English Bible, American English, Protestant Edition (engwebp)",
@@ -52,7 +52,7 @@ def main():
document = import_archive(args.archive)
args.output.write_text(json.dumps(document, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8", newline="\n")
print(f"Imported {len(document['verses'])} verses of John; source {document['source_date']}")
print(f"Imported {len(document['verses'])} Reader verses; source {document['source_date']}")
if __name__ == "__main__":
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Pack the pinned public-domain WEB John source into independent DEFLATE blocks."""
"""Pack the pinned public-domain source for the offline Reader into DEFLATE blocks."""
import argparse
import hashlib
import json
@@ -8,8 +8,8 @@ import unicodedata
import zlib
ROOT = Path(__file__).resolve().parents[2]
SOURCE = Path(__file__).with_name("john-web.json")
HEADER = ROOT / "src/helpers/bible/JohnData.generated.h"
SOURCE = Path(__file__).with_name("reader-web.json")
HEADER = ROOT / "src/helpers/bible/ReaderData.generated.h"
BLOCK_SIZE = 2048
ASCII_TYPOGRAPHY = str.maketrans({
"\u00a0": " ", "\u2013": "-", "\u2014": "-",
@@ -42,7 +42,7 @@ def ascii_verse(value, limit):
try:
encoded = value.translate(ASCII_TYPOGRAPHY).encode("ascii")
except UnicodeEncodeError as error:
raise ValueError("unsupported non-ASCII character in John source") from error
raise ValueError("unsupported non-ASCII character in Reader source") from error
if len(encoded) > limit:
raise ValueError(f"text exceeds {limit} ASCII bytes")
return encoded
@@ -55,7 +55,7 @@ def pack(document, block_size=BLOCK_SIZE):
checked_text(document["attribution"], 256)
verses = document["verses"]
if not isinstance(verses, dict) or set(verses) != set(references()):
raise ValueError("source must explicitly include all 879 John references")
raise ValueError("source must explicitly include all 879 Reader references")
blocks, starts, current = [], [0], bytearray()
for index, reference in enumerate(references()):
verse = verses[reference]
@@ -82,24 +82,24 @@ def render(document):
packed, blocks = pack(document)
digest = hashlib.sha256(json.dumps(document, ensure_ascii=False, sort_keys=True,
separators=(",", ":")).encode()).hexdigest()
lines = ["// Generated by tools/bible/pack_john.py; do not edit.",
"// World English Bible (engwebp), John. Public domain; see tools/bible/README.md.",
lines = ["// Generated by tools/bible/pack_reader.py; do not edit.",
"// Offline Reader corpus. Public domain; see tools/bible/README.md.",
"// Firmware typography: ASCII quotes, apostrophes, dashes and spaces; wording unchanged.",
f"// Canonical source SHA-256: {digest}",
"#pragma once", '#include "JohnLookup.h"',
"#pragma once", '#include "ReaderLookup.h"',
"namespace mesh { namespace bible { namespace generated {",
f'static_assert(kBlockSize == {BLOCK_SIZE}, "Regenerate John data after changing block size");',
'static_assert(sizeof(BlockIndex) == 12, "John flash-size accounting expects a 12-byte index");',
"static const uint8_t johnData[] = {"]
f'static_assert(kBlockSize == {BLOCK_SIZE}, "Regenerate Reader data after changing block size");',
'static_assert(sizeof(BlockIndex) == 12, "Reader flash-size accounting expects a 12-byte index");',
"static const uint8_t readerData[] = {"]
for start in range(0, len(packed), 20):
lines.append(" " + ",".join(f"0x{v:02x}" for v in packed[start:start + 20]) + ",")
lines += ["};", "static const BlockIndex johnBlocks[] = {"]
lines += ["};", "static const BlockIndex readerBlocks[] = {"]
lines += [" {%d, %d, %d, %d}," % block for block in blocks]
lines += ["};", "static const Corpus johnCorpus = {",
lines += ["};", "static const Corpus readerCorpus = {",
f" {json.dumps(document['translation'], ensure_ascii=False)},",
f" {json.dumps(document['attribution'], ensure_ascii=False)},",
" johnData, sizeof(johnData), johnBlocks,",
" sizeof(johnBlocks) / sizeof(johnBlocks[0]), kVerseCount", "};", "}}}", ""]
" readerData, sizeof(readerData), readerBlocks,",
" sizeof(readerBlocks) / sizeof(readerBlocks[0]), kVerseCount", "};", "}}}", ""]
return "\n".join(lines)
@@ -124,14 +124,14 @@ def main():
header = render(document)
if args.check:
if args.output.read_text(encoding="utf-8") != header:
raise SystemExit("John header differs; regenerate with tools/bible/pack_john.py")
raise SystemExit("Reader header differs; regenerate with tools/bible/pack_reader.py")
else:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(header, encoding="utf-8", newline="\n")
packed, blocks = pack(document)
plain = sum(block[2] for block in blocks)
tables = len(blocks) * 12
print(f"John: {len(references())} verses, {len(blocks)} blocks; "
print(f"Reader: {len(references())} verses, {len(blocks)} blocks; "
f"{plain} plain bytes -> {len(packed)} DEFLATE + {tables} index bytes "
f"({100 * (1 - (len(packed) + tables) / plain):.1f}% smaller incl. indexes)")
File renamed without changes.