Live feed: the lane is about the packet, whatever the packet is about

The NODE lane suited the two classes that name a node and left every
other one reading a dash. On this mesh that is half the traffic: the
direct messages, requests, responses, returned paths, acks and traces
that relay past us naming no origin. So the lane asks each class its own
question instead, and the header says SUBJECT — the word that stays true
down the whole column.

An advert or a telemetry frame is still about its node. A channel text is
about its channel, named by the key whose MAC confirms the frame — never
by the fingerprint alone, which 256 buckets make no proof at all; a
channel we hold no key for reads as that fingerprint and says so. A
direct message, request, response, returned path or anonymous request is
about the two nodes it travels between, drawn sender → recipient on the
hop-sequence widget, so endpoints read exactly as route hops do: resolved
names, key-derived hues, our own node white the moment a frame is
addressed to us. Too wide, and the sender drops to its hash — the frame
is addressed, so the addressee is the half worth the cells. An ack is
about the message it answers, a trace about its tag. And an unsigned
channel post is filed under its channel rather than the bare word
"channel", which named nothing.

Half of that was unreadable before because nobody had opened the frame.
The companion hands every overheard packet to the library, which decodes
the body for exactly two classes — advert and channel text — and leaves
the rest as bytes. Those bytes are not opaque: MeshCore addresses a frame
in the first few of them, and core/frames.py now reads them, class by
class, against the layouts the protocol's own docs/payloads.md publishes.
The trace tag is the little-endian uint32 the firmware writes, so it
reads as the same number a trace reply reports; an ack's checksum keeps
wire order, the form the companion reports its own delivery acks in. Two
checks guard every decode, because a wrong slice yields not an error but
a plausible-looking hash: the frame's payload version must be v1 (the
header reserves two bits for a v2 that widens both hash and MAC), and the
body must be long enough for the layout its class promises. Either
failing yields nothing at all.

The feed opens seeded from history, so any of this that lived only in a
live event's raw payload would leave the opening screen blank — the same
problem the channel-text crypto trio had. Same answer: three columns
(dest/src/tag), written on capture and restored to the very keys the
event used, so a replayed frame and a fresh one read identically all the
way up. Older rows carry NULL until re-heard.

The viewer gained the matching rows: to, from, tag, acks, and a channel
for a datagram — whose body is not text, but whose MAC still names it.

Verified against the MeshCore protocol documentation and firmware source,
then corroborated on the air: every frame captured off the radio has a
body length that decomposes exactly into its documented layout, an
anonymous request's 51 bytes (1 + 32 + 2 + one AES block) included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jean-Pierre MartineauandClaude Opus 5 committed 2026-07-28 19:11:07 -04:00
1 parent 64ac7dc2ed
commit 01ca242c98
11 files changed
+897 -110

No files matched your search

+137
View File
@@ -0,0 +1,137 @@
"""Frame-body decoding: what an overheard packet is addressed to.
The companion's RX log hands us every frame's header parsed and its body raw; these
cover the layouts :mod:`meshterm.core.frames` reads out of that body, the length checks
that keep a truncated frame from inventing addressing, and the round trip through
storage — the feed opens on stored history, so what a live frame said about itself has to
survive being written down and read back.
"""
from __future__ import annotations
from datetime import timedelta
from pathlib import Path
from meshterm.core.connection import packet_observation_from_event
from meshterm.core.frames import frame_addressing
from meshterm.core.models import Observation, utcnow
from meshterm.persistence.repository import Repository
class _Event:
"""A meshcore event stand-in: anything exposing a ``payload`` mapping."""
def __init__(self, **payload) -> None:
self.payload = payload
def _frame(typename: str, body: bytes, **extra) -> dict:
return {"payload_typename": typename, "pkt_payload": body, **extra}
def test_addressed_classes_name_both_ends() -> None:
"""A direct message, request, response or returned path opens with dest then src."""
body = bytes.fromhex("3da1") + b"\xab\xcd" + b"\x00" * 16
for typename in ("TEXT_MSG", "REQ", "RESPONSE", "PATH"):
assert frame_addressing(_frame(typename, body)) == {
"dest_hash": "3d", "src_hash": "a1",
}
def test_anonymous_request_carries_its_senders_whole_key() -> None:
"""No shared secret yet, so the sender identifies itself in full — 32 bytes of key."""
key = bytes(range(32))
decoded = frame_addressing(_frame("ANON_REQ", b"\x3d" + key + b"\xab\xcd" + b"\x00" * 16))
assert decoded == {"dest_hash": "3d", "src_key": key.hex()}
def test_tokened_classes_name_their_token() -> None:
"""An ack carries the checksum of what it answers; a trace carries its tag.
The ack's checksum keeps wire order — the form the companion reports its own delivery
acks in — while the trace's tag is the little-endian ``uint32`` the firmware wrote, so
it reads as the same number a trace reply's ``tag`` field does.
"""
assert frame_addressing(_frame("ACK", bytes.fromhex("9b71e004"))) == {
"ack_crc": "9b71e004",
}
trace = bytes.fromhex("102a3c5f") + bytes.fromhex("deadbeef") + b"\x01"
assert frame_addressing(_frame("TRACE", trace)) == {"trace_tag": "5f3c2a10"}
def test_a_future_payload_version_is_left_undecoded() -> None:
"""v2 widens the hashes and the MAC, so v1's offsets would slice the wrong bytes."""
body = bytes.fromhex("3da1") + b"\xab\xcd" + b"\x00" * 16
assert frame_addressing(_frame("TEXT_MSG", body, payload_ver=0))["dest_hash"] == "3d"
assert frame_addressing(_frame("TEXT_MSG", body, payload_ver=1)) == {}
assert frame_addressing(_frame("TEXT_MSG", body))["dest_hash"] == "3d" # unstated: v1
def test_channel_datagram_is_broken_out_like_a_channel_text() -> None:
"""``GRP_DATA`` shares the channel envelope the library only decodes for ``GRP_TXT``."""
decoded = frame_addressing(_frame("GRP_DATA", b"\xa3\xab\xcd" + bytes(range(16))))
assert decoded == {
"chan_hash": "a3", "cipher_mac": "abcd", "crypted": bytes(range(16)).hex(),
}
def test_a_body_too_short_for_its_layout_decodes_to_nothing() -> None:
"""Better a silent lane than a plausible hash sliced out of the wrong bytes."""
assert frame_addressing(_frame("TEXT_MSG", b"\x3d")) == {}
assert frame_addressing(_frame("ANON_REQ", b"\x3d" + bytes(20))) == {}
assert frame_addressing(_frame("ACK", b"\x9b")) == {}
assert frame_addressing(_frame("TRACE", bytes(6))) == {}
assert frame_addressing(_frame("GRP_DATA", b"\xa3")) == {}
def test_classes_that_address_nothing_stay_silent() -> None:
"""An advert names its node in the header; multipart and control say nothing at all."""
for typename in ("ADVERT", "MULTIPART", "CONTROL", "UNK"):
assert frame_addressing(_frame(typename, bytes(64))) == {}
assert frame_addressing({"payload_typename": "TEXT_MSG"}) == {} # no body at all
assert frame_addressing({"pkt_payload": bytes(20)}) == {} # no class at all
def test_rx_log_events_carry_their_addressing_into_the_observation() -> None:
"""The decode happens once, at the edge, so every reader downstream sees the same keys."""
event = _Event(
payload_typename="TEXT_MSG", pkt_payload=bytes.fromhex("3da1abcd") + bytes(16),
path_len=1, path_hash_size=1, path="3d", snr=6.5,
)
obs = packet_observation_from_event(event)
assert obs is not None and obs.raw is not None
assert obs.raw["dest_hash"] == "3d" and obs.raw["src_hash"] == "a1"
assert obs.node is None # an addressed frame still names no *origin* node
def test_stored_frames_remember_what_they_addressed(tmp_path: Path) -> None:
"""A replayed row reads back exactly as the live frame did — same keys, same values.
The feed opens seeded from history, so addressing that lived only in the raw payload
of a live event would leave every row on the opening screen blank.
"""
repo = Repository(tmp_path / "frames.db")
run = repo.start_run("monitor", {}, None)
frames = {
"TEXT_MSG": {"dest_hash": "3d", "src_hash": "a1"},
"ANON_REQ": {"dest_hash": "3d", "src_key": bytes(range(32)).hex()},
"TRACE": {"trace_tag": "5f3c2a10"},
"ACK": {"ack_crc": "9b71e004"},
"GRP_DATA": {"chan_hash": "a3", "cipher_mac": "abcd", "crypted": "00" * 16},
}
for typename, decoded in frames.items():
repo.record_observation(
run,
Observation(
node=None, kind="packet", path="3d",
raw={"payload_typename": typename, **decoded},
),
)
stored = repo.recent_observations(since=utcnow() - timedelta(hours=2))
read_back = {(o.raw or {}).get("payload_typename"): (o.raw or {}) for o in stored}
assert set(read_back) == set(frames)
for typename, decoded in frames.items():
for key, value in decoded.items():
assert read_back[typename][key] == value, (typename, key)
repo.close()
+108 -15
View File
@@ -24,12 +24,18 @@ class _FakeSession:
self.repaints += 1
def _screen(seed=None, active=True) -> LiveFeedScreen:
#: Nodes the feed's resolver can name, by the hash a frame addresses them with (one byte)
#: and by the wider prefixes an advert carries.
_KNOWN = {"a1b2": "Alice", "3d63": "YUL", "a1": "Alice", "3d": "YUL", "c0": "Us"}
def _screen(seed=None, active=True, **kwargs) -> LiveFeedScreen:
screen = LiveFeedScreen(
session=_FakeSession(),
resolve=lambda h: {"a1b2": "Alice", "3d63": "YUL"}.get(h, ""),
resolve=lambda h: _KNOWN.get(h, ""),
seed=list(seed or []),
hub_active=lambda: active,
**kwargs,
)
screen.note_viewport(48) # the frame records this before every real paint
return screen
@@ -105,30 +111,31 @@ def test_livefeed_rows_leave_the_relay_path_to_the_viewer() -> None:
def test_livefeed_class_lane_names_the_payload_class_once() -> None:
"""A raw frame is filed under what it *is*, and the node lane doesn't repeat it.
"""A raw frame is filed under what it *is*, and the subject lane doesn't repeat it.
``packet`` names only the event family the frame arrived in, so the class lane
reads its parsed payload class — the very label the viewer's card headlines. The
node lane is then free to be about the node, and an origin-less flood says so with
a dash rather than standing the class in a second time.
subject lane is then free to be about the packet's own subject, and a class that
carries nothing to be about says so with a dash rather than standing the class in a
second time.
"""
screen = _screen()
raw = {"payload_typename": "TRACE", "route_typename": "FLOOD"}
raw = {"payload_typename": "MULTIPART", "route_typename": "FLOOD"}
screen.on_event(
MeshEvent.observation_event(
_obs(node="", kind="packet", snr=1.0, path="3d63,a1b2", raw=raw)
)
)
row = _rows(screen, 100)[0]
assert "🎯 trace" in row # the class lane, under the payload class's own icon
assert row.count("trace") == 1 # said once, not once per lane
assert "packet" not in row # …and never as the generic event family
assert "—" in row # the node lane: nobody identified themselves
assert "?" not in row # …but never the useless placeholder
assert "🧩 multipart" in row # the class lane, under the payload class's own icon
assert row.count("multipart") == 1 # said once, not once per lane
assert "packet" not in row # …and never as the generic event family
assert "—" in row # the subject lane: the class is about nothing
assert "?" not in row # …but never the useless placeholder
def test_livefeed_names_a_channel_message_by_its_sender() -> None:
"""A channel message's ``Name:`` prefix names the node lane, not the bare channel."""
"""A channel message's ``Name:`` prefix names the subject lane, not the bare channel."""
screen = _screen()
screen.on_event(
MeshEvent.message_event(Message(text="Alice: hi all", channel=3, is_channel=True))
@@ -138,16 +145,102 @@ def test_livefeed_names_a_channel_message_by_its_sender() -> None:
assert "ch 3" in body # …with the channel kept as the trailing context
def test_livefeed_unsigned_channel_post_is_filed_under_its_channel() -> None:
"""With nobody signing it, a channel message is about the channel — named, once."""
screen = _screen(channel_names={3: "Alerts"})
screen.on_event(
MeshEvent.message_event(Message(text="beep boop", channel=3, is_channel=True))
)
row = _rows(screen, 100)[0]
assert "Alerts" in row # the channel name, not its slot number
assert row.count("Alerts") == 1 # …and the trailing note doesn't say it again
assert "ch 3" not in row
def test_livefeed_channel_frame_is_about_its_channel() -> None:
"""An overheard channel frame is filed under the channel whose key confirms its MAC."""
from Crypto.Hash import HMAC, SHA256
from meshterm.core.channels import channel_hash, derive_secret
secret = derive_secret("Public")
crypted = bytes(range(16))
mac = HMAC.new(secret, digestmod=SHA256)
mac.update(crypted)
raw = {
"payload_typename": "GRP_TXT", "chan_hash": channel_hash(secret),
"cipher_mac": mac.digest()[:2].hex(), "crypted": crypted.hex(),
}
screen = _screen(channels=[("Public", secret)])
screen.on_event(MeshEvent.observation_event(_obs(node="", kind="packet", raw=raw)))
assert "Public" in _rows(screen, 100)[0]
# A channel we hold no key for can only be named by the fingerprint it advertised —
# never by a name, since a fingerprint alone is not proof of which channel it is.
other = _screen(channels=[("Public", secret)])
other.on_event(
MeshEvent.observation_event(_obs(node="", kind="packet", raw={**raw, "chan_hash": "a3"}))
)
row = _rows(other, 100)[0]
assert "hash a3" in row and "Public" not in row
def test_livefeed_addressed_frame_is_about_its_two_ends() -> None:
"""A frame that names a recipient reads as sender → recipient, resolved to names."""
screen = _screen(self_name="Us")
raw = {"payload_typename": "TEXT_MSG", "dest_hash": "c0", "src_hash": "a1"}
screen.on_event(MeshEvent.observation_event(_obs(node="", kind="packet", raw=raw)))
row = _rows(screen, 100)[0]
assert "📩 direct message" in row
assert "Alice → Us" in row # …including us, when a frame is addressed to us
# A pair too wide for the lane spends its cells on the addressee: the sender drops to
# the hash it was named by rather than the recipient being the half cut off.
wide = _screen()
wide._resolve = lambda h: {"a1": "Alice-With-A-Long-Name", "3d": "YUL-Cartierville"}.get(h, "")
wide.on_event(
MeshEvent.observation_event(
_obs(node="", kind="packet",
raw={"payload_typename": "REQ", "dest_hash": "3d", "src_hash": "a1"})
)
)
assert "a1 → YUL-Cartierv" in _rows(wide, 100)[0]
def test_livefeed_tokened_classes_are_about_their_token() -> None:
"""An ack names the message it answers; a trace names its tag. Neither says "packet"."""
screen = _screen()
screen.on_event(
MeshEvent.observation_event(
_obs(node="", kind="packet", age_s=1,
raw={"payload_typename": "TRACE", "trace_tag": "5f3c2a10"})
)
)
screen.on_event(
MeshEvent.observation_event(
_obs(node="", kind="packet",
raw={"payload_typename": "ACK", "ack_crc": "9b71e004"})
)
)
acked, traced = _rows(screen, 100)[:2]
assert "for 9b71e004" in acked
assert "tag 5f3c2a10" in traced
def test_livefeed_column_header_sits_over_the_lanes_it_names() -> None:
"""The header names each lane, and every label lands on the values beneath it."""
screen = _screen(seed=[_obs(node="3d63", kind="telemetry", snr=12.8, rssi=-105.0)])
header, row = _stripped(screen.render_body(72))[1:3]
assert header.split() == ["TIME", "CLASS", "NODE", "SNR", "RSSI"]
assert header.split() == ["TIME", "CLASS", "SUBJECT", "SNR", "RSSI"]
# Measured in display cells, not characters — the class icon is one character wide
# but two cells, so a character index would report every later lane one column early.
assert _col(header, "TIME") == _col(row, "03:")
# The clock is read off the row itself: a literal needle would only match during the
# minute it was written in.
stamp = re.search(r"\d\d:\d\d:\d\d", row)
assert stamp is not None
assert _col(header, "TIME") == _col(row, stamp.group(0))
assert _col(header, "CLASS") == _col(row, "telemetry") - _ICON_LANE
assert _col(header, "NODE") == _col(row, "YUL")
assert _col(header, "SUBJECT") == _col(row, "YUL")
# The two readings right-align their number, so their labels end where the digits do.
assert _col(header, "SNR") + 3 == _col(row, "+12.8") + 5
assert _col(header, "RSSI") + 4 == _col(row, "-105") + 4
+62
View File
@@ -264,3 +264,65 @@ def test_packet_viewer_graph_names_a_known_origin_else_a_question_mark() -> None
nameless = PacketEntry(when=utcnow(), kind="packet", path="3d63")
body2 = _plain(_viewer(nameless).render_body(80))
assert "?" in body2 # an origin-less flood draws a plain "?" endpoint
def test_packet_viewer_lays_out_what_a_frame_addressed() -> None:
"""A frame that names no origin still names its two ends — as rows, not raw hex.
The endpoints come out of the frame body (``meshterm.core.frames``), so the card can
say who a relayed direct message was for even though the class carries no origin node.
"""
entry = PacketEntry(
when=utcnow(), kind="packet", path="3d63",
raw={
"payload_typename": "TEXT_MSG", "route_typename": "FLOOD",
"dest_hash": "c0", "src_hash": "a1",
},
)
viewer = PacketViewer(
[entry], 0, resolve=lambda h: {"c0": "Waymarker", "a1": "Alice"}.get(h, ""),
prefix_bytes=1, self_name="Waymarker",
)
lines = _stripped(viewer.render_body(80))
to_row = next(l for l in lines if l.startswith("to"))
from_row = next(l for l in lines if l.startswith("from"))
assert "Waymarker" in to_row and "c0" in to_row # the name, then the hash it was named by
assert "Alice" in from_row and "a1" in from_row
# …and neither is repeated by the generic raw dump at the foot of the card.
assert _plain(lines).count("dest_hash") == 0
def test_packet_viewer_names_a_tokened_frame_by_its_token() -> None:
"""An ack points at the message it answers; a trace at its own tag."""
ack = PacketEntry(
when=utcnow(), kind="packet", path="3d63",
raw={"payload_typename": "ACK", "ack_crc": "9b71e004"},
)
assert "9b71e004" in _plain(_viewer(ack).render_body(80))
trace = PacketEntry(
when=utcnow(), kind="packet", path="3d63",
raw={"payload_typename": "TRACE", "trace_tag": "5f3c2a10"},
)
body = _plain(_viewer(trace).render_body(80))
assert "tag" in body and "5f3c2a10" in body
def test_packet_viewer_names_a_channel_datagram_by_its_confirmed_channel() -> None:
"""A datagram's body is not text, but the MAC that guards it still names its channel."""
secret = derive_secret("Public")
crypted = bytes(range(16))
mac = HMAC.new(secret, digestmod=SHA256)
mac.update(crypted)
entry = PacketEntry(
when=utcnow(), kind="packet", path="3d63",
raw={
"payload_typename": "GRP_DATA", "chan_hash": channel_hash(secret),
"cipher_mac": mac.digest()[:2].hex(), "crypted": crypted.hex(),
},
)
body = _plain(_viewer(entry, channels=[("Public", secret)]).render_body(80))
assert "💽 CHANNEL DATA" in body and "Public" in body
unknown = _plain(_viewer(entry).render_body(80)) # no keys held: named by fingerprint only
assert "unknown" in unknown and "Public" not in unknown