Topology: a stub the hash can't place, placed by the company it keeps
A 1-byte path hash off an RX-logged relay chain often can't be widened: `3d` opens both YUL-Cartierville (3d63…) and T MeshPocket (3d1f…), so coalesce_prefixes refused to guess and left it standing as its own node. On the live graph that was 26 stubs — and they are not harmless labels. `3d` sat in the first ring beside YUL-Cartierville at the same +13.5 dB with 885 of its own samples, claiming *more* onward links than the real node. Worse, a stub pools the links of every node it could be, so it acts as a phantom hub joining neighbourhoods that were never adjacent, and every hop distance measured through it comes back short. Prefix alone is a coin toss, but the graph is not out of evidence: the stub carries the links of whichever node it really was, so the company it keeps names its owner. Neighbours that can tell the candidates apart — ones linking to exactly one of them — vote; neighbours both candidates share abstain, because two repeaters in the same city have plenty of those and they would drown the signal in local density. The bar is high, since a wrong merge does the very thing this routine exists to prevent. A winner needs two discriminating votes and a 3× lead, so a genuinely contested stub — one whose neighbours point both ways, i.e. one that really is two nodes' traffic pooled under one hash — stays exactly where it was: under-named but honest. The two passes interleave, certain-first, because every merge can resolve the next ambiguity. On the live graph: 244 nodes / 852 links → 229 / 731, ambiguous stubs 26 → 11, and 29 nodes land on a truer ring. Adds ~2% to a build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
722c8f0e4f
commit
0756537bdf
2 files changed
+186
-17
No files matched your search
+127
-17
@@ -63,6 +63,15 @@ _HOP_PENALTY = 0.35
|
||||
_MAX_SCENARIOS = 5
|
||||
_MAX_SCENARIO_HOPS = 5
|
||||
|
||||
#: The bar an ambiguous short hash must clear before it folds onto one of the nodes it
|
||||
#: could name (see :meth:`MeshTopology._corroborated_owner`): how many *discriminating*
|
||||
#: neighbours — ones that link to exactly one candidate — have to vote for the winner, and
|
||||
#: how far ahead of the runner-up it has to finish. Both are set to leave a contested stub
|
||||
#: alone: two independent neighbours agreeing with nothing dissenting is evidence, one is a
|
||||
#: coincidence, and a close second means the hash is probably pooling two nodes' traffic.
|
||||
_MIN_CORROBORATION = 2
|
||||
_CORROBORATION_MARGIN = 3
|
||||
|
||||
_HEX_DIGITS = frozenset("0123456789abcdef")
|
||||
|
||||
|
||||
@@ -380,21 +389,36 @@ class MeshTopology:
|
||||
ranker offers a redundant weaker route through the stub.
|
||||
|
||||
This closes the gap using the evidence the graph *actually holds* rather than the
|
||||
whole contact list: a short id is merged into a longer node id present in the graph
|
||||
when it is a strict prefix of **exactly one** of them (following a prefix chain —
|
||||
``65`` → ``6532`` → ``6532eb`` — to its longest end). A short id that opens two
|
||||
distinct longer nodes (``c5`` → ``c5bc…`` and ``c5ba…``) is genuinely ambiguous and
|
||||
is left as its own node; a short id that opens none (a node we have only ever heard
|
||||
narrowly) keeps its width too — it is one node, merely under-named. Merging folds
|
||||
the short id's links into the wide id's, summing samples, pooling SNR readings and
|
||||
sources, and keeping the freshest sighting, so the wide node inherits every reading
|
||||
the stub had gathered. Idempotent: a second call finds nothing left to merge.
|
||||
whole contact list, in two passes of descending confidence:
|
||||
|
||||
1. **By prefix alone** (:meth:`_next_prefix_merge`) — a short id is merged into a
|
||||
longer node id present in the graph when it is a strict prefix of **exactly
|
||||
one** of them (following a prefix chain — ``65`` → ``6532`` → ``6532eb`` — to
|
||||
its longest end). Nothing is inferred: the short can only mean that one node.
|
||||
2. **By corroboration** (:meth:`_next_corroborated_merge`) — a short id that opens
|
||||
two distinct longer nodes (``c5`` → ``c5bc…`` and ``c5ba…``) is ambiguous by
|
||||
prefix, but the graph usually knows which one it is anyway: the stub carries the
|
||||
links of whichever node it really was, so its *neighbourhood* names the owner.
|
||||
Neighbours that can tell the candidates apart vote, and a decisive result folds
|
||||
the stub onto the winner. See that method for the bar a vote has to clear.
|
||||
|
||||
The passes interleave: the graph changes with every merge, so an ambiguity can
|
||||
resolve itself once a neighbouring stub folds, and the certain pass is always
|
||||
re-run before the inferring one gets another turn.
|
||||
|
||||
A short id that opens no longer node at all (one we have only ever heard narrowly)
|
||||
keeps its width — it is one node, merely under-named — and so does one whose vote
|
||||
stays contested. Merging folds the short id's links into the wide id's, summing
|
||||
samples, pooling SNR readings and sources, and keeping the freshest sighting, so
|
||||
the wide node inherits every reading the stub had gathered. Idempotent: a second
|
||||
call finds nothing left to merge.
|
||||
|
||||
Called once at the end of :func:`build_topology`, so every consumer sees each node
|
||||
once. Safe for our own node (its 12-hex id is never a short prefix candidate).
|
||||
"""
|
||||
while True:
|
||||
merge = self._next_prefix_merge(self._node_ids())
|
||||
nodes = self._node_ids()
|
||||
merge = self._next_prefix_merge(nodes) or self._next_corroborated_merge(nodes)
|
||||
if merge is None:
|
||||
return
|
||||
self._merge_node(*merge)
|
||||
@@ -417,13 +441,7 @@ class MeshTopology:
|
||||
end inward over successive calls.
|
||||
"""
|
||||
for short in sorted(nodes, key=len):
|
||||
if len(short) >= 12: # a full canonical id is never under-specified
|
||||
continue
|
||||
exts = [
|
||||
other
|
||||
for other in nodes
|
||||
if other != short and len(other) > len(short) and other.startswith(short)
|
||||
]
|
||||
exts = self._extensions(short, nodes)
|
||||
if not exts:
|
||||
continue
|
||||
longest = max(exts, key=len)
|
||||
@@ -431,6 +449,98 @@ class MeshTopology:
|
||||
return short, longest
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _extensions(short: str, nodes: set[str]) -> list[str]:
|
||||
"""Every node id in ``nodes`` that strictly extends the under-specified ``short``.
|
||||
|
||||
Empty for a full-width canonical id (12 hex is never under-specified) and for a
|
||||
short id nothing in the graph extends.
|
||||
"""
|
||||
if len(short) >= 12:
|
||||
return []
|
||||
return [
|
||||
other
|
||||
for other in nodes
|
||||
if other != short and len(other) > len(short) and other.startswith(short)
|
||||
]
|
||||
|
||||
def _next_corroborated_merge(self, nodes: set[str]) -> Optional[tuple[str, str]]:
|
||||
"""The next ambiguous stub the *neighbourhood evidence* resolves, or ``None``.
|
||||
|
||||
Where :meth:`_next_prefix_merge` folds only what the hash alone settles, this
|
||||
answers the case it walks away from: a short id like ``bf`` that opens two real
|
||||
nodes (``bf61f2…`` and ``bfbeef…``). Prefix-wise that is a coin toss — but the stub
|
||||
is not an empty label. It carries the links of whichever node it actually was, so
|
||||
the company it keeps names its owner: every neighbour that can *tell the candidates
|
||||
apart* — one it links to that links to exactly one candidate — is a vote, and the
|
||||
rest (neighbours both candidates share, which two repeaters in the same city have
|
||||
plenty of) abstain rather than drown the signal in local density.
|
||||
|
||||
The bar is deliberately high, because a wrong merge does what this whole routine
|
||||
exists to prevent — attributes one node's evidence to another. The winner must draw
|
||||
at least :data:`_MIN_CORROBORATION` discriminating votes *and* lead the runner-up by
|
||||
:data:`_CORROBORATION_MARGIN`×, so a genuinely contested stub (one whose neighbours
|
||||
point both ways, i.e. one that really is two nodes' traffic pooled under one hash)
|
||||
stays exactly where it was: standing on its own, under-named but honest.
|
||||
|
||||
Shortest ids are offered first, matching :meth:`_next_prefix_merge`, and only the
|
||||
*maximal* candidates run — a chain's inner links (``f0`` → ``f062eb`` →
|
||||
``f062eb…``) are the same node, so they never split their own vote.
|
||||
"""
|
||||
adjacency = self._adjacency()
|
||||
for short in sorted(nodes, key=len):
|
||||
exts = self._extensions(short, nodes)
|
||||
candidates = [
|
||||
ext for ext in exts
|
||||
if not any(other != ext and other.startswith(ext) for other in exts)
|
||||
]
|
||||
if len(candidates) < 2: # settled (or ignored) by the certain pass
|
||||
continue
|
||||
owner = self._corroborated_owner(short, candidates, adjacency)
|
||||
if owner is not None:
|
||||
return short, owner
|
||||
return None
|
||||
|
||||
def _adjacency(self) -> dict[str, set[str]]:
|
||||
"""The graph as neighbour sets, for the questions link pairs answer awkwardly."""
|
||||
adjacency: dict[str, set[str]] = {}
|
||||
for a, b in self._links:
|
||||
adjacency.setdefault(a, set()).add(b)
|
||||
adjacency.setdefault(b, set()).add(a)
|
||||
return adjacency
|
||||
|
||||
@staticmethod
|
||||
def _corroborated_owner(
|
||||
short: str, candidates: list[str], adjacency: dict[str, set[str]]
|
||||
) -> Optional[str]:
|
||||
"""Which candidate the stub's discriminating neighbours elect, if any.
|
||||
|
||||
A neighbour votes only when it separates the field — it neighbours exactly one
|
||||
candidate — so shared company counts for nothing and the ``short``/candidate ids
|
||||
themselves never vote for their own case. ``None`` means no decisive winner: too
|
||||
few votes, or a runner-up close enough that the stub may well be both nodes'
|
||||
traffic under one hash.
|
||||
|
||||
Args:
|
||||
short: The under-specified id being resolved.
|
||||
candidates: The distinct node ids it could name (two or more).
|
||||
adjacency: Neighbour sets over the whole graph (see :meth:`_adjacency`).
|
||||
|
||||
Returns:
|
||||
The elected candidate, or ``None`` to leave the stub standing.
|
||||
"""
|
||||
field = set(candidates) | {short}
|
||||
votes = dict.fromkeys(candidates, 0)
|
||||
for neighbour in adjacency.get(short, ()) - field:
|
||||
electors = [c for c in candidates if neighbour in adjacency.get(c, ())]
|
||||
if len(electors) == 1:
|
||||
votes[electors[0]] += 1
|
||||
ranked = sorted(votes.items(), key=lambda pair: (-pair[1], pair[0]))
|
||||
(winner, top), (_runner, second) = ranked[0], ranked[1]
|
||||
if top < _MIN_CORROBORATION or top < _CORROBORATION_MARGIN * second:
|
||||
return None
|
||||
return winner
|
||||
|
||||
def _merge_node(self, src: str, dst: str) -> None:
|
||||
"""Relabel every link touching ``src`` onto ``dst``, folding shared links together."""
|
||||
for key in list(self._links):
|
||||
|
||||
@@ -104,6 +104,65 @@ def test_coalesce_leaves_a_genuinely_ambiguous_stub_alone() -> None:
|
||||
assert {"27", "27d4396a2967", "27e41e2d7cb5"} <= ids # the stub stays its own node
|
||||
|
||||
|
||||
def test_coalesce_folds_an_ambiguous_stub_its_neighbourhood_elects() -> None:
|
||||
"""A stub the hash can't place is placed by the company it keeps.
|
||||
|
||||
``3d`` opens both ``3d63…`` and ``3d99…``, so prefix alone is a coin toss — but the
|
||||
stub carries links to Far and Leaf, and only ``3d63…`` is seen with either. Two
|
||||
discriminating votes, none against, so the stub folds and pools its evidence.
|
||||
"""
|
||||
twin = Contact(name="Twin", public_key="3d99" + "0" * 60)
|
||||
walks = [
|
||||
_traced(("3d63", 5.0), ("f2", 5.0), ("3d63", 5.0), (None, 5.0)),
|
||||
_traced(("3d63", 5.0), ("27d4", 5.0), ("3d63", 5.0), (None, 5.0)),
|
||||
_traced(("3d99", 5.0), (None, 5.0)), # the rival is in the graph, just not out here
|
||||
_traced(("3d", 5.0), ("f2", 5.0), ("3d", 5.0), (None, 5.0)),
|
||||
_traced(("3d", 5.0), ("27d4", 5.0), ("3d", 5.0), (None, 5.0)),
|
||||
]
|
||||
topo = _topo(trace_paths=walks, contacts=[REPEATER, FAR, LEAF, twin])
|
||||
ids = {node for link in topo.links() for node in (link.a, link.b)}
|
||||
assert "3d" not in ids # elected onto the node its neighbours agree it must be
|
||||
assert {"3d63c6429436", "3d9900000000"} <= ids # the rival is untouched
|
||||
link = topo.link(topo.self_id, "3d63c6429436")
|
||||
assert link is not None and link.samples == 8 # the stub's readings came with it
|
||||
|
||||
|
||||
def test_coalesce_leaves_a_contested_stub_standing() -> None:
|
||||
"""Neighbours pointing both ways mean the hash is pooling two nodes — leave it alone."""
|
||||
twin = Contact(name="Twin", public_key="3d99" + "0" * 60)
|
||||
edge = Contact(name="Edge", public_key="b1" + "0" * 62)
|
||||
walks = [
|
||||
_traced(("3d63", 5.0), ("f2", 5.0), ("3d63", 5.0), (None, 5.0)),
|
||||
_traced(("3d63", 5.0), ("27d4", 5.0), ("3d63", 5.0), (None, 5.0)),
|
||||
_traced(("3d99", 5.0), ("b1", 5.0), ("3d99", 5.0), (None, 5.0)),
|
||||
_traced(("3d", 5.0), ("f2", 5.0), ("3d", 5.0), (None, 5.0)),
|
||||
_traced(("3d", 5.0), ("27d4", 5.0), ("3d", 5.0), (None, 5.0)),
|
||||
_traced(("3d", 5.0), ("b1", 5.0), ("3d", 5.0), (None, 5.0)),
|
||||
]
|
||||
topo = _topo(trace_paths=walks, contacts=[REPEATER, FAR, LEAF, twin, edge])
|
||||
ids = {node for link in topo.links() for node in (link.a, link.b)}
|
||||
# 2 votes to 1 is a plurality, not a verdict: the margin keeps the stub its own node.
|
||||
assert {"3d", "3d63c6429436", "3d9900000000"} <= ids
|
||||
|
||||
|
||||
def test_coalesce_ignores_neighbours_both_candidates_share() -> None:
|
||||
"""Shared company is local density, not evidence — only discriminating neighbours vote."""
|
||||
twin = Contact(name="Twin", public_key="3d99" + "0" * 60)
|
||||
walks = [
|
||||
_traced(("3d63", 5.0), ("f2", 5.0), ("3d63", 5.0), (None, 5.0)),
|
||||
_traced(("3d99", 5.0), ("f2", 5.0), ("3d99", 5.0), (None, 5.0)),
|
||||
_traced(("3d63", 5.0), ("27d4", 5.0), ("3d63", 5.0), (None, 5.0)),
|
||||
_traced(("3d99", 5.0), ("27d4", 5.0), ("3d99", 5.0), (None, 5.0)),
|
||||
_traced(("3d", 5.0), ("f2", 5.0), ("3d", 5.0), (None, 5.0)),
|
||||
_traced(("3d", 5.0), ("27d4", 5.0), ("3d", 5.0), (None, 5.0)),
|
||||
]
|
||||
topo = _topo(trace_paths=walks, contacts=[REPEATER, FAR, LEAF, twin])
|
||||
ids = {node for link in topo.links() for node in (link.a, link.b)}
|
||||
# Every neighbour of the stub is a neighbour of *both* candidates, so nobody can tell
|
||||
# them apart and the stub is not guessed onto the busier one.
|
||||
assert {"3d", "3d63c6429436", "3d9900000000"} <= ids
|
||||
|
||||
|
||||
def test_contact_routes_and_packet_paths_feed_the_graph() -> None:
|
||||
"""Firmware-learned routes and RX-logged packet paths both count as evidence."""
|
||||
routed = Contact(
|
||||
|
||||
Reference in new issue
Block a user