diff --git a/README.md b/README.md index 0aa8d9a..afb02c6 100644 --- a/README.md +++ b/README.md @@ -144,7 +144,9 @@ agreement, because it records their GPS position. | `meshcore_repeater_telemetry` | `REQ_TYPE_GET_TELEMETRY_DATA` (CayenneLPP) | one point per sensor, tagged `channel` + `sensor` (voltage, temperature, humidity, gps…). Usually at least battery voltage and MCU temperature | | `meshcore_repeater_neighbours` | `REQ_TYPE_GET_NEIGHBOURS` | `count` | | `meshcore_repeater_neighbour` | same | per zero-hop neighbouring repeater: `snr_db`, `secs_ago`, `last_heard` | -| `meshcore_poll` | this script | repeater polls: per-request success flags, path length, poll duration, `low_power_mode`, `interval_s`, configured battery thresholds | +| `meshcore_repeater_info` | `REQ_TYPE_GET_OWNER_INFO` | `firmware` (version string), `owner`, `changed` (1 when the version differs from the last one seen). Checked daily and after a reboot | +| `meshcore_repeater_history` | repeater CLI `get telemetry.temp` / `get telemetry.volt` (keymind/Cascade firmware) | `battery_v`, `temperature_c` (MCU), half-hourly, backfilled after missed polls. Only with `history = true` | +| `meshcore_poll` | this script | repeater polls: per-request success flags, path length, poll duration, `low_power_mode`, `interval_s`, configured battery thresholds, `history_ok` (when a backfill was attempted) | | `meshcore_companion_telemetry` | remote companion, telemetry request | one point per sensor, tagged `companion`, `channel`, `sensor` | | `meshcore_companion_poll` | this script | remote companion polls: `success`, path length, duration, `low_power_mode`, `interval_s` | | `meshcore_local_stats` | local companion, `CMD_GET_STATS` | battery, uptime, TX queue, noise floor, last RSSI/SNR, TX/RX airtime, packet counters, contact counts (`contacts`, `contacts_repeater`, `contacts_chat`, …, `heard_1h`) | @@ -159,9 +161,9 @@ tagged `companion`. Both also carry `pubkey` (12 hex characters). recent companion firmware. On older firmware the Companion radio tab lists what isn't supported, and the rest is still collected. -## Does the repeater store telemetry history? No. +## Does the repeater store telemetry history? Upstream: no. keymind/Cascade: a little. -I checked this against the firmware source (`examples/simple_repeater`, v1.17.x): +I checked upstream MeshCore against the firmware source (`examples/simple_repeater`, v1.17.x): - **Nothing is buffered for later.** Every reply is built from live values when the request arrives. The min/max/avg history request (`REQ_TYPE_GET_AVG_MIN_MAX`, backed by @@ -177,6 +179,27 @@ I checked this against the firmware source (`examples/simple_repeater`, v1.17.x) - **The on-flash packet log** (`log start`) can only be dumped over the local serial console (`sender_timestamp == 0`), not over the mesh. +### On-device history (keymind/Cascade firmware) + +The keymind/Cascade fork (mikecarper/MeshCore, `v1.17.1.x-halo-keymind-cascade`) keeps +**battery voltage and MCU temperature every 30 minutes for 7 days** in RAM, and an admin +can read it back one day at a time with `get telemetry.volt N` and `get telemetry.temp N`. +STM32 builds leave it out. + +Set `history = true` on a repeater (or tick *Backfill gaps* in the editor) to use it. +After a successful poll, if the last data written is more than an hour and 1.5 intervals +old, the collector fetches enough days to cover the gap (at most 7). It writes them +to `meshcore_repeater_history`. The first poll with it enabled fetches one day. Notes: + +- It needs the **admin** password; guests get "not permitted". The node list shows why + a backfill was skipped or failed. +- Each day is two CLI round-trips, so a long gap costs some airtime once. +- A reboot clears the history, so gaps across a reboot stay gaps. +- It's skipped when the repeater's clock (reported at login) is more than an hour off, + because the sample times come from that clock. +- CLI replies arrive as messages on the companion. While it waits for one, the collector + discards (and logs) any other queued messages. + ### So how often should you poll? Because the counters are cumulative, a missed poll costs resolution, not data. The @@ -240,12 +263,14 @@ dropdowns at the top. Sections: uptime, noise floor, temperature, neighbour count. - **Power:** battery voltage with the low/recover thresholds drawn as dashed lines. The thresholds are read from the data, so they always match your config. There's also - a polling-mode / reachability timeline, temperature, and poll interval. + a polling-mode / reachability timeline, temperature, and poll interval. Backfilled + on-device history is drawn dotted on the battery and temperature graphs. - **Radio:** noise floor and RSSI, SNR, TX/RX duty cycle %, packets per hour, flood vs direct, duplicates and receive errors, TX queue and error flags. - **Neighbours:** SNR history and latest SNR for each neighbouring repeater. - **Collector & link health:** poll duration, hop count, uptime (reboots show as - drops), and any extra telemetry sensors. + drops), firmware version, and any extra telemetry sensors. Firmware changes are + also marked as annotations across all graphs. - **This server's companion:** noise floor, RSSI, duty cycle, packets per hour, contacts heard, battery and SNR. - **Remote companions** (filtered by the *Companion* selector): battery, latest diff --git a/config.example.toml b/config.example.toml index a6983ec..43145e8 100644 --- a/config.example.toml +++ b/config.example.toml @@ -58,6 +58,8 @@ enabled = true status = true telemetry = true neighbours = true +# history = true # keymind/Cascade firmware + admin password: backfill missed + # polls from the repeater's 7-day half-hourly battery/temp history # [[repeater]] # name = "Hilltop" diff --git a/grafana/meshcore-repeater-dashboard.json b/grafana/meshcore-repeater-dashboard.json index 9eb40b4..735decc 100644 --- a/grafana/meshcore-repeater-dashboard.json +++ b/grafana/meshcore-repeater-dashboard.json @@ -90,6 +90,20 @@ "iconColor": "rgba(0, 211, 255, 1)", "name": "Annotations & Alerts", "type": "dashboard" + }, + { + "name": "Firmware changes", + "enable": true, + "hide": false, + "iconColor": "purple", + "datasource": { + "type": "influxdb", + "uid": "${datasource}" + }, + "query": "SELECT \"firmware\" FROM \"meshcore_repeater_info\" WHERE \"repeater\" =~ /^$repeater$/ AND \"changed\" = 1 AND $timeFilter", + "textColumn": "firmware", + "titleColumn": "", + "tagsColumn": "" } ] }, @@ -674,9 +688,20 @@ "query": "SELECT \"recover_battery_v\" FROM \"meshcore_poll\" WHERE \"repeater\" =~ /^$repeater$/ AND $timeFilter", "resultFormat": "time_series", "alias": "Recover threshold" + }, + { + "refId": "D", + "datasource": { + "type": "influxdb", + "uid": "${datasource}" + }, + "rawQuery": true, + "query": "SELECT \"battery_v\" FROM \"meshcore_repeater_history\" WHERE \"repeater\" =~ /^$repeater$/ AND $timeFilter", + "resultFormat": "time_series", + "alias": "Battery (on-device history)" } ], - "description": "Threshold lines come from the collector's config, so they always match what it is using.", + "description": "Threshold lines come from the collector's config, so they always match what it is using. Dotted: half-hourly readings the repeater kept itself, backfilled after missed polls (if enabled).", "fieldConfig": { "defaults": { "custom": { @@ -783,6 +808,39 @@ "value": 1 } ] + }, + { + "matcher": { + "id": "byName", + "options": "Battery (on-device history)" + }, + "properties": [ + { + "id": "color", + "value": { + "mode": "fixed", + "fixedColor": "green" + } + }, + { + "id": "custom.lineStyle", + "value": { + "fill": "dot", + "dash": [ + 2, + 6 + ] + } + }, + { + "id": "custom.fillOpacity", + "value": 0 + }, + { + "id": "custom.lineWidth", + "value": 1 + } + ] } ] }, @@ -978,6 +1036,17 @@ "query": "SELECT \"value\" FROM \"meshcore_repeater_telemetry\" WHERE \"repeater\" =~ /^$repeater$/ AND $timeFilter AND \"sensor\" = 'temperature' GROUP BY \"channel\"", "resultFormat": "time_series", "alias": "Temperature ch$tag_channel" + }, + { + "refId": "B", + "datasource": { + "type": "influxdb", + "uid": "${datasource}" + }, + "rawQuery": true, + "query": "SELECT \"temperature_c\" FROM \"meshcore_repeater_history\" WHERE \"repeater\" =~ /^$repeater$/ AND $timeFilter", + "resultFormat": "time_series", + "alias": "MCU temperature (on-device history)" } ], "fieldConfig": { @@ -1005,7 +1074,34 @@ }, "unit": "celsius" }, - "overrides": [] + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "MCU temperature (on-device history)" + }, + "properties": [ + { + "id": "custom.lineStyle", + "value": { + "fill": "dot", + "dash": [ + 2, + 6 + ] + } + }, + { + "id": "custom.fillOpacity", + "value": 0 + }, + { + "id": "custom.lineWidth", + "value": 1 + } + ] + } + ] }, "options": { "legend": { @@ -1022,7 +1118,8 @@ "mode": "multi", "sort": "desc" } - } + }, + "description": "Dotted: half-hourly MCU temperature the repeater kept itself, backfilled after missed polls (if enabled)." }, { "id": 13, @@ -1881,7 +1978,7 @@ "gridPos": { "x": 0, "y": 58, - "w": 8, + "w": 6, "h": 8 }, "targets": [ @@ -1950,9 +2047,9 @@ "uid": "${datasource}" }, "gridPos": { - "x": 8, + "x": 6, "y": 58, - "w": 8, + "w": 6, "h": 8 }, "targets": [ @@ -2022,9 +2119,9 @@ "uid": "${datasource}" }, "gridPos": { - "x": 16, + "x": 12, "y": 58, - "w": 8, + "w": 6, "h": 8 }, "targets": [ @@ -2085,6 +2182,70 @@ } } }, + { + "id": 42, + "type": "stat", + "title": "Firmware", + "datasource": { + "type": "influxdb", + "uid": "${datasource}" + }, + "gridPos": { + "x": 18, + "y": 58, + "w": 6, + "h": 8 + }, + "targets": [ + { + "refId": "A", + "datasource": { + "type": "influxdb", + "uid": "${datasource}" + }, + "rawQuery": true, + "query": "SELECT last(\"firmware\") FROM \"meshcore_repeater_info\" WHERE \"repeater\" =~ /^$repeater$/ AND $timeFilter", + "resultFormat": "time_series", + "alias": "Firmware" + } + ], + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "text", + "value": null + } + ] + } + }, + "overrides": [] + }, + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "/.*/", + "values": false + }, + "colorMode": "value", + "graphMode": "none", + "textMode": "value", + "justifyMode": "center", + "orientation": "auto", + "wideLayout": true, + "text": { + "valueSize": 16 + } + }, + "description": "Version reported by the repeater, checked daily and after every reboot. Upgrades also show as annotations." + }, { "id": 29, "type": "timeseries", @@ -2951,7 +3112,7 @@ "rawQuery": true, "query": "SELECT \"value\" FROM \"meshcore_companion_telemetry\" WHERE \"companion\" =~ /^$companion$/ AND $timeFilter AND \"sensor\" !~ /^(voltage|temperature|gps)$/ GROUP BY \"companion\", \"sensor\", \"channel\"", "resultFormat": "time_series", - "alias": "$tag_companion \u00b7 $tag_sensor ch$tag_channel" + "alias": "$tag_companion · $tag_sensor ch$tag_channel" } ], "description": "Any other sensors the companion shares, e.g. illuminance, humidity or pressure (environment telemetry).", diff --git a/meshcore_influx.py b/meshcore_influx.py index 758253d..2bcd240 100644 --- a/meshcore_influx.py +++ b/meshcore_influx.py @@ -20,10 +20,12 @@ Usage: import argparse import asyncio +import base64 import collections import getpass import json import logging +import math import os import signal import sys @@ -94,7 +96,7 @@ class BatteryPolicy: # Keys that may be set in [poll] and overridden per [[repeater]]. POLICY_KEYS = ("interval_seconds", "low_battery_mv", "recover_battery_mv", "low_battery_interval_seconds", "low_battery_skip") -REPEATER_KEYS = ("name", "password", "pubkey_prefix", "enabled", *REQUEST_KINDS) +REPEATER_KEYS = ("name", "password", "pubkey_prefix", "enabled", *REQUEST_KINDS, "history") COMPANION_KEYS = ("name", "pubkey_prefix", "enabled", "location") NODE_SECTIONS = {"repeater": REPEATER_KEYS, "companion": COMPANION_KEYS} # TOML [[section]] -> keys @@ -115,6 +117,7 @@ class NodeCfg: telemetry: bool = True neighbours: bool = True location: bool = False # companion: also request GPS location telemetry + history: bool = False # repeater: backfill gaps from its on-device history (admin only) interval: int = 900 battery: BatteryPolicy = field(default_factory=BatteryPolicy) @@ -155,7 +158,7 @@ def build_node(raw: dict, poll: dict, kind: str = "repeater") -> NodeCfg: raise ValueError(f"{name}: pubkey_prefix must be hex") if kind == "companion": # Companions only answer telemetry requests; there is no login. - raw = {**raw, "status": False, "telemetry": True, "neighbours": False} + raw = {**raw, "status": False, "telemetry": True, "neighbours": False, "history": False} return NodeCfg( name=name, kind=kind, @@ -166,6 +169,7 @@ def build_node(raw: dict, poll: dict, kind: str = "repeater") -> NodeCfg: telemetry=bool(raw.get("telemetry", True)), neighbours=bool(raw.get("neighbours", True)), location=bool(raw.get("location", False)), + history=bool(raw.get("history", False)), interval=_int(eff["interval_seconds"] or 900, f"{name}: interval_seconds", 60), battery=pol, ) @@ -266,6 +270,10 @@ class RepeaterState: battery_mv: Optional[int] = None last_success: float = 0.0 # epoch of last fully successful poll last_result: dict = field(default_factory=dict) # summary for the web UI + firmware: str = "" # version string from the owner-info request + firmware_checked: float = 0.0 # epoch of the last owner-info request + uptime_s: Optional[int] = None # last reported uptime, to spot reboots (and possible upgrades) + history_through: float = 0.0 # epoch up to which battery/temperature are written (live or backfilled) def interval(self, rpt: NodeCfg) -> int: return rpt.battery.interval if self.low_power else rpt.interval @@ -356,16 +364,164 @@ async def retry(what: str, attempts: int, coro_fn): return None -async def login(mc: MeshCore, contact: dict, rpt: NodeCfg, cfg: Config) -> bool: +async def login(mc: MeshCore, contact: dict, rpt: NodeCfg, cfg: Config) -> Optional[dict]: + """Returns the LOGIN_SUCCESS payload (is_admin, server_timestamp, ...), or None.""" async def _do(): ev = await mc.commands.send_login_sync(contact, rpt.password, timeout=cfg.timeout) if ev is None or ev.type != EventType.LOGIN_SUCCESS: return None return ev ev = await retry(f"{rpt.name} login", cfg.retries, _do) - if ev: - log.info("%s: logged in (admin=%s)", rpt.name, ev.payload.get("is_admin")) - return ev is not None + if ev is None: + return None + log.info("%s: logged in (admin=%s)", rpt.name, ev.payload.get("is_admin")) + return ev.payload or {} + + +REQ_TYPE_GET_OWNER_INFO = 0x07 # repeater firmware; not in meshcore's BinaryReqType yet + + +async def req_owner_info(mc: MeshCore, contact: dict, timeout: float) -> Optional[dict]: + """Firmware version, node name and owner info ("ver\\nname\\nowner"). Guests may ask.""" + res = await mc.commands.send(b"\x32" + bytes.fromhex(contact["public_key"]) + bytes([REQ_TYPE_GET_OWNER_INFO]), + [EventType.MSG_SENT, EventType.ERROR]) + if res is None or res.type == EventType.ERROR: + return None + wait = timeout or res.payload["suggested_timeout"] / 800 + ev = await mc.dispatcher.wait_for_event(EventType.BINARY_RESPONSE, + attribute_filters={"tag": res.payload["expected_ack"].hex()}, + timeout=wait) + if ev is None: + return None + text = bytes.fromhex(ev.payload["data"]).decode("utf-8", "replace").strip("\x00") + firmware, _, rest = text.partition("\n") + _, _, owner = rest.partition("\n") + return {"firmware": firmware.strip(), "owner": owner.strip()} if firmware.strip() else None + + +class CliSession: + """Remote CLI commands to a logged-in repeater. The repeater drops a command whose + timestamp isn't newer than the last one it saw from us (including the login, which + the companion stamps with its own clock), and replays its cached reply when the same + timestamp is resent, so each command gets a fresh, increasing timestamp and retries + reuse it.""" + + def __init__(self, mc: MeshCore, contact: dict, timeout: float): + self.mc, self.contact, self.timeout = mc, contact, timeout + self.last_ts = 0 + + async def start(self) -> None: + dev = 0 + try: + ev = await self.mc.commands.get_time() + if ev is not None and ev.type != EventType.ERROR: + dev = int(ev.payload.get("time", 0)) + except Exception as e: # noqa: BLE001 + log.debug("companion clock read failed: %s", e) + self.last_ts = max(int(time.time()), dev) + + def next_ts(self) -> int: + self.last_ts = max(int(time.time()), self.last_ts + 1) + return self.last_ts + + async def send(self, cmd: str, ts: int) -> Optional[str]: + res = await self.mc.commands.send_cmd(self.contact, cmd, timestamp=ts) + if res is None or res.type == EventType.ERROR: + return None + deadline = time.monotonic() + (self.timeout or res.payload["suggested_timeout"] / 800) + prefix = self.contact["public_key"][:12] + # The reply is queued on the companion like any message; fetch until it turns up. + while time.monotonic() < deadline: + while True: + ev = await self.mc.commands.get_msg() + if ev is None or ev.type in (EventType.NO_MORE_MSGS, EventType.ERROR): + break + p = ev.payload + if ev.type == EventType.CONTACT_MSG_RECV and p.get("pubkey_prefix") == prefix and p.get("txt_type") == 1: + return p.get("text", "") + log.info("discarded a %s message from %s while waiting for a CLI reply", + ev.type.value, p.get("pubkey_prefix") or p.get("channel_idx", "?")) + await asyncio.sleep(1) + return None + + +# On-device telemetry history (keymind/Cascade repeater firmware): `get telemetry.temp N` and +# `get telemetry.volt N` return one day (48 half-hour samples) per page as Base64. Page 1 is +# the newest; there are 7. The history lives in RAM, so a reboot clears it. +HISTORY_TEMPERATURE, HISTORY_VOLTAGE = 0x11, 0x12 +HISTORY_PAGES = 7 +HISTORY_COMMANDS = {"temperature_c": "get telemetry.temp", "battery_v": "get telemetry.volt"} + + +def decode_history_page(reply: str) -> tuple[str, list[tuple[int, float]]]: + """Decode one history page into (field, [(epoch, value), ...]). + Missing and out-of-range samples are dropped. Raises ValueError if it isn't a page.""" + text = reply.strip() + if not text.startswith(">"): + raise ValueError(text[:80] or "empty reply") + raw = base64.b64decode(text[1:].strip(), validate=True) + if len(raw) < 7: + raise ValueError("page too short") + kind, start, step, count = raw[0], int.from_bytes(raw[1:5], "little"), raw[5] * 60, raw[6] + body = raw[7:] + out: list[tuple[int, float]] = [] + if kind == HISTORY_VOLTAGE: + if len(body) < count: + raise ValueError("voltage page truncated") + # 0 = none, 1 = below 1.88 V, 2..254 = 1.88..4.40 V in 10 mV steps, 255 = above 4.40 V + for i, code in enumerate(body[:count]): + if 2 <= code <= 254: + out.append((start + i * step, round((1880 + (code - 2) * 10) / 1000, 2))) + return "battery_v", out + if kind == HISTORY_TEMPERATURE: + # 2-bit status per sample (1 = value), then 7-bit values (0 = -50 C), both MSB first. + status_len, data_len = (count * 2 + 7) // 8, (count * 7 + 7) // 8 + if len(body) < status_len + data_len: + raise ValueError("temperature page truncated") + status = int.from_bytes(body[:status_len], "big") + values = int.from_bytes(body[status_len:status_len + data_len], "big") + for i in range(count): + if (status >> (status_len * 8 - 2 * (i + 1))) & 0x3 == 1: + out.append((start + i * step, float(((values >> (data_len * 8 - 7 * (i + 1))) & 0x7F) - 50))) + return "temperature_c", out + raise ValueError(f"unknown page type 0x{kind:02x}") + + +def history_pages_needed(state: "RepeaterState", interval: int, now: float) -> int: + """How many days of history cover the gap since the last data we wrote (0 = no gap).""" + if not state.history_through: + return 1 + gap = now - state.history_through + if gap < max(3600, 1.5 * interval): # samples are half-hourly; smaller gaps aren't worth the airtime + return 0 + return min(HISTORY_PAGES, math.ceil(gap / 86400)) + + +async def fetch_history(cli: CliSession, rpt: NodeCfg, key: str, pages: int, cfg: Config, + now: float) -> Optional[list[Point]]: + """Fetch and decode `pages` days of battery and temperature history. None on failure.""" + pts: list[Point] = [] + for fld, cmd in HISTORY_COMMANDS.items(): + for page in range(1, pages + 1): + ts = cli.next_ts() + reply = await retry(f"{rpt.name} {cmd} {page}", cfg.retries, lambda: cli.send(f"{cmd} {page}", ts)) + if reply is None: + return None + if "history is empty" in reply: + break + try: + got, samples = decode_history_page(reply) + except ValueError as e: + log.warning("%s: %s %d: unexpected reply: %s", rpt.name, cmd, page, e) + return None + if got != fld: + log.warning("%s: %s %d: got %s page", rpt.name, cmd, page, got) + return None + for t, v in samples: + if SANE_EPOCH <= t <= now + 1800: + pts.append(Point("meshcore_repeater_history").tag("repeater", rpt.name) + .tag("pubkey", key[:12]).field(fld, v).time(t, WritePrecision.S)) + return pts # -------------------------------------------------------------------------- @@ -464,7 +620,10 @@ async def poll_repeater(mc: MeshCore, rpt: NodeCfg, cfg: Config, state: Repeater .field("error", "contact not found").time(ts, WritePrecision.S)] key = contact["public_key"] - logged_in = await login(mc, contact, rpt, cfg) + session = await login(mc, contact, rpt, cfg) + logged_in = session is not None + rebooted = False + history: Optional[str] = None # backfill outcome for the UI: None = not attempted if logged_in: # Status first: it carries the battery reading. if wanted["status"]: @@ -474,6 +633,9 @@ async def poll_repeater(mc: MeshCore, rpt: NodeCfg, cfg: Config, state: Repeater pts.append(p) ok["status"] = True battery_mv = st.get("bat") + if (up := st.get("uptime")) is not None: + rebooted = state.uptime_s is not None and up < state.uptime_s + state.uptime_s = int(up) if wanted["telemetry"]: lpp = await retry(f"{rpt.name} telemetry", cfg.retries, @@ -491,6 +653,44 @@ async def poll_repeater(mc: MeshCore, rpt: NodeCfg, cfg: Config, state: Repeater pts += neighbour_points(rpt, key, nb, mc, ts) ok["neighbours"] = True + # Firmware version: daily, and after a reboot (which is how an upgrade shows up). + if not state.firmware or rebooted or time.time() - state.firmware_checked > 86400: + info = await retry(f"{rpt.name} owner info", cfg.retries, + lambda: req_owner_info(mc, contact, cfg.timeout)) + state.firmware_checked = time.time() # also on failure: old firmware may never answer + if info: + changed = bool(state.firmware) and info["firmware"] != state.firmware + if changed: + log.warning("%s: firmware changed %s -> %s", rpt.name, state.firmware, info["firmware"]) + state.firmware = info["firmware"] + pts.append(Point("meshcore_repeater_info") + .tag("repeater", rpt.name).tag("pubkey", key[:12]) + .field("firmware", info["firmware"]).field("owner", info["owner"]) + .field("changed", int(changed)) + .time(ts, WritePrecision.S)) + + # Fill the gap since the last good poll from the repeater's own half-hourly history. + live = ok["status"] or ok["telemetry"] + if live and rpt.history and (pages := history_pages_needed(state, state.interval(rpt), ts)): + clock = session.get("server_timestamp") + if not session.get("is_admin"): + history = "needs the admin password" + elif clock and abs(clock - ts) > 3600: + history = f"repeater clock is off by {int(clock - ts)}s" + else: + cli = CliSession(mc, contact, cfg.timeout) + await cli.start() + got = await fetch_history(cli, rpt, key, pages, cfg, ts) + if got is None: + history = "failed (firmware without telemetry history?)" + else: + pts += got + history = f"backfilled {len(got)} samples from {pages} day(s)" + state.history_through = ts + log.info("%s: history %s", rpt.name, history) + if live and history is None: # no gap to fill (or backfill is off) + state.history_through = ts + try: await mc.commands.send_logout(contact) except Exception: # noqa: BLE001 - best effort @@ -505,7 +705,7 @@ async def poll_repeater(mc: MeshCore, rpt: NodeCfg, cfg: Config, state: Repeater state.last_result = { "ts": ts, "success": success, "logged_in": logged_in, "ok": ok, "wanted": wanted, "battery_mv": battery_mv, "duration_s": duration, "path_len": contact.get("out_path_len", -1), - "pubkey": key[:12], + "pubkey": key[:12], "firmware": state.firmware, "history": history, "error": "" if success else ("login failed (wrong password or no response)" if not logged_in else "no response for: " + ", ".join(missing)), } @@ -522,6 +722,8 @@ async def poll_repeater(mc: MeshCore, rpt: NodeCfg, cfg: Config, state: Repeater .field("duration_s", float(duration)) .field("low_power_mode", int(bool(state.low_power))) .field("interval_s", int(state.interval(rpt)))) + if history is not None: + p = p.field("history_ok", int(history.startswith("backfilled"))) if rpt.battery.enabled: p = (p.field("low_battery_v", rpt.battery.low_mv / 1000.0) .field("recover_battery_v", rpt.battery.recover_mv / 1000.0)) diff --git a/static/index.html b/static/index.html index 22de45d..4aac238 100644 --- a/static/index.html +++ b/static/index.html @@ -329,6 +329,7 @@ code { font-size: 12.5px; background: var(--bg); padding: 1px 5px; border-radius + @@ -465,9 +466,11 @@ function renderRepeaters() { const [cls, label] = repStatus(r); const v = r.battery_mv ? (r.battery_mv / 1000).toFixed(2) + " V" : "—"; const err = r.enabled && r.result && !r.result.success && r.result.error ? `