Every oldest(), stats() and put() globbed and sorted the whole spool directory, so draining a backlog cost O(N^2 log N). After an outage on a Pi 3B+ the writer thread sat at 100% CPU for minutes working through 3,300 batches. The directory is now scanned once at startup and the names, sizes and total are tracked as batches are added and removed; draining 3,300 batches drops from 59s to 0.24s on a laptop. Bump to 0.1.6. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
openHop Repeater → InfluxDB plugin
An openHop Repeater plugin that
pushes the repeater's data to a remote InfluxDB (v1, v2 or v3), ready for
Grafana dashboards. A matching Grafana dashboard ships in dashboards/.
openHop Repeater ──REST /api/*────────► metrics (every 30 s)
──WS /ws/packets───► packets (real time, gap backfill)
──SSE /api/logs_stream► logs (real time, resumable)
──REST /api/*────────► snapshots (every 5 min)
──REST history APIs──► history (full import, then every 5 min)
│
batch + gzip + retry + disk spool
▼
InfluxDB ──► Grafana
| Plugin id | Entrypoint | Distribution |
|---|---|---|
community.influxdb |
openhop-influxdb |
openhop-influxdb-plugin |
What is shipped
Every point is tagged repeater=<node name> (override with repeater_tag), so
several repeaters can share one bucket.
| Measurement | Source | Tags | Fields |
|---|---|---|---|
openhop_repeater_*, openhop_system_* (≈120 families) |
/api/stats, /api/packet_stats, /api/packet_type_stats, /api/route_stats, /api/lbt_diagnostics, /api/noise_floor_stats, /api/neighbor_links |
the metric's labels | value |
openhop_packet |
/ws/packets + /api/bulk_packets |
type, type_name, route, route_name, transmitted, is_duplicate, dropped, drop_reason, rx_radio_id, tx_radio_id, src_pubkey (ADVERT only: the advertiser's public key) |
rssi, snr, score, length, payload_length, tx_delay_ms, airtime_ms, lbt_attempts, lbt_channel_busy, hop_count, original_path, forwarded_path, packet_hash, src_hash, dst_hash, path_hash, … |
openhop_log |
/api/logs_stream |
level, logger |
message, log_id |
openhop_advert |
/api/stats neighbors |
pubkey, contact_type, is_repeater, zero_hop |
node_name, rssi, snr, advert_count, first_seen, last_seen, last_seen_age_seconds, latitude, longitude |
openhop_radio |
/api/stats radios + radio_stack (one point per radio; air settings inherit from the top-level radio: like the Repeater does) |
radio_id, radio_type, default |
frequency_hz, bandwidth_hz, spreading_factor, coding_rate, tx_power_dbm, preamble_length, configured |
openhop_gps |
/api/gps |
state |
fix, satellites, DOP, speed, latitude, longitude, altitude_m |
openhop_identities, openhop_acl, openhop_rooms, openhop_db, openhop_mqtt, openhop_crc_errors, openhop_advert_rate_limit |
matching /api/* endpoints |
item for list payloads |
every numeric value, flattened (a_b_c) |
openhop_advert_received |
/api/adverts_by_contact_type (every advert the Repeater stored) |
pubkey, contact_type, is_repeater, zero_hop, new_neighbor |
node_name, rssi, snr, route_type, advert_count, first_seen, last_seen, latitude, longitude |
openhop_noise_floor |
/api/noise_floor_history |
— | noise_floor_dbm (each stored sample) |
openhop_crc_error |
/api/crc_error_history |
— | count (each stored CRC error event) |
openhop_rrd |
/api/rrd_data (imported once) |
resolution (1m, 5m, 1h) |
rx_count_per_second, tx_count_per_second, drop_count_per_second, type_N_per_second, avg_rssi, avg_snr, avg_length, avg_score, neighbor_count |
openhop_exporter |
the plugin itself | stream |
lines written, errors, spool size, connection state |
The metric families come from the official
openhop-prometheus-plugin
collector (MIT), vendored under src/openhop_influxdb_plugin/vendor/. Its metric
names and labels are reused as InfluxDB measurement names and tags.
Notes on the schema:
- Numbers are always floats. InfluxDB rejects a whole batch when a field changes type, and Repeater JSON mixes ints and floats.
- Hashes are fields, not tags, to keep series cardinality low.
pubkeyonopenhop_advertandpeer_hashon neighbour links are tags. Expect one series per heard node. - Per-radio data comes from packets. On a multi-radio Repeater the counters,
noise floor, radio status and LBT stats in
/api/statscover the whole stack (noise floor and status are the default radio's). Only packet records say which radio received (rx_radio_id) and sent (tx_radio_id) them, so the dashboard's Per radio row is built fromopenhop_packet. - Which radio hears which node comes from adverts. Only ADVERT packets name
their sender (
src_pubkey). Nodes heard by radio lists each radio/node pair. Node coverage by radio has one row per node and a column per radio, and highlights nodes heard on only one radio. Both count only the first copy of each advert (is_duplicate = false). The Repeater dedups across all its radios, so a later copy, such as another repeater re-flooding what we forwarded, is not counted as a receive on the radio that heard it. Adverts are flooded, so set Advert hops to Direct only to count only adverts heard straight from the node rather than through a relay. - Packets are idempotent. A packet seen live and again during backfill gets the same timestamp and tags, so InfluxDB overwrites it instead of duplicating it.
Privacy
- Off by default: packet payload (
include_payload) and raw packet bytes (include_raw_packet). - On by default, because the goal is "all data": node and repeater GPS
coordinates (
include_locations). Turn it off to drop all lat/lon/alt. - The generic snapshot endpoints (identities, ACL, rooms, DB, MQTT, …) contribute
only numeric values, so names, keys and other strings in those payloads never
leave the repeater. The only free-text values shipped are packet hashes and paths,
advert
node_name, and logmessage. - Tokens and passwords are never logged.
--check-configprints them as***.
Requirements
- An openHop Repeater with the plugin manager (Repeater ≥ 0.3.0; validated against
source
13eb8b2). - A Repeater API token, created in the Repeater web UI. Current Repeater tokens
are administrator-equivalent, so the plugin config holds an admin credential.
Treat
config.jsonaccordingly. - A reachable InfluxDB:
- v2: URL, org, bucket and an API token with write access to the bucket.
- v3 (Core/Enterprise/Cloud): URL, database name in
influx_bucket, and a token. - v1 (1.8 and 1.11 tested): URL,
influx_database, optionalinflux_retention_policy, andinflux_username/influx_passwordif auth is enabled.influx_tokenset tousername:passwordalso works on 1.8+. InfluxDB 1.x does not create databases on write, so the plugin issuesCREATE DATABASEonce on the first "database not found" error (influx_create_database, default on). That needs an admin user; with a write-only user, create the database yourself and grantWRITEon it.
Install
-
Download
openhop_influxdb_plugin-<version>-py3-none-any.whl(or build it with./build_wheel.sh). -
In the Repeater UI go to Plugins → Upload and select the wheel. A locally uploaded plugin starts disabled.
-
Open Settings for InfluxDB Exporter and fill in at least the following:
{ "repeater_api_token": "<repeater API token>", "influx_url": "https://influx.example.com:8086", "influx_version": "v2", "influx_org": "home", "influx_bucket": "openhop", "influx_token": "<influx write token>" }For InfluxDB 1.x use this instead:
{ "repeater_api_token": "<repeater API token>", "influx_url": "http://influx.example.com:8086", "influx_version": "v1", "influx_database": "openhop", "influx_username": "openhop", "influx_password": "<password>" } -
Enable the plugin. Check Logs and the runtime status (
GET /api/plugins/runtime?id=community.influxdb). Each stream should showrunningand the writer should showlines_writtenincreasing. -
In Grafana, add an InfluxDB data source using the InfluxQL query language (database = your bucket or 1.x database; for 1.x with auth, set user and password; for v2, add header
Authorization: Token <read token>). Then importdashboards/openhop-overview.json.
The plugin and Repeater usually run on the same host, so the default repeater_url
of http://127.0.0.1:8000 works. In the Docker image this also holds, because the
plugin manager runs in the same container.
Configuration reference
| Key | Default | Meaning |
|---|---|---|
repeater_url |
http://127.0.0.1:8000 |
Repeater web/API base URL |
repeater_api_token |
"" |
Sent as X-API-Key (a leading Bearer is stripped) |
repeater_verify_tls |
true |
Verify Repeater TLS certificates |
repeater_tag |
"" |
Value of the repeater tag; empty uses the node name from /api/stats |
influx_url |
"" |
InfluxDB base URL. Empty means the plugin idles and reports "not configured" |
influx_version |
v2 |
v1, v2 or v3 |
influx_token |
"" |
Authorization: Token … |
influx_org |
"" |
v2 organisation (required for v2) |
influx_bucket |
openhop |
v2 bucket / v3 database |
influx_database / influx_retention_policy |
openhop / "" |
v1 target |
influx_username / influx_password |
"" |
v1 basic auth |
influx_create_database |
true |
v1 only: run CREATE DATABASE if the database is missing (needs admin) |
influx_verify_tls |
true |
Verify InfluxDB TLS certificates |
stream_metrics / stream_packets / stream_logs / stream_snapshots / stream_history |
true |
Enable each stream |
metrics_interval_seconds |
30 |
Metrics poll interval (5–3600) |
snapshot_interval_seconds |
300 |
Snapshot poll interval (30–86400) |
request_timeout_seconds |
10 |
HTTP timeout for Repeater and InfluxDB calls |
initial_backfill_hours |
"all" |
On first start, import the Repeater's stored history: "all", a number of hours, or 0 for none |
backfill_hours |
"all" |
Window for a manual re-import (--backfill default, backfill_request_id) |
backfill_request_id |
"" |
Set to any new value and save with restart to re-import backfill_hours of history once |
log_min_level |
INFO |
Minimum Repeater log level to ship |
include_payload / include_raw_packet |
false |
Ship packet payload hex / raw packet hex |
include_locations |
true |
Ship GPS and advert coordinates |
batch_size |
5000 |
Max lines per InfluxDB write |
flush_interval_seconds |
5 |
Max time lines wait before being written |
spool_max_mb |
50 |
Disk spool cap for undeliverable batches (0 disables spooling) |
Restart the plugin after changing settings (the Settings dialog can do this).
Importing history
On its first start the plugin imports everything the Repeater has stored
(initial_backfill_hours, default "all"), alongside the live streams:
| Stored on the Repeater | How far back | Imported as |
|---|---|---|
| Every packet (SQLite) | storage.retention.sqlite_cleanup_days (default 31 days) |
openhop_packet |
| Every advert received | same retention | openhop_advert_received |
| Noise-floor samples | same retention | openhop_noise_floor |
| CRC error events | same retention | openhop_crc_error |
| Aggregated traffic/RF (RRD) | 1-min steps for 7 days, 5-min for 30 days, hourly for 1 year | openhop_rrd |
| Logs | only the in-memory buffer | openhop_log (sent on connect) |
Counters such as total RX/forwarded have no history on the Repeater; their series start when the plugin starts (the RRD import covers the past year at lower resolution).
After the import, adverts, noise floor and CRC errors keep flowing every
snapshot_interval_seconds, and packets keep flowing live. Each source resumes
from its own cursor in state-influxdb.json and only advances it after a whole
window has shipped, so an interrupted import is redone rather than left with a gap.
Packets are paged by timestamp, so even a month of history on a Pi imports with
bounded memory.
Re-importing is safe at any time: points keep their original timestamps and tags, so InfluxDB overwrites them instead of duplicating. Two ways:
- From the Repeater UI (no shell needed): in the plugin's Settings, set
backfill_request_idto any new value (for example today's date), optionally setbackfill_hours, and save with restart. The import runs once in the background;GET /api/plugins/runtime?id=community.influxdbshowsbackfill→done. - From a shell:
openhop-influxdb --backfill(everything) or--backfill 72(last 72 hours). It flushes and exits; the running service's cursors are untouched.
The overwrite relies on identical tags. Version 0.1.2 added the src_pubkey tag
to ADVERT packets, so ADVERT points written by 0.1.1 or earlier get no tag on
re-import; they would land in a second series and be counted twice. The
Nodes heard by radio and Node coverage by radio panels fill in from the
upgrade onwards. To fill in history as well, first delete the old ADVERT points
for the window the Repeater still holds, then re-import it. For example, for the last 7 days:
- InfluxDB 1.x:
DELETE FROM "openhop_packet" WHERE "type_name" = 'ADVERT' AND time > now() - 7d - InfluxDB 2.x:
influx delete --bucket openhop --start <7 days ago, RFC3339> --stop <now> --predicate '_measurement="openhop_packet" AND type_name="ADVERT"'
then run openhop-influxdb --backfill 168.
Reliability
- Independent streams. Each stream runs in its own thread with exponential reconnect backoff, so one failing stream never stops the others.
- Write failures. If InfluxDB is down, slow or returns 5xx/429, batches go to
a gzip spool in
$OPENHOP_PLUGIN_DATA/spool/and are replayed oldest-first when it recovers.Retry-Afteris honoured. HTTP 413 splits the batch. HTTP 400/422 (bad data) drops only that batch and logs the reason. - Packet gaps. The newest shipped packet timestamp is persisted in
state-influxdb.json. After any disconnect or restart, the plugin backfills the gap from/api/bulk_packets. - Log resume. The log stream resumes with
Last-Event-IDand resynchronises when the Repeater restarts and its log ids reset. - Clean shutdown. SIGTERM flushes (or spools) queued data within the plugin manager's 5 s stop budget.
- Bad config. Invalid or incomplete config keeps the process alive and reports
the problem in
runtime.jsonand the logs, instead of crash-looping intoFAILED.
CLI
openhop-influxdb --check-config # validate config.json, print it redacted
openhop-influxdb --once --dry-run # one collection cycle, print line protocol
openhop-influxdb --once # one collection cycle, write to InfluxDB
openhop-influxdb --backfill [HOURS|all] # import stored Repeater history, then exit
openhop-influxdb # run as a service (what the plugin manager does)
The data directory is $OPENHOP_PLUGIN_DATA (default ./devdata), or set it with
--data-dir.
Development
python3 -m venv .venv && .venv/bin/pip install -e '.[dev]'
.venv/bin/ruff check src tests scripts && .venv/bin/pytest -q
.venv/bin/python scripts/build_dashboard.py # regenerate the Grafana dashboard
./build_wheel.sh
# Validate the wheel with the real Repeater plugin parser
git clone --no-checkout https://github.com/openhop-dev/openhop_repeater.git repeater-source
git -C repeater-source checkout --detach 13eb8b2ea8b1cdb4a07ed6e282dc99e3aa8a5a8b
PYTHONPATH=repeater-source .venv/bin/python scripts/validate_wheel.py dist/*.whl
Local InfluxDB + Grafana
docker compose -f dev/docker-compose.yml up -d # InfluxDB 2 on :8086, Grafana on :3000
# Add InfluxDB 1.8 on :8087 (admin / openhop-admin); set INFLUXDB1_TAG=1.11 for 1.11
docker compose -f dev/docker-compose.yml --profile v1 up -d
mkdir -p devdata && cp dev/config.example.json devdata/config.json # then edit it
.venv/bin/openhop-influxdb --data-dir devdata --once --dry-run | head
.venv/bin/openhop-influxdb --data-dir devdata
Grafana (admin/admin) comes pre-provisioned with data sources for both InfluxDB versions and the openHop Repeater dashboard (pick the data source at the top).
No repeater handy? dev/fake_repeater.py serves the REST API, the /ws/packets
WebSocket and the /api/logs_stream SSE feed with live synthetic data:
.venv/bin/python dev/fake_repeater.py --port 8765 --token dev-token
# add --radios 2 for a two-radio bridge stack (SP0.0 / SP0.1)
# then use "repeater_url": "http://127.0.0.1:8765", "repeater_api_token": "dev-token"
The fake keeps 35 days of packets, 30 days of adverts, 31 days of noise samples and a year of RRD data, so the history import can be exercised end to end.
Note: --dry-run and --backfill never save resume points, so a later service run
still performs its own first-start import.
Licence
MIT. Includes MIT-licensed code from openhop-prometheus-plugin; see NOTICE.