Files
MeshCore/MQTT_IMPLEMENTATION.md
T
2026-01-02 13:36:41 -08:00

537 lines
17 KiB
Markdown

# MQTT Bridge Implementation for MeshCore
This document describes the MQTT bridge implementation that allows MeshCore repeaters to uplink packet data to multiple MQTT brokers.
## Quick Start Guide
### Essential Commands to Get MQTT Repeater Running
**1. Connect to device console via repeater login or serial console (115200 baud)**
```bash
# Connect to device via serial
```
**2. Configure WiFi Credentials**
```bash
set wifi.ssid YourWiFiNetwork
set wifi.pwd YourWiFiPassword
```
**3. Reboot to Connect to WiFi**
```bash
reboot
```
**4. Verify Configuration**
```bash
get wifi.ssid
get bridge.enabled
get mqtt.origin
get mqtt.iata
```
If you wish to change mqtt.iata, use `set mqtt.iata XXX`
**5. Restart Bridge (if needed)**
```bash
# Option A: Toggle bridge off then on
set bridge.enabled off
set bridge.enabled on
# Option B: Full device reboot
reboot
```
**That's it!** The device will now:
- Connect to WiFi automatically
- Start uplinking mesh packets to Let's Mesh Analyzer
- Publish to both custom MQTT broker and Let's Mesh Analyzer servers
- Use device name as MQTT origin (set automatically)
---
## Overview
The MQTT bridge implementation provides:
- Multiple MQTT broker support (up to 3 brokers)
- Automatic reconnection with exponential backoff
- JSON message formatting for status, packets, and raw data
- Configurable topics and QoS levels
- Packet queuing during connection issues
## Files Added
### Core Implementation
- `src/helpers/bridges/MQTTBridge.h` - MQTT bridge class definition
- `src/helpers/bridges/MQTTBridge.cpp` - MQTT bridge implementation
- `src/helpers/MQTTMessageBuilder.h` - JSON message formatting utilities
- `src/helpers/MQTTMessageBuilder.cpp` - JSON message formatting implementation
### Integration
- Updated `examples/simple_repeater/MyMesh.h` - Added MQTT bridge support
- Updated `examples/simple_repeater/MyMesh.cpp` - Added MQTT bridge integration and raw radio data capture
- Updated `src/helpers/CommonCLI.h` - Added MQTT, WiFi, and timezone configuration fields
- Updated `src/helpers/CommonCLI.cpp` - Added MQTT, WiFi, and timezone CLI commands
- Updated `variants/heltec_v3/platformio.ini` - Added MQTT build configuration
- Updated `variants/station_g2/platformio.ini` - Added MQTT build configuration for Station G2
## Build Configuration
To build the MQTT bridge firmware:
### Heltec V3
```bash
pio run -e Heltec_v3_repeater_observer_mqtt
```
### Station G2
```bash
pio run -e Station_G2_repeater_observer_mqtt
```
### Custom MQTT Server Configuration
You can configure a custom MQTT server using build flags in `platformio.ini`:
```ini
[env:Heltec_v3_repeater_observer_mqtt]
build_flags =
${Heltec_lora32_v3.build_flags}
-D WITH_MQTT_BRIDGE=1
-D MQTT_SERVER='"your-mqtt-broker.com"'
-D MQTT_PORT=1883
-D MQTT_USERNAME='"your-username"'
-D MQTT_PASSWORD='"your-password"'
```
**Build Flags:**
- `MQTT_SERVER` - MQTT broker hostname
- `MQTT_PORT` - MQTT broker port (default: 1883)
- `MQTT_USERNAME` - MQTT username
- `MQTT_PASSWORD` - MQTT password
## Default Configuration
The MQTT bridge comes with the following defaults:
- **Origin**: "MeshCore-Repeater"
- **IATA**: "SEA"
- **Status Messages**: Enabled
- **Packet Messages**: Enabled
- **Raw Messages**: Disabled
- **TX Messages**: Disabled (RX only by default)
- **Status Interval**: 5 minutes (300000 ms)
- **Default Broker**: meshtastic.pugetmesh.org:1883 (username: meshdev, password: large4cats)
- **WiFi SSID**: "ssid_here" (must be configured)
- **WiFi Password**: "password_here" (must be configured)
- **Timezone**: "America/Los_Angeles" (Pacific Time with DST support)
- **Timezone Offset**: -8 hours (fallback)
- **Let's Mesh Analyzer US**: Enabled (mqtt-us-v1.letsmesh.net:443)
- **Let's Mesh Analyzer EU**: Enabled (mqtt-eu-v1.letsmesh.net:443)
## CLI Commands
### MQTT Commands
#### Get Commands
- `get mqtt.origin` - Get device origin name
- `get mqtt.iata` - Get IATA code
- `get mqtt.status` - Get status message setting (on/off)
- `get mqtt.packets` - Get packet message setting (on/off)
- `get mqtt.raw` - Get raw message setting (on/off)
- `get mqtt.tx` - Get TX message setting (on/off)
- `get mqtt.interval` - Get status publish interval (ms)
- `get mqtt.server` - Get MQTT server hostname
- `get mqtt.port` - Get MQTT server port
- `get mqtt.username` - Get MQTT username
- `get mqtt.password` - Get MQTT password
- `get mqtt.analyzer.us` - Get US Let's Mesh Analyzer server setting (on/off)
- `get mqtt.analyzer.eu` - Get EU Let's Mesh Analyzer server setting (on/off)
#### Set Commands
- `set mqtt.origin <name>` - Set device origin name
- `set mqtt.iata <code>` - Set IATA code
- `set mqtt.status on|off` - Enable/disable status messages
- `set mqtt.packets on|off` - Enable/disable packet messages
- `set mqtt.raw on|off` - Enable/disable raw messages
- `set mqtt.tx on|off` - Enable/disable TX packet messages
- `set mqtt.interval <ms>` - Set status publish interval (1000-3600000 ms)
- `set mqtt.server <hostname>` - Set MQTT server hostname
- `set mqtt.port <port>` - Set MQTT server port (1-65535)
- `set mqtt.username <username>` - Set MQTT username
- `set mqtt.password <password>` - Set MQTT password
- `set mqtt.analyzer.us on|off` - Enable/disable US Let's Mesh Analyzer server
- `set mqtt.analyzer.eu on|off` - Enable/disable EU Let's Mesh Analyzer server
### WiFi Commands
#### Get Commands
- `get wifi.ssid` - Get WiFi SSID
- `get wifi.pwd` - Get WiFi password
#### Set Commands
- `set wifi.ssid <ssid>` - Set WiFi SSID
- `set wifi.pwd <password>` - Set WiFi password
### Timezone Commands
#### Get Commands
- `get timezone` - Get timezone string (e.g., "America/Los_Angeles")
- `get timezone.offset` - Get timezone offset in hours (-12 to +14)
#### Set Commands
- `set timezone <string>` - Set timezone string (IANA format or abbreviation)
- `set timezone.offset <offset>` - Set timezone offset in hours (-12 to +14)
#### Supported Timezone Formats
- **IANA strings**: `America/Los_Angeles`, `Europe/London`, `Asia/Tokyo`, etc.
- **Common abbreviations**: `PDT`, `PST`, `MDT`, `MST`, `CDT`, `CST`, `EDT`, `EST`, `BST`, `GMT`, `CEST`, `CET`
- **UTC offsets**: `UTC-8`, `UTC+5`, `+5`, `-8`, etc.
### Bridge Commands
#### Get Commands
- `get bridge.source` - Get packet source (rx/tx)
- `get bridge.enabled` - Get bridge enabled status (on/off)
#### Set Commands
- `set bridge.source rx|tx` - Set packet source (rx for received, tx for transmitted)
- `set bridge.enabled on|off` - Enable/disable bridge
## Command Architecture
The CLI commands are organized into two levels:
### Bridge Commands (`bridge.*`)
**Low-level bridge control** - These settings apply to all bridge types (MQTT, RS232, ESP-NOW, etc.):
- `bridge.enabled` - Master switch for the entire bridge system
- `bridge.source` - Controls which packet events to capture (RX vs TX)
### Bridge-Specific Commands (`mqtt.*`, `wifi.*`, `timezone.*`)
**Implementation-specific settings** - These only apply to the MQTT bridge:
- `mqtt.*` - MQTT broker configuration, message types, and formatting
- `wifi.*` - WiFi connection settings for MQTT connectivity
- `timezone.*` - Timezone configuration for accurate timestamps
This design allows MeshCore to support multiple bridge types simultaneously while keeping configuration clean and logical.
## MQTT Topics
The bridge publishes to three main topics with the following structure:
### Status Topic: `meshcore/{IATA}/{DEVICE_PUBLIC_KEY}/status`
Device connection status and metadata (retained messages).
### Packets Topic: `meshcore/{IATA}/{DEVICE_PUBLIC_KEY}/packets`
Full packet data with RF characteristics and metadata.
### Raw Topic: `meshcore/{IATA}/{DEVICE_PUBLIC_KEY}/raw`
Minimal raw packet data for map integration.
**Note**: `{DEVICE_PUBLIC_KEY}` is the device's public key in hexadecimal format (64 characters).
## JSON Message Formats
### Status Message
```json
{
"status": "online|offline",
"timestamp": "2024-01-01T12:00:00.000000",
"origin": "Device Name",
"origin_id": "DEVICE_PUBLIC_KEY",
"model": "device_model",
"firmware_version": "firmware_version",
"radio": "radio_info",
"client_version": "meshcore-custom-repeater/{build_date}"
}
```
### Packet Message
```json
{
"origin": "MeshCore-HOWL",
"origin_id": "A1B2C3D4E5F67890...",
"timestamp": "2024-01-01T12:00:00.000000",
"type": "PACKET",
"direction": "rx|tx",
"time": "12:00:00",
"date": "01/01/2024",
"len": "45",
"packet_type": "4",
"route": "F|D|T|U",
"payload_len": "32",
"raw": "F5930103807E5F1E...",
"SNR": "12.5",
"RSSI": "-65",
"hash": "A1B2C3D4E5F67890",
"path": "node1,node2,node3"
}
```
### Raw Message
```json
{
"origin": "MeshCore-HOWL",
"origin_id": "A1B2C3D4E5F67890...",
"timestamp": "2024-01-01T12:00:00.000000",
"type": "RAW",
"data": "F5930103807E5F1E..."
}
```
## Key Features
### Raw Radio Data Capture
- Captures actual raw radio transmission data (including radio headers)
- Uses proper MeshCore packet hashing (SHA256-based)
- Provides accurate SNR/RSSI values from actual radio reception
- Supports both RX and TX packet uplinking (configurable)
### Timezone Support
- Full timezone support with automatic DST handling
- Supports IANA timezone strings, common abbreviations, and UTC offsets
- Separates local time (for timestamps) and UTC time (for time/date fields)
- Uses JChristensen/Timezone library for accurate timezone conversions
### WiFi Configuration
- Runtime WiFi credential management via CLI
- Persistent storage across reboots
- Automatic reconnection with exponential backoff
### NTP Time Synchronization
- Automatic time synchronization with NTP servers
- Periodic time updates (every hour)
- Proper UTC system time handling
### Let's Mesh Analyzer Integration
- **JWT Authentication**: Ed25519-signed tokens for secure MQTT authentication
- **WebSocket MQTT**: Support for MQTT over WebSocket connections (TLS/SSL)
- **Dual Server Support**: Both US and EU servers enabled by default
- **Automatic Token Generation**: Creates authentication tokens using device's Ed25519 keys
- **Username Format**: `v1_{UPPERCASE_PUBLIC_KEY}` (e.g., `v1_7E7662676F7F0850A8A355BAAFBFC1EB7B4174C340442D7D7161C9474A2C9400`)
- **Server Configuration**:
- US Server: `mqtt-us-v1.letsmesh.net:443` (WebSocket with TLS)
- EU Server: `mqtt-eu-v1.letsmesh.net:443` (WebSocket with TLS)
## First-Time Setup
### Prerequisites
- MeshCore device with MQTT bridge firmware flashed
- WiFi network credentials
- MQTT broker (optional - default broker is provided)
- LoRa-capable device for configuration (repeater console)
- MeshCore network access
### Step 1: Initial Boot and Network Connection
1. **Flash the firmware** to your device using PlatformIO or the build script
2. **Deploy the device** in your mesh network location
3. **Ensure WiFi connectivity** - the device will automatically connect to WiFi if credentials are pre-configured
4. **Verify mesh network access** - device should be discoverable by other mesh nodes
### Step 2: Connect via LoRa Repeater Console
Use a MeshCore companion device to configure the Repeater's MQTT bridge.
1. **Connect to the mesh** using your companion
2. **Locate the MQTT bridge device** in your contacts
3. **Log into your Repeater** using the default password (password) or whatever you configured via serial console
3. **Tap on the repeater console** on your repeater's settings
4. **Send configuration commands** via LoRa to the MQTT bridge device
### Step 3: Configure WiFi Connection
The device needs internet connectivity to publish to MQTT brokers.
**Via LoRa Repeater Console:**
```
# Set your WiFi credentials
set wifi.ssid "YourWiFiNetwork"
set wifi.pwd "YourWiFiPassword"
# Verify WiFi settings
get wifi.ssid
get wifi.pwd
```
### Step 4: Configure Device Identity
Set up your device's identity for MQTT topics and status messages.
**Via LoRa Repeater Console:**
```
# Set IATA code for topic structure (e.g., airport code)
set mqtt.iata "SEA"
# Verify settings (origin is set automatically to device name)
get mqtt.origin
get mqtt.iata
```
### Step 5: Configure Timezone
Set your local timezone for accurate timestamps.
**Via LoRa Repeater Console:**
```
# Set timezone (choose one method)
set timezone "America/Los_Angeles" # IANA format
set timezone "PDT" # Abbreviation
set timezone "UTC-8" # UTC offset
# Verify timezone
get timezone
```
### Step 6: Configure MQTT Settings
Customize which messages to publish and how often.
**Via LoRa Repeater Console:**
```
# Configure MQTT server (optional - uses defaults if not set)
set mqtt.server "your-mqtt-broker.com"
set mqtt.port 1883
set mqtt.username "your-username"
set mqtt.password "your-password"
# Enable/disable message types
set mqtt.status on # Device status messages
set mqtt.packets on # Packet data messages
set mqtt.raw off # Raw packet data (optional)
set mqtt.tx off # Transmitted packets (optional)
# Set status publish interval (default: 5 minutes)
set mqtt.interval 300000
# Verify settings
get mqtt.server
get mqtt.port
get mqtt.username
get mqtt.status
get mqtt.packets
get mqtt.interval
```
### Step 7: Verify MQTT Broker Connection
Check that the device can connect to MQTT brokers.
**Via LoRa Repeater Console:**
```
# Check bridge status
get bridge.enabled
# If disabled, enable it
set bridge.enabled on
# Check MQTT analyzer servers (optional)
get mqtt.analyzer.us
get mqtt.analyzer.eu
```
### Step 8: Monitor MQTT Messages
Once configured, the device will automatically publish messages to MQTT brokers.
**Default MQTT Broker**: `meshtastic.pugetmesh.org:1883`
- Username: `meshdev`
- Password: `large4cats`
**Topic Structure**:
- Status: `meshcore/{IATA}/{DEVICE_PUBLIC_KEY}/status`
- Packets: `meshcore/{IATA}/{DEVICE_PUBLIC_KEY}/packets`
- Raw: `meshcore/{IATA}/{DEVICE_PUBLIC_KEY}/raw`
**Example Topics**:
- `meshcore/SEA/7E7662676F7F0850A8A355BAAFBFC1EB7B4174C340442D7D7161C9474A2C9400/status`
- `meshcore/SEA/7E7662676F7F0850A8A355BAAFBFC1EB7B4174C340442D7D7161C9474A2C9400/packets`
### Step 9: Troubleshooting
#### Device Won't Connect to WiFi
**Via LoRa Repeater Console:**
```
# Check WiFi settings
get wifi.ssid
get wifi.pwd
# Reset WiFi settings
set wifi.ssid ""
set wifi.pwd ""
# Reconfigure with correct credentials
set wifi.ssid "YourWiFiNetwork"
set wifi.pwd "YourWiFiPassword"
```
#### No MQTT Messages Appearing
**Via LoRa Repeater Console:**
```
# Check bridge status
get bridge.enabled
# Check message types
get mqtt.status
get mqtt.packets
# Check device identity (origin is set automatically)
get mqtt.origin
get mqtt.iata
# Enable bridge if needed
set bridge.enabled on
```
#### Timezone Issues
**Via LoRa Repeater Console:**
```
# Check current timezone
get timezone
# Try different timezone formats
set timezone "America/New_York" # IANA format
set timezone "EST" # Abbreviation
set timezone "UTC-5" # UTC offset
```
#### LoRa Configuration Issues
- **Device not responding**: Ensure both devices are on the same mesh network
- **Commands not working**: Check that the target device is reachable via LoRa
- **No response to get commands**: Verify the device is powered and in range
### Step 10: Advanced Configuration (Optional)
#### Custom MQTT Broker
If you want to use your own MQTT broker instead of the default:
```
# Note: Custom broker configuration requires code modification
# The default broker is: meshtastic.pugetmesh.org:1883
# Username: meshdev, Password: large4cats
```
#### Let's Mesh Analyzer Servers
The device automatically connects to Let's Mesh Analyzer servers for additional monitoring:
- **US Server**: `mqtt-us-v1.letsmesh.net:443` (WebSocket with TLS)
- **EU Server**: `mqtt-eu-v1.letsmesh.net:443` (WebSocket with TLS)
These are enabled by default and use JWT authentication with your device's Ed25519 keys.
## Testing
1. Flash the MQTT bridge firmware to your device
2. Follow the first-time setup instructions above
3. Monitor MQTT broker for incoming messages
4. Verify message formats match the JSON schemas in this document
## Dependencies
- **PubSubClient**: MQTT client library
- **ArduinoJson**: JSON message formatting (v6.17.3)
- **NTPClient**: Network time protocol client
- **Timezone**: Timezone conversion library (JChristensen/Timezone)
- **WiFi**: ESP32 WiFi functionality
- **Ed25519**: Cryptographic library for JWT token signing
- **JWTHelper**: Custom JWT token generation for Let's Mesh Analyzer authentication
## Future Enhancements
- Full WebSocket MQTT implementation (currently JWT tokens are generated but WebSocket publishing is pending)
- Multiple broker configuration via CLI
- Advanced packet filtering
- Custom topic templates
- TLS/SSL support for secure connections
- Real-time WebSocket MQTT publishing to Let's Mesh Analyzer servers