MeshCore Analytics

A real-time analytics platform for MeshCore networks. It ingests MQTT packets from mctomqtt-compatible observers, decodes them with @michaelhart/meshcore-decoder, stores them in TimescaleDB, and serves interactive dashboards plus public-facing site pages with live mapping, link intelligence, coverage modelling, packet analytics, and worker/system health.

---

Features

• Real-time node map with animated packet arcs and live WebSocket updates
• RF coverage viewshed polygons per repeater using SRTM terrain data
• Link intelligence overlay with directional observations and path-loss viability
• Beta path prediction model with concurrent worker pool and hourly path-learning prior rebuilds
• Multibyte path-hash support (1-byte, 2-byte, 3-byte) throughout the live ingest and pathing stack
• Decoded live packet feed (Advert, GroupText, DM, ACK, Path, Trace)
• Stats pages and chart endpoints for packet rates, radios, hops, and activity
• Public Health page with worker status/history + server resource metrics
• UK site Feed page for public MQTT observer traffic visibility
• Repeater owner portal with MQTT username/password login and encrypted cookie session
• Owner dashboard with repeater summary, direct sender map, live packets, advert trend, heard-by list, link health, and alerts
• Multi-network ingestion (meshcore/* and ukmesh/*) with per-site filtering
• Isolated test-feed support via meshcore-test/* and test.ukmesh.com
• Multi-observer deduplication by packet hash
• MQTT connection monitor with Mosquitto log parsing and reconnect tracking

---

Roadmap

Phase 1 - Core platform (complete)

• MQTT ingestion via mctomqtt with multi-observer support
• Packet decoding with @michaelhart/meshcore-decoder
• TimescaleDB storage and live WebSocket fan-out
• React dashboard: node map, animated packet arcs, decoded live feed
• Packet deduplication by hash across observers

Phase 2 - RF coverage and link intelligence (complete)

• Viewshed worker: SRTM terrain-aware radio horizon computation per repeater
• Coverage polygons served as GeoJSON and rendered on the map
• Link worker: observed relay-path processing into node-to-node link intelligence
• Directional link counts and path-loss viability modelling
• UK mainland clipping to remove sea coverage artifacts

Phase 3 - Path learning and predictions (beta)

• Hourly path-learning prior rebuild worker
• Beta path overlays and confidence scoring
• Historical calibration using observed packet behavior
• Multibyte path-hash aware path resolution
• Concurrent resolve worker pool for high-throughput path matching

Phase 4 - Public website and operations (complete)

• Separate public-facing website pages (install, MQTT, packets, stats)
• Public Health page with worker/system status and history
• Click-to-explain worker cards
• UK Feed page for live public observer traffic

Phase 5 - Repeater owner portal (in progress)

• MQTT username/password owner login with encrypted cookie session
• Dedicated owner auth database for username → repeater ownership mapping
• Owner-facing dashboard: repeater summary, packet history, advert counts, direct sender map, heard-by list, link health, and alerts
• Planned node placement tool: drop a marker on the map, preview estimated RF coverage before deploying hardware
• Planned repeater registration/claim workflow improvements

Phase 6 - Network intelligence expansion (planned)

• Network topology graph showing strongest relay relationships
• Better cross-network correlation and packet-hearing analysis
• Operator-facing diagnostics for path model explainability

Phase 7 - Predicted vs observed RF model validation (planned)

• Compare terrain-predicted links against real observed relay behavior
• Highlight high-confidence mismatches for network tuning
• Improve viability heuristics from observed false-positive/false-negative links

---

Current State

• Split worker architecture for resilience:
  • viewshed-worker (coverage compute)
  • link-worker (link/path-loss processing)
  • path-learning-worker (hourly model rebuild)
  • path-history-worker (historical path resolution backfill)
  • health-worker (health snapshots)
  • link-backfill-worker (one-shot historical backfill)
• Path resolver runs a concurrent worker pool (resolveWorker, resolvePool, resolveCache) to handle high packet volumes without blocking the main ingest loop.
• Nginx frontend proxies use Docker DNS resolver-based upstreams to avoid stale backend IP issues after container recreates.
• Owner authentication uses MQTT credentials plus a separate owner-auth mapping database rather than public-key login.
• Live coverage is currently served by an RF radial model calibrated against observed repeater links and terrain data.
• Public/test feeds are isolated at the topic level, with meshcore-test/* excluded from the public sites.
• MQTT connection state is tracked via Mosquitto log parsing — connect/disconnect events are available in the health feed.

---

Quick Start

# 1. Clone and enter the project
git clone https://github.com/gadgethd/ukmesh.git
cd ukmesh

# 2. Copy and configure environment
cp .env.example .env
# Edit .env — at minimum set POSTGRES_PASSWORD, JWT_SECRET, MQTT_PASSWORD

# 3. Start everything
docker compose up -d

# 4. Check logs
docker compose logs -f backend

Local endpoints:

• Backend API/WS: http://localhost:3000
• App frontend: http://localhost:3001 (if configured)
• App (ukmesh): http://localhost:3003
• Website (ukmesh): http://localhost:3004
• Dev/test site: http://localhost:3005 / http://localhost:3006

To expose it publicly, configure a Cloudflare Tunnel (see below) or reverse proxy of your choice.

---

Environment Variables

Copy .env.example to .env and fill in your values. All variables used by the app:

Variable	Default	Description
POSTGRES_DB	meshcore	TimescaleDB database name
POSTGRES_USER	meshcore	TimescaleDB user
POSTGRES_PASSWORD	(required)	TimescaleDB password
MQTT_BROKER_URL	ws://mosquitto:9001	Mosquitto WebSocket URL (internal)
MQTT_USERNAME	backend	MQTT client username
MQTT_PASSWORD	(required)	MQTT client password
REDIS_URL	redis://redis:6379	Redis URL for WebSocket pub/sub
JWT_SECRET	(required)	Secret for JWT verification
ALLOWED_ORIGINS	http://localhost:3001,http://localhost:3002	Comma-separated browser origins allowed for CORS and WebSocket
VITE_APP_HOSTNAME	(blank — always shows dashboard)	If set, only this hostname serves the analytics dashboard; all others serve the public website layout
MESHCORE_CHANNEL_SECRETS	(blank)	Comma-separated channel secrets for decrypting GroupText packets. Format: name:hex or bare hex. The default MeshCore public channel key is always included.
OPENTOPODATA_API	https://api.opentopodata.org	Elevation API endpoint for viewshed computation
OWNER_DATABASE_URL	(optional)	Separate Postgres database URL for owner portal username → repeater mappings
OWNER_COOKIE_SECRET	(optional but recommended)	Secret used to encrypt/sign the owner session cookie
OWNER_MQTT_USERNAME_MAP	(optional fallback)	Legacy static mapping in the format `user=nodeId1
COVERAGE_MODEL	terrain_los	Coverage model used by viewshed-worker
COVERAGE_MODEL_VERSION	2	Coverage schema/version gate used to trigger recomputation
CLOUDFLARE_TUNNEL_TOKEN	(optional)	Cloudflare Zero Trust tunnel token
PORT	3000	Internal app port

---

Mosquitto Setup

Mosquitto is configured for WebSocket-only access with password authentication. After first starting the stack, add a password for the backend client and any node clients:

# Add the backend client password (must match MQTT_PASSWORD in .env)
docker exec meshcore-analytics-mosquitto-1 \
  mosquitto_passwd -b /mosquitto/config/passwd backend your_password

# Add a node client
docker exec meshcore-analytics-mosquitto-1 \
  mosquitto_passwd -b /mosquitto/config/passwd node1 another_password

docker compose restart mosquitto

Edit mosquitto/acl to grant the appropriate topic permissions to each user.

---

Cloudflare Tunnel (optional)

To expose the app and MQTT broker publicly without opening firewall ports:

1. Go to Cloudflare Zero Trust → Networks → Tunnels
2. Create a tunnel and copy the token
3. Add to .env: CLOUDFLARE_TUNNEL_TOKEN=<token>
4. Start with the tunnel profile: docker compose --profile tunnel up -d
5. Configure public hostnames in the Cloudflare dashboard (example):
   • app.example.com → http://app-ukmesh:80
   • www.example.com → http://website-ukmesh:80
   • mqtt.example.com → http://mosquitto:9001
   • healthcheck.example.com → http://mesh-health-check:3090

For UKMesh health checks, point healthcheck.ukmesh.com at http://mesh-health-check:3090 in the same tunnel. The container uses the internal Mosquitto WebSocket listener and persists observer/result state in the mesh_health_check_data Docker volume. By default it uses HEALTHCHECK_TEST_CHANNEL_NAME=ukmeshtest and reuses the existing test:... entry from MESHCORE_CHANNEL_SECRETS via HEALTHCHECK_TEST_CHANNEL_SECRET_SOURCE_NAME=test.

---

MQTT Topic Structure

The backend subscribes to meshcore/#, ukmesh/#, and meshcore-test/#. MeshCore observers publish mctomqtt-compatible JSON envelopes to topics of the form:

meshcore/<IATA>/<observer-public-key>/packets   # received/transmitted packets
meshcore/<IATA>/<observer-public-key>/status    # node status advertisement
ukmesh/<IATA>/<observer-public-key>/packets
ukmesh/<IATA>/<observer-public-key>/status
meshcore-test/<IATA>/<observer-public-key>/packets
meshcore-test/<IATA>/<observer-public-key>/status

Payloads are JSON envelopes containing a raw hex field (the MeshCore packet) plus metadata such as RSSI, SNR, direction, and hash. The ingest path supports 1-byte, 2-byte, and 3-byte path hashes carried inside the raw packet.

---

Architecture

MeshCore Devices
     │ LoRa RF
     ▼
 mctomqtt-compatible observer
     │ MQTT over WebSocket/TLS
     ▼
 Mosquitto ─────────────────────────────── (optional Cloudflare Tunnel)
     │ subscribe meshcore/# + ukmesh/#
     ▼
 Backend (Node.js/TypeScript)
     │
     ├─ meshcore-decoder → TimescaleDB (packets, nodes, coverage, priors, health snapshots)
     │
     ├─ Path resolver worker pool (concurrent resolve workers + LRU cache)
     │
     ├─ Redis pub/sub
     │
     ├─ WebSocket → frontend live updates
     └─ REST API /api/*

 App/Web Frontends (Nginx + React)
     └─ app-ukmesh / website-ukmesh / website-dev (interactive dashboard + public site + owner portal)

 Python Workers
     ├─ viewshed-worker (meshcore:viewshed_jobs)
     ├─ link-worker (meshcore:link_jobs)
     ├─ SRTM terrain tiles (auto-downloaded)
     └─ node_coverage + node_links updates

 Backend Workers (Node.js)
     ├─ path-learning-worker (hourly prior rebuild)
     ├─ path-history-worker (historical path resolution)
     ├─ health-worker (minute snapshots)
     └─ link-backfill-worker (one-shot historical backfill)

 Owner Auth
     └─ separate Postgres DB for MQTT username → repeater ownership mapping

---

Services

Service	Image	Purpose
timescaledb	timescale/timescaledb:latest-pg16	Time-series and relational data storage
mosquitto	eclipse-mosquitto:2	MQTT broker (WebSocket only)
redis	redis:7-alpine	WebSocket fan-out pub/sub and job queue
backend	Built from Dockerfile.backend	MQTT ingest, decoding, API, WebSocket
path-learning-worker	Built from Dockerfile.backend	Hourly path-learning model rebuilds
path-history-worker	Built from Dockerfile.backend	Historical path resolution backfill
health-worker	Built from Dockerfile.backend	Periodic health snapshot capture
link-backfill-worker	Built from Dockerfile.backend	One-shot historical link backfill
viewshed-worker	Built from viewshed-worker/Dockerfile	Terrain-aware RF coverage computation
link-worker	Built from viewshed-worker/Dockerfile	Link/path-loss processing from observed paths
app-ukmesh	Built from Dockerfile.app	Interactive dashboard frontend
website-ukmesh	Built from Dockerfile.website	Public website frontend
mesh-health-check	Built from pinned yellowcooln/meshcore-health-check source	MeshCore observer coverage health-check app
app-dev	Built from Dockerfile.app	Isolated test/dev dashboard frontend
website-dev	Built from Dockerfile.website	Isolated test/status site for meshcore-test/* traffic
cloudflared	cloudflare/cloudflared	Optional Cloudflare Tunnel (use --profile tunnel)

---

Data Retention

• Packet retention policy is currently disabled. Historical data is kept indefinitely unless explicitly pruned.
• Node/link/coverage/path-learning/health tables are also retained indefinitely by default.

---

Acknowledgements

This project is built on the following open source libraries and tools:

Frontend

Package	License
React	MIT
Vite	MIT
TypeScript	Apache 2.0
Leaflet	BSD 2-Clause
react-leaflet	Hippocratic 2.1
deck.gl	MIT
react-router-dom	MIT
Recharts	MIT
polygon-clipping	MIT

Backend

Package	License
Express	MIT
MQTT.js	MIT
ws	MIT
ioredis	MIT
node-postgres	MIT
cors	MIT
express-rate-limit	MIT
@michaelhart/meshcore-decoder	MIT

Viewshed worker (Python)

Package	License
NumPy	BSD 3-Clause
SciPy	BSD 3-Clause
Shapely	BSD 3-Clause
GDAL	MIT/X
psycopg2	LGPL v3
redis-py	MIT
Requests	Apache 2.0

Infrastructure

Tool	License
TimescaleDB	Apache 2.0 (Community)
Redis	BSD 3-Clause
Eclipse Mosquitto	EPL 2.0 / EDL 1.0
Docker	Apache 2.0

Data

Source	License
SRTM Elevation Data	Public Domain (NASA)
Natural Earth / world-atlas	Public Domain

---

License

This project is licensed under MIT — see LICENSE.

Note on dependencies: react-leaflet (Hippocratic License 2.1) and Eclipse Mosquitto (EPL 2.0) are used as dependencies but not modified or redistributed. All other runtime dependencies use MIT, BSD, or Apache 2.0 licenses. The Hippocratic License adds ethical use clauses not present in standard open source licenses.