Compare commits

..

17 Commits

Author SHA1 Message Date
Brent Perteet
e552ae4e5c fix(S2-a): mount broker auth files outside source tree 2026-08-20 15:53:20 -05:00
Brent Perteet
8aea87c3cd fix(S2-a): keep broker credential database outside Git 2026-08-20 15:52:14 -05:00
Brent Perteet
8f638bfa4f fix(S2-a): keep plaintext MQTT inside compose network 2026-08-20 15:44:38 -05:00
Brent Perteet
9ffe021354 fix(S2-a): expose scoped app MQTT through WSS 2026-08-20 15:43:42 -05:00
Brent Perteet
b66ae2cc47 fix(S2-a): load public MQTT TLS cert from broker volume 2026-08-20 15:27:03 -05:00
Brent Perteet
8bcc12ec96 feat(S2-a): ingest durable app MQTT points securely 2026-08-20 15:22:40 -05:00
Brent Perteet
daa3407b9d test: wire jest harness for QA gate (S1-f)
Add jest + ts-jest to the NestJS backend with an app.service smoke spec that
exercises Nest DI. `npm test` is the documented command the QA gate runs.

Trace: SRS §6 gate, NFR-8.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-20 12:53:36 -05:00
ulhub
9eefe50401 Replace Google Maps with Esri (ArcGIS) basemaps
Swaps the map provider behind the existing provider-neutral JobMapProps
interface: EsriJobMap.tsx (ArcGIS Maps SDK) replaces GoogleJobMap.tsx,
loaded via esri-loader's CDN script rather than bundled through webpack —
next dev's inline source-mapping of a library this size was OOM-killing
the whole host on first compile. Also fixes a bug in the fit-bounds camera
call: view.goTo() needs real Graphic/Geometry instances, not plain point
literals, so the map was never zooming to the plotted points.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-16 21:06:05 +00:00
ulhub
5de7b4d7a2 Add BLE challenge-response for short-lived MQTT session certs
BLE-only locators can't hold the MQTT/TLS connection themselves — a phone
relays their data — so handing the phone a device's permanent client-cert
key would export its identity to every phone it pairs with. Instead the
device signs a server-issued nonce with its permanent key over BLE; once
verified, the backend mints a short-lived session certificate for the
phone's actual MQTT connection, keeping the permanent key on-device always.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-13 20:51:58 +00:00
ulhub
842cb23e1f Add device-certificate mTLS auth, live position tracking, and API docs
Introduces a CA/PKI module so field devices can authenticate to Mosquitto
over TLS (8883) with per-device client certificates (CN = serial number)
instead of a shared password, with matching Devices/MQTT-Certs UI. Adds
live transmitter position tracking alongside logged points, an MQTTS
transport option in the simulator for exercising the real cert-auth path,
and Swagger API docs at /api/docs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 01:40:12 +00:00
ulhub
f1c94e9279 Add remote device disable with reason, and a public device status check
Device gains disabledReason (cleared automatically on re-enable). The
devices admin page prompts for a reason when disabling, and shows it
under the device's status once disabled.

New public GET /api/devices/:serial/status lets a field device check
whether it's disabled and why, before any user session exists —
unauthenticated by design, matching the existing serial-based trust
model used for devices/<serial>/log ingestion, and only ever reveals
a boolean plus a short reason string.

The devices/<serial>/log ingest path didn't check isActive at all
(the devices/<mqttUsername>/points path already did) — closed that
gap for both "log" and "status" message types so a disabled device's
data is rejected regardless of which path it arrives on.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 15:11:11 +00:00
ulhub
8c5de40c5d Add live transmitter position (status) alongside logged points
devices/<serial>/log messages now carry a "type": "log" or "status".
"log" persists a LocatePoint as before; "status" carries the same
position + telemetry shape but is broadcast live over the job's
realtime channel without touching locate_points — it's a current-
position update, not a recorded point.

The job map renders the latest status as a blue "you are here" marker
(with a soft accuracy-radius halo) that moves in place as new updates
arrive, separate from the colored polyline of logged points. Clicking
it shows the same detail readout as a logged point.

The simulator gained a message-type toggle (Log point / Status
update) so both paths can be exercised from /sim.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 14:52:53 +00:00
ulhub
e91c91e037 Tie points to locator serial numbers and store receiver telemetry
Adds devices/<serial>/log as a job-anchored MQTT ingest path: a locator
identifies itself by serial in the topic (auto-registered on first
sight, org resolved from the job in the payload) rather than by a
pre-provisioned broker credential. Device.serialNumber is now globally
unique so the bare topic segment is enough to resolve identity.
Locator lookup/auto-register logic is shared (LocatorRegistryService)
between this and the existing devices/<mqttUsername>/points path.

New /sim page (own header, outside the main app nav) simulates a
transmitter: pick an open job or create one, set a serial number and
telemetry defaults, and send points one at a time or on an interval
along a simulated walking path. It calls a new authenticated backend
endpoint (POST /orgs/:orgId/sim/publish) that publishes onto the real
broker rather than writing the DB directly, so the simulator exercises
the actual ingest pipeline end-to-end.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-14 19:41:58 +00:00
ulhub
b7479fa68f Show point details on marker click
Clicking a point on the job map opens an InfoWindow with its full
telemetry readout (utility, fix, recorded time, depth, frequency,
current, signal, gain, locate mode, phase, compass, distortion, GPS
accuracy/satellites/HDOP, altitude, coordinates) — only the fields the
point actually has, since instruments vary in what they report.
Clicking empty map area dismisses it.

MapPoint gained altitude/hAccuracy/vAccuracy/satellites/hdop/phaseDeg,
which the backend already returned but the frontend type omitted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 16:28:07 +00:00
ulhub
21f5a04133 Tie points to locator serial numbers and store receiver telemetry
Field setups often relay through a phone/gateway that owns the MQTT
connection, so the publishing credential and the instrument are now
separate concepts:

- Points messages carry a "serial" identifying the locator receiver;
  unknown serials are auto-registered as devices in the publisher's
  org (name "Locator <serial>"). Device.mqttUsername is now optional
  and serialNumber is unique per org.
- LocatePoint gains standard receiver telemetry: frequencyHz,
  currentMa, signalDb, gainDb, locateMode (PEAK/NULL/BROAD_PEAK/SONDE),
  phaseDeg, compassDeg, distortionPct, plus GPS quality columns
  vAccuracy, satellites, hdop. All optional; raw payload still kept.
- Migration hand-edited to preserve the generated geom column and its
  GIST index (Prisma diff wanted to drop both).
- REST create/point DTOs, map tooltips, job-detail latest-point
  readout, devices settings form, seed, and the sample publisher all
  carry the new fields.
- Added .dockerignore for backend/web: COPY . . was clobbering the
  image's freshly generated Prisma client with the host's stale
  node_modules, breaking image builds after schema changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 14:58:12 +00:00
ulhub
eae4075265 Build core domain: orgs, users, jobs, locate points, auth, live map
Replaces the device-events demo with the actual product:

- Prisma + PostGIS data layer (postgis/postgis:17-3.5). Lat/lng decimals
  are the source of truth; a generated geometry(Point,4326) column with
  a GIST index backs bbox queries. Migrations apply on container boot.
- JWT auth (bcryptjs + httpOnly cookie) with public registration that
  creates an org; per-org roles (ORG_ADMIN/MEMBER/VIEWER) enforced by
  guards on all /orgs/:orgId routes.
- Scoped API keys (X-API-Key, sha256-hashed, shown once) for
  programmatic access, manageable by org admins.
- REST API: jobs/tickets CRUD with filters, points query (time range,
  recordedAt cursor, bbox), members, devices, api-keys.
- MQTT ingest: devices publish to devices/{username}/points and /jobs;
  unknown tickets auto-create stub jobs (source=DEVICE); every message
  is raw-logged to device_events; acks on devices/{username}/jobs/ack.
  Broker gets a dedicated backend user; testuser is now a plain device.
- Realtime: plain-WS gateway at /api/ws (socket.io removed) with
  cookie auth and per-job channels feeding the map live.
- Next.js frontend: login/register, jobs list with filters, job detail
  with live Google map (APWA utility colors, polylines per run) behind
  a provider-neutral JobMap abstraction for a future Esri swap, and
  settings pages for members/devices/api-keys.
- Seed: Umagul org, admin user, testuser device, demo job with RTK
  points. Sample publisher updated to the new topic contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 12:43:25 +00:00
ulhub
cbe0cc6da2 Replace browser MQTT bridge with backend device-events service
The frontend previously connected directly to the MQTT broker via a
CDN-loaded Paho client and a Paho-specific ws proxy. Move that
responsibility into the backend: DeviceDataService persists MQTT
messages to Postgres, and DeviceEventsController exposes them over a
REST endpoint plus a WebSocket gateway that the frontend now consumes
directly. Also adds a pgadmin service for inspecting the database.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-13 23:03:28 +00:00
123 changed files with 24678 additions and 213 deletions

1
.gitignore vendored
View File

@@ -7,3 +7,4 @@ yarn-error.log*
**/node_modules
**/.next
**/dist
mosquitto/certs/

205
MQTT_DEVICE_AUTH.md Normal file
View File

@@ -0,0 +1,205 @@
# MQTT Device Authentication & Authorization
This document describes how devices authenticate to the Mosquitto broker using
client certificates, and how the ACL restricts each device to its own topic
namespace. It covers the same broker that [MQTT_SETUP.md](MQTT_SETUP.md)
describes from a usage/dashboard perspective.
## Overview
The broker (`eclipse-mosquitto`, service `mqtt` / container `ul-hub-mqtt`)
exposes three listeners, each with a different trust model:
| Port | Protocol | Auth | Who it's for |
|------|----------|------|---------------|
| `1883` | MQTT (plaintext, Compose network only) | username/password | internal backend service; not host-published |
| `8883` | MQTT over TLS | **client certificate** | field devices |
| `443` (`/mqtt` → loopback `9001`) | MQTT over WSS/TLS | username/password | scoped app clients |
| `8884` | MQTT over TLS | username/password (server cert only) | scoped app clients and administrators on networks that expose the raw port |
The former anonymous WebSocket listener on `9001` is now authenticated, uses the same ACL as the
TLS listeners, and is bound to host loopback only. Nginx exposes it as WSS at `/mqtt` on port 443;
the web portal itself continues to receive live updates from the backend.
The TLS listeners use `mosquitto/certs/public-fullchain.pem` and
`mosquitto/certs/public-privkey.pem`, copied from the host's Let's Encrypt certificate during
deployment with owner `1883:1883` and mode `0600`. These files are deployment secrets/artifacts
and are excluded from Git.
The live password database is likewise outside Git at
`/home/ubuntu/.config/ul-platform/mosquitto.passwd`, bind-mounted read-only as
`/run/secrets/mosquitto_passwd`. The deployed ACL is copied to
`/home/ubuntu/.config/ul-platform/mosquitto.acl` and bind-mounted beside it. Both files are owned
by broker uid/gid 1883 with mode 0600. Provisioning updates the host password file and restarts Mosquitto; the
tracked `mosquitto/config/passwd` is only a legacy/bootstrap sample and must not receive new
organization credentials.
Device authentication happens on **port 8883**. A device presents a client
certificate signed by the app's own Certificate Authority (CA); Mosquitto
verifies the chain and uses the certificate's Common Name (CN) as the MQTT
username, which the ACL then uses to scope the device to its own topic tree.
## Certificate authority & issuance (Laravel)
All PKI operations are handled in the webapp, not by hand with `openssl` on
the host. See `CertificateController` (`webapp/app/Http/Controllers/CertificateController.php`)
and the `/certificates` admin page (`certificates.index` route,
`webapp/resources/views/certificates/index.blade.php`).
1. **Initialize the root CA**`POST /certificates/ca/init`
Generates a 4096-bit RSA key and a 10-year self-signed cert with
`CN=UL Hub Device CA`. Stored at:
- `webapp/storage/app/private/ca/ca.key` (kept secret, never leaves the server)
- `webapp/storage/app/private/ca/ca.crt`
2. **Issue the MQTT broker's server certificate**`POST /certificates/mqtt/provision {hostname}`
Generates a server key/cert pair signed by the CA (`CN=<hostname>`) and
writes them, along with a copy of `ca.crt`, into
`webapp/storage/app/private/mosquitto-certs/`:
- `ca.crt`, `server.crt`, `server.key` (chmod 644 so the Mosquitto
container, running as a different uid, can read them)
Requires an `mqtt` container restart to pick up.
3. **Issue a device certificate**`POST /certificates {serial_number}`
Generates a 2048-bit RSA key and a CSR with `CN=<SERIAL_NUMBER>`
(uppercased, alphanumeric + hyphens only), signs it with the CA, and
stores the result in the `device_certificates` table
(`DeviceCertificate` model — `serial_number`, `common_name`, `certificate`,
`private_key`, `fingerprint`, `issued_at`, `expires_at`). Certs are valid
for 10 years. The private key is marked `hidden` on the model but is
stored in the DB in plaintext, so DB access is effectively key access.
4. **Distribute the cert/key to a device**:
- Download from the UI (`/certificates/{id}/cert`, `/certificates/{id}/key`)
and the CA cert (`/certificates/ca/download`), or
- `php artisan cert:export <SERIAL_NUMBER> --out=certs` — exports
`<SERIAL>.crt`, `<SERIAL>.key` (chmod 600), and `ca.crt` to a local
directory on the host running artisan.
5. **Revoke a certificate**`DELETE /certificates/{id}` simply deletes the
DB row. **There is no CRL or OCSP** — Mosquitto only checks the cert
against the CA chain and expiry, not against the `device_certificates`
table. A deleted/"revoked" certificate will still authenticate
successfully against the broker until it expires, since revocation is
Laravel-side bookkeeping only, not broker-enforced.
## Broker TLS configuration (port 8883)
From `mosquitto/mosquitto.conf`:
```
listener 8883 0.0.0.0
cafile /mosquitto/certs/ca.crt
certfile /mosquitto/certs/server.crt
keyfile /mosquitto/certs/server.key
require_certificate true
use_identity_as_username true
allow_anonymous false
acl_file /mosquitto/config/devices.acl
```
- `cafile` — the CA used to verify client certs presented by devices (also
serves as the trust anchor for the server's own cert chain).
- `require_certificate true` — TLS handshake fails unless the client
presents a certificate signed by `cafile`.
- `use_identity_as_username true` — the certificate's CN is used directly as
the MQTT username for ACL purposes. No separate password is needed or
accepted on this listener.
- `allow_anonymous false` — belt-and-suspenders; without a valid client
cert, the connection is rejected outright by the TLS handshake anyway.
### Where the runtime files actually come from
`docker-compose.yml` does **not** mount `mosquitto/devices.acl`,
`mosquitto/passwd`, or a local certs folder — it mounts the Laravel-managed
copies instead:
```yaml
volumes:
- ./mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro
- ./webapp/storage/app/private/mosquitto.acl:/mosquitto/config/devices.acl:ro
- ./webapp/storage/app/private/mosquitto.passwd:/mosquitto/config/passwd:ro
- ./webapp/storage/app/private/mosquitto-certs:/mosquitto/certs:ro
```
So `webapp/storage/app/private/mosquitto.acl` and `mosquitto.passwd` (both
generated by `CertificateController::writePasswdFile()`) are the live ACL
and password files — the checked-in `mosquitto/devices.acl` and
`mosquitto/passwd` in the repo root are stale/unused leftovers. Any change to
admin users regenerates these files, but Mosquitto only rereads them on
container restart (`docker compose restart mqtt`) — there's no SIGHUP reload
wired up.
## Topic authorization (ACL)
The ACL file is generated by `writePasswdFile()` in `CertificateController`
and applies to **all three authenticated listeners** (1883, 8883, 8884),
since they all share `acl_file /mosquitto/config/devices.acl`:
```
# Certificate CN becomes the MQTT username — restrict each device to its own namespace.
pattern readwrite devices/%u/#
# App username is orgId; app clients may publish durable points and read acks only in that org.
pattern write ul/%u/app/+/log/points
pattern read ul/%u/app/+/ack
# Admin: <username>
user <username>
topic readwrite #
topic readwrite $SYS/#
```
- **`pattern readwrite devices/%u/#`** applies to every authenticated
client, including devices. `%u` is substituted with the connection's
username at auth time — for a cert-authenticated device on 8883, that's
the certificate's CN (i.e. its serial number). This means:
- A device with `CN=UL-12345` can publish and subscribe to
`devices/UL-12345/#` and nothing else (e.g. `devices/UL-12345/status`,
`devices/UL-12345/telemetry/temp`).
- It **cannot** read or write another device's namespace
(`devices/UL-99999/#`), nor any topic outside `devices/*` (e.g.
`sensor/data`, `$SYS/#`).
- **Admin users** (rows in `mqtt_admin_users`, authenticated by
username/password on 1883 or 8884) get an explicit `user <name>` block
granting `readwrite` on `#` and `$SYS/#` — full access to every topic,
including all devices' namespaces. Mosquitto's `#` wildcard does not match
`$SYS/#`, hence the second explicit line.
- There is currently **no per-device fine-grained restriction** beyond the
serial-number namespace — a device has full read/write on its entire
subtree, so a compromised device credential can, for example, forge its
own "ack" topics or overwrite its own config topics if those live under
the same `devices/<serial>/` prefix.
## Admin users (port 8884 / 1883)
Managed via the same `/certificates` page:
- `POST /certificates/mqtt/admins``storeMqttAdmin` — creates a row in
`mqtt_admin_users` (`MqttAdminUser` model) with a Mosquitto-compatible
PBKDF2 password hash (`$7$<iterations>$<salt>$<hash>`, matching
`mosquitto_passwd`'s format), then rewrites `mosquitto.passwd` and
`mosquitto.acl`.
- Password reset / delete endpoints follow the same pattern, always
rewriting both files afterward.
- Every admin mutation requires `docker compose restart mqtt` to take
effect — the UI messages remind the operator of this each time.
## Summary: who can talk to what
| Client | Listener | Auth | Can publish/subscribe |
|---|---|---|---|
| Field device (cert CN = serial) | 8883 (TLS) | client cert | `devices/<serial>/#` only |
| App (username = orgId) | 8884 (TLS) | scoped per-org username/password | publish `ul/<orgId>/app/+/log/points`; read `ul/<orgId>/app/+/ack` |
| Internal service (e.g. subscriber) | 1883 | username/password | depends on ACL entry for that username — none defined by default beyond `devices/%u/#`, so a plain username with no matching device row is effectively scoped to `devices/<username>/#` too, unless added as an admin |
| Admin | 8884 (TLS) or 1883 | username/password | `#` and `$SYS/#` (everything) |
## Known gaps
- No certificate revocation enforcement at the broker (DB delete ≠ broker
rejection) — mitigate with short-lived certs or a CRL/OCSP setup if this
needs to be production-grade.
- No automatic reload of `mosquitto.conf`/ACL/passwd changes — every
provisioning action requires a manual `docker compose restart mqtt`.
- Sprint 2's per-org app credential permits ack visibility across clients in the same org. This
accepted interim limitation is removed when OIDC-derived per-client broker identity lands.

316
README.md Normal file
View File

@@ -0,0 +1,316 @@
# UlHub
UlHub is a web platform for utility-locating field data. Locator receivers with
high-precision GPS report their position and telemetry over MQTT; the backend
ingests, stores, and streams that data live to a per-job map in the browser.
Core domain: **organizations** have **users** (with roles) and **devices**
(locator receivers). Users create **jobs** ("tickets") — a utility to be
located — either in the web app or from a field device. Locators post
**points** along the located utility to a job, plus a live **status** feed of
their current position. Everything is scoped to an organization; **API keys**
grant scoped, org-limited programmatic access.
## Status
This is an early-stage internal build (not yet hardened for production
exposure — see [Known gaps](#known-gaps)). The core domain, auth, MQTT
ingest, realtime map, and a device simulator are all implemented and
end-to-end verified. Esri (ArcGIS) is the map provider, behind a
provider-neutral interface so a different one could be swapped in later.
## Architecture
```
┌─────────────┐
Browser ───────► │ web (Next) │ ── same-origin proxy ──► backend (Nest)
:3000 └─────────────┘ /api/* , /api/ws :3001
┌───────┴────────┐
│ │
Postgres+PostGIS Mosquitto
(data) (MQTT broker)
▲ ▲
└── backend subscribes to
devices/# and ingests
Locator receivers /
gateways / the /sim tool
```
- **backend/** — NestJS 10 API (REST + WebSocket), Prisma/PostGIS data layer, MQTT
ingest pipeline. Everything lives under the `/api` prefix.
- **web/** — Next.js 14 (pages router). Talks to the backend only through a
same-origin rewrite (`/api/*``http://backend:3001/api/*`), so cookies and
the WebSocket upgrade work without CORS.
- **postgres** — `postgis/postgis:17-3.5`. Schema is managed by Prisma
migrations (`backend/prisma/migrations/`), applied automatically on backend
container boot (`prisma migrate deploy`).
- **mosquitto** — MQTT broker. Devices publish telemetry; the backend is
itself an MQTT client (subscribes to `devices/#`, publishes acks and
simulator messages).
- **pgadmin** — optional DB inspection UI.
- **nginx/** — example reverse-proxy config for a real deployment (terminates
TLS, proxies `/` to the web container). Not used in local dev.
### Data model
Defined in `backend/prisma/schema.prisma`:
| Model | Purpose |
|---|---|
| `Organization` | Tenant boundary. Everything else hangs off an org. |
| `User` / `OrgMembership` | Users can belong to multiple orgs, with a role per org: `ORG_ADMIN`, `MEMBER`, `VIEWER`. |
| `Job` | A locate ticket: ticket number (unique per org), status, address, assignment, source (`WEB` or `DEVICE`). |
| `Device` | A locator receiver and/or MQTT publisher. Identified by a globally-unique `serialNumber`, an `mqttUsername`, or both. Can be remotely disabled with a reason. |
| `LocatePoint` | One recorded reading: lat/lng (high-precision decimals), altitude, GPS quality (fix type, accuracy, satellites, HDOP), and locator telemetry (depth, frequency, current, signal, gain, locate mode, phase, compass, distortion). A generated PostGIS `geometry(Point,4326)` column (`geom`, GIST-indexed) is derived from lat/lng for spatial queries. |
| `ApiKey` | Scoped (`jobs:read`, `points:write`, etc.), org-limited, sha256-hashed, shown once at creation. |
| `DeviceCertificate` | An mTLS client certificate issued to a device (`CN` = serial number) for the broker's 8883 listener. One per device; the CA/server keys themselves live only on disk, never in this table. |
| `DeviceEvent` | Raw log of every MQTT message on `devices/#`, matched or not — an audit/debug trail. |
### Auth
- Email/password, bcrypt-hashed, JWT in an httpOnly `ulhub_token` cookie
(7-day expiry). Registration is public and creates a new org with the
registrant as `ORG_ADMIN`.
- Every domain route is nested under `/api/orgs/:orgId/...` and guarded by
`OrgRolesGuard` (role/membership check) plus, for API-key callers,
`ScopesGuard`.
- API keys authenticate via `X-API-Key: ulh_<random>` and carry a scope list
instead of a role.
### MQTT ingest
Devices publish under `devices/<id>/...`. Two identity schemes coexist:
1. **`devices/<mqttUsername>/points`** and **`.../jobs`** — the publisher
(a gateway/app, possibly relaying several locators) is a pre-registered
`Device` with broker credentials. Points name a `ticket` or `jobId`;
locators within the batch are attributed by an optional `serial` in the
payload. Unknown tickets auto-create a stub job (`source: DEVICE`) so
field data is never dropped ahead of the ticket being opened in the app.
2. **`devices/<serial>/log`** — a locator identifies itself by serial number
directly in the topic (no pre-provisioned broker credential needed); the
org is resolved from the `jobId` in the payload instead. Unknown serials
are auto-registered. Every message carries `"type": "log" | "status"`:
- `log` persists a `LocatePoint` (same shape as above).
- `status` is the same reading shape but is broadcast live over the job's
realtime channel and **never persisted** — it drives the "current
position" blue dot on the map, not the historical point trail.
Every message on `devices/#` is written to `DeviceEvent` regardless of
whether it's understood, for audit purposes. Disabled devices (see below)
are rejected on both ingest paths.
The broker (`mosquitto/config/`) uses a `pattern readwrite devices/%u/#` ACL
so each device's own MQTT username scopes its access; the backend connects
as a dedicated `backend` user with read access to `devices/#` and write
access to ack/log topics.
### Device certificate authentication (mTLS)
Devices identified by serial number can authenticate to a dedicated TLS
listener (port 8883) with a client certificate instead of a shared broker
password. `backend/src/certificates/` acts as a small CA: it shells out to
`openssl` to generate a root CA (once), a broker server certificate, and
per-device client certificates (`CN` = serial number, signed by the CA).
Mosquitto's `use_identity_as_username` turns that `CN` directly into the MQTT
username, so the existing `pattern readwrite devices/%u/#` ACL scopes the
device exactly as it would for a password-authenticated user — no separate
ACL logic needed. Manage it from the "MQTT Certs" settings page (CA
init/broker cert) and each device's "Certificate" action (issue/download/
revoke). See [Known gaps](#known-gaps) for the CA's on-disk storage and
revocation caveats.
### BLE-relayed devices: short-lived session certs
Some locators are BLE-only and have no network stack of their own — a phone
app relays their data, so it's the phone, not the locator, that would open
the MQTT/TLS connection. Handing the phone the locator's *permanent*
client-cert private key (as the mTLS flow above allows any ORG_ADMIN to
download) would export that device's identity to every phone it ever pairs
with, so `backend/src/device-mqtt-auth/` instead uses a challenge-response
handshake that never moves the permanent key off the device:
1. `POST /api/devices/:serial/mqtt-session/challenge` (public, unauthenticated
— same trust model as the device-status check below) returns a one-time
`nonce` and the exact `payload` string
(`ulhub-mqtt-auth-v1:<serial>:<nonce>`) the locator's firmware must sign
with its permanent private key (RSA-SHA256, PKCS#1v1.5) over BLE.
2. `POST /api/devices/:serial/mqtt-session` with `{ nonce, signature }`
(base64) verifies that signature against the device's stored permanent
certificate. On success it mints a **short-lived** client certificate
(`MQTT_SESSION_CERT_HOURS`, default 24h) — same `CN`, so the existing ACL
applies unchanged — and returns it plus the CA cert, for the phone to
connect to the *same* 8883 listener with. On failure: 400 for an
invalid/expired/reused nonce, 403 for a disabled device, 404 for an
unknown serial or a device with no permanent cert yet, 401 for a bad
signature.
Nonces live in memory only (single-use, `MQTT_CHALLENGE_TTL_SECONDS`, default
120s) and session certs are never persisted — Mosquitto validates any
CA-signed cert at connect time regardless of whether the backend remembers
issuing it. Both endpoints are rate-limited (`@nestjs/throttler`, 10
requests/min) since, unlike the read-only device-status check, each one does
real work (an openssl signature verification and/or a fresh cert issuance).
### Realtime
A plain WebSocket gateway at `/api/ws` (not socket.io) authenticates off the
same `ulhub_token` cookie on the upgrade request. Clients subscribe to
per-job channels (`{"type":"subscribe","channel":"job:<id>"}`) and receive
`points` (new logged points) or `status` (live position update) messages,
gated by the same org-membership check as the REST API.
### Device remote disable
A device can be marked disabled (with a free-text reason) from the org's
Devices settings page. Disabled devices' MQTT messages are dropped on
ingest. A field device can self-check via an unauthenticated
`GET /api/devices/:serial/status` — returns whether it's registered,
disabled, and why, so it can show that on its own screen before anyone logs
into the app. (Deliberately public: it reveals nothing beyond a boolean and
a short string, mirroring the serial-based trust model already used for
`.../log` ingestion.)
### Frontend
Next.js pages router, no UI framework (inline styles). Key pieces:
- `lib/auth-context.tsx` — bootstraps session from `/api/auth/me`, tracks the
active org (persisted in `localStorage`), exposes `login`/`register`/`logout`.
- `lib/use-job-stream.ts` — the `/api/ws` client hook (reconnects with
backoff, dispatches `points` vs `status` messages).
- `components/map/` — the map is behind a provider-neutral interface
(`JobMapProps`, `MapPoint`, `LiveStatus`) so swapping providers is a new
component, not a rewrite. `components/map/esri/EsriJobMap.tsx` is the only
file that imports the ArcGIS Maps SDK (`@arcgis/core`): a hybrid
(satellite + labels) basemap, colored markers + polylines per utility type
(APWA color code), a blue "current position" marker with an accuracy halo
for live status, and click-to-inspect detail popups (Esri's built-in
`Popup`, opened natively via each graphic's `popupTemplate`).
- Pages: `login`/`register`, `/` (job list with search/status filter),
`/jobs/new`, `/jobs/[jobId]` (detail + live map), `/settings/{members,
devices,api-keys}`.
### Simulator (`/sim`)
A standalone page (own header, outside the main app nav, but same Next app —
no extra infra) for testing without real hardware: pick an open job or
create one, set a serial number and telemetry defaults, and send points one
at a time or on a timer along a simulated walking path. It calls an
authenticated backend endpoint (`POST /api/orgs/:orgId/sim/publish`) that
**publishes onto the real MQTT broker** rather than writing the database
directly — so it exercises the actual ingest pipeline, not a shortcut around
it. A message-type toggle lets you send either `log` (persisted) or `status`
(live-only) readings.
A second toggle picks the **transport**:
- **HTTP relay** (default) — the backend forwards the message over its own
already-open, privileged broker connection. No device provisioning needed;
good for quick payload testing.
- **MQTTS** — the backend instead opens a real TLS connection to port 8883
and authenticates as the serial's own issued client certificate (see
`SimMqttsService`, `backend/src/sim/sim-mqtts.service.ts`), so the message
is subject to the exact same mTLS handshake and `devices/%u/#` ACL a real
field device would be. Requires a device with that serial number to exist
in the org and have a certificate issued from `/settings/devices` first;
connections are cached per serial so an auto-send session reuses one TLS
connection rather than reconnecting on every publish.
## Repository layout
```
backend/
src/
auth/ JWT + API-key auth, guards, decorators
orgs/ org CRUD, membership management
jobs/ job/ticket CRUD
devices/ device CRUD, remote disable
device-status/ public GET /api/devices/:serial/status
points/ point query/creation REST API
ingest/ MQTT client + routing + point/job/log ingest services
realtime/ WebSocket gateway + pub/sub service
api-keys/ API key issuance/revocation
certificates/ MQTT device mTLS CA (openssl-backed)
device-mqtt-auth/ BLE challenge-response -> short-lived MQTT session certs
sim/ simulator's publish-to-broker endpoint (HTTP relay + real MQTTS/mTLS transport)
prisma/ PrismaService/PrismaModule
prisma/
schema.prisma
migrations/
seed.ts demo org/user/device/job/points
web/
pages/ routes (see above)
components/ Layout, map abstraction
lib/ api client, auth context, WS hook
mosquitto/config/ broker config, passwd, ACL
mosquitto/certs/ CA/server/device certs (gitignored, generated at runtime)
nginx/ example reverse-proxy config for real deployment
test/ Python MQTT test scripts (see test/README.md)
docker-compose.yml
.env.example
```
## Running locally
Requires Docker (no local Node install needed — the containers do
everything).
```bash
cp .env.example .env # fill in JWT_SECRET, MQTT_BACKEND_PASSWORD, NEXT_PUBLIC_ARCGIS_API_KEY
docker compose up -d --build
docker compose exec backend npm run db:seed # optional demo data
```
- Web: http://localhost:3000 (simulator at `/sim`)
- Backend: http://localhost:3001/api
- pgAdmin: http://localhost:5050
Seeded login (if you ran the seed): `brent.perteet@gmail.com` /
`changeme123`, org "umagul", demo device `testuser` / job `TKT-2026-0001`.
Useful commands:
```bash
# create a new Prisma migration after editing schema.prisma
docker compose exec backend npx prisma migrate dev --name <description>
# tail backend logs
docker compose logs -f backend
# publish sample MQTT data from the CLI (alternative to /sim)
python3 test/publish_sample.py
```
## Known gaps
- **Device MQTT credential provisioning is manual for `mqttUsername`
devices.** Creating a device with an `mqttUsername` in the UI doesn't
create real broker credentials — that's still a manual `mosquitto_passwd`
on the mosquitto container. Devices identified by serial number can instead
use the mTLS client-certificate flow below, which is fully self-service.
- **No certificate revocation enforcement at the broker.** The `certificates`
module (`backend/src/certificates/`) acts as a CA for MQTT device client
certs (port 8883, `CN` = serial number, scoped by the existing
`pattern readwrite devices/%u/#` ACL). Revoking a device certificate
(`DELETE .../devices/:deviceId/certificate`) deletes its DB row only —
there's no CRL/OCSP, so the same certificate still authenticates until its
10-year expiry. Short-lived certs or a CRL/OCSP setup would close this.
- **No automatic reload of Mosquitto config or certs.** Initializing the CA,
provisioning the broker's server certificate, or editing `mosquitto.conf`
all require a manual `docker compose restart mosquitto` — there's no
hot-reload.
- **Device private keys are stored in Postgres in plaintext**
(`device_certificates.privateKeyPem`) — DB access is effectively key
access, same tradeoff as most self-hosted device-cert setups without an
HSM.
- **No password reset or org-invite email flow.** Adding a member requires
they've already registered themselves.
- **Public registration.** Anyone can self-register and create a new org;
there's no invite-only mode.
- **`GET /api/devices/:serial/status` is unauthenticated** by design (see
above) — worth revisiting if device identity ever needs to be harder to
spoof.
- No automated test suite yet; `test/` is manual/interactive MQTT scripts.

141
Secure MQTT.md Normal file
View File

@@ -0,0 +1,141 @@
There is **nothing fundamentally wrong** with the approach, provided the devices **private key never leaves the device**. The main complication is that the phone—not the BLE device—is establishing the MQTT/TLS connection.
## The key distinction
The certificate itself is public. Authentication occurs because the client proves possession of the corresponding **private key** by signing part of the TLS handshake. In TLS 1.3, this happens in the `CertificateVerify` step. ([IETF Datatracker][1])
Therefore, simply reading the certificate over BLE and giving it to the phone is insufficient. You need one of these architectures.
### 1. Device acts as a remote TLS signer
The phone performs the MQTT/TLS connection but delegates private-key operations to the BLE device:
```text
MQTT broker
↕ TLS
Phone MQTT/TLS client
↕ BLE signing request
Embedded device/private key
```
During the TLS handshake:
1. Phone sends the device certificate.
2. TLS library generates the `CertificateVerify` data.
3. Phone sends the digest/signing request over BLE.
4. Device signs it using its private key.
5. Device returns only the signature.
6. Phone completes mutual TLS.
This is cryptographically valid and is similar to using a secure element, smart card, TPM, or hardware-backed keystore. Hardware-backed key systems are specifically designed so applications can use a key without extracting it. ([Android Open Source Project][2])
The difficulty is implementation: your Android/iOS MQTT and TLS libraries must support an **external or asynchronous private-key signer**. Many high-level MQTT libraries expect a conventional `PrivateKey` object and do not readily support a BLE-backed signing operation.
### 2. Device signs an application-level authentication challenge
This is usually easier and often cleaner:
```text
Phone → server: Request device-authentication challenge
Server → phone: Random nonce
Phone → BLE device: Sign nonce + context
Device → phone: Signature
Phone → server: Certificate + signature
Server → phone: Short-lived MQTT credential
Phone → MQTT broker: Connect using short-lived credential
```
The short-lived MQTT credential could be:
* A JWT or similar access token
* A temporary username/password
* A temporary MQTT client certificate
* A session credential issued by your backend
I would generally favor this architecture. It lets your backend authenticate the device certificate while the phone uses an ordinary MQTT client library.
Include at least this in the signed data:
```text
protocol_version
device_serial_number
server_nonce
intended_server/audience
timestamp or expiration
requested permissions
phone/app session identifier
```
That prevents the signature from being reused for another server, session, or purpose.
## Principal security concerns
### Do not export the private key
Sending both the certificate and private key to the phone eliminates much of the value of provisioning a unique device credential. A compromised or rooted phone could copy the identity permanently.
Only send signatures from the device.
### Secure the BLE connection
BLE should use authenticated LE Secure Connections, preferably Numeric Comparison, Passkey Entry, or an equivalent product-specific enrollment process. “Just Works” pairing encrypts traffic but does not provide meaningful protection against an active man-in-the-middle during pairing. ([Nordic Developer Academy][3])
For stronger protection, add application-layer security and mutual device/app authentication rather than relying solely on BLE pairing.
### Avoid creating an unrestricted signing oracle
Do not expose a generic BLE command such as:
```text
SIGN arbitrary_bytes
```
A malicious phone application could use the device to sign unrelated material.
Instead expose a narrowly scoped operation:
```text
AUTHENTICATE_MQTT(challenge, broker_id, expiration)
```
The firmware should:
* Validate the request format
* Require an approved broker or backend identifier
* Apply a domain-separation prefix such as `MYPRODUCT-MQTT-AUTH-V1`
* Reject stale challenges
* Rate-limit attempts
* Optionally require that the phone was previously provisioned or bonded
### Broker authorization remains necessary
A valid certificate should identify the device, but it should not automatically grant access to every MQTT topic. Use broker ACLs tied to the authenticated device identity, for example:
```text
devices/DEVICE123/telemetry publish
devices/DEVICE123/commands subscribe
```
MQTT itself recommends using authentication, authorization, and secure communication mechanisms for sensitive systems. ([OASIS Open][4])
## My recommendation
For a phone acting as a passive BLE-to-cloud data ferry, I would use:
```text
Permanent private key: embedded device only
Device certificate: embedded device, readable by phone
Device authentication: signed server challenge over BLE
Phone MQTT credential: short-lived token or temporary certificate
MQTT authorization: device-specific topic ACL
BLE protection: authenticated pairing plus app-level binding
```
That preserves the devices permanent identity, works with normal phone MQTT libraries, supports credential expiration and revocation, and prevents the permanent private key from ever reaching the phone.
Direct BLE-backed TLS signing is also sound, but it is substantially harder to integrate and may tie you to a particular TLS/MQTT implementation.
[1]: https://datatracker.ietf.org/doc/html/rfc8446?utm_source=chatgpt.com "The Transport Layer Security (TLS) Protocol Version 1.3"
[2]: https://source.android.com/docs/security/features/keystore?utm_source=chatgpt.com "Hardware-backed Keystore"
[3]: https://academy.nordicsemi.com/courses/bluetooth-low-energy-fundamentals/lessons/lesson-5-bluetooth-le-security-fundamentals/topic/security-modes/?utm_source=chatgpt.com "Security modes - Nordic Developer Academy"
[4]: https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html?utm_source=chatgpt.com "MQTT Version 5.0 | OASIS Standard"

2
backend/.dockerignore Normal file
View File

@@ -0,0 +1,2 @@
node_modules
dist

View File

@@ -1,8 +1,12 @@
FROM node:20-alpine
# Needed by certificates/pki.service.ts to shell out for CA/device cert generation.
RUN apk add --no-cache openssl
WORKDIR /usr/src/app
COPY package.json package-lock.json* ./
COPY prisma ./prisma
RUN npm install
COPY . .

16
backend/jest.config.js Normal file
View File

@@ -0,0 +1,16 @@
/**
* Jest config for the UlHub backend (NestJS).
* ts-jest transpiles the TypeScript sources; decorators/metadata are honored via tsconfig.
* Specs live next to sources as *.spec.ts (unit) — e2e would go under test/ with a separate config.
*/
module.exports = {
moduleFileExtensions: ['js', 'json', 'ts'],
rootDir: 'src',
testRegex: '.*\\.spec\\.ts$',
transform: {
'^.+\\.ts$': ['ts-jest', { tsconfig: '<rootDir>/../tsconfig.json' }],
},
collectCoverageFrom: ['**/*.(t|j)s'],
coverageDirectory: '../coverage',
testEnvironment: 'node',
};

View File

@@ -1,4 +1,7 @@
{
"collection": "@nestjs/schematics",
"sourceRoot": "src"
"sourceRoot": "src",
"compilerOptions": {
"plugins": ["@nestjs/swagger"]
}
}

8609
backend/package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -6,21 +6,53 @@
"start:dev": "nest start --watch",
"start:prod": "node dist/main",
"build": "nest build",
"format": "prettier --write \"src/**/*.ts\""
"format": "prettier --write \"src/**/*.ts\"",
"postinstall": "prisma generate",
"db:seed": "ts-node prisma/seed.ts",
"test": "jest",
"test:ci": "jest --ci --runInBand"
},
"prisma": {
"seed": "ts-node prisma/seed.ts"
},
"dependencies": {
"@nestjs/common": "^10.0.0",
"@nestjs/core": "^10.0.0",
"@nestjs/jwt": "^10.2.0",
"@nestjs/passport": "^10.0.0",
"@nestjs/platform-express": "^10.0.0",
"reflect-metadata": "^0.1.13"
"@nestjs/platform-ws": "^10.0.0",
"@nestjs/swagger": "^7.4.2",
"@nestjs/throttler": "^5.1.2",
"@nestjs/websockets": "^10.0.0",
"@prisma/client": "^6.10.0",
"bcryptjs": "^2.4.3",
"class-transformer": "^0.5.1",
"class-validator": "^0.14.1",
"cookie-parser": "^1.4.6",
"mqtt": "^5.15.2",
"passport": "^0.7.0",
"passport-custom": "^1.1.1",
"passport-jwt": "^4.0.1",
"reflect-metadata": "^0.1.13",
"rxjs": "^7.0.0",
"ws": "^8.18.0"
},
"devDependencies": {
"@nestjs/cli": "^10.0.0",
"@nestjs/schematics": "^10.0.0",
"@nestjs/testing": "^10.0.0",
"@types/bcryptjs": "^2.4.6",
"@types/cookie-parser": "^1.4.7",
"@types/express": "^4.17.17",
"@types/node": "^20.11.0",
"@types/jest": "^29.5.12",
"@types/passport-jwt": "^4.0.1",
"@types/ws": "^8.5.10",
"jest": "^29.7.0",
"prettier": "^3.0.0",
"ts-jest": "^29.2.5",
"prisma": "^6.10.0",
"ts-node": "^10.9.1",
"typescript": "^5.5.0"
}

View File

@@ -0,0 +1,201 @@
-- CreateSchema
CREATE SCHEMA IF NOT EXISTS "public";
-- PostGIS (required for locate_points.geom)
CREATE EXTENSION IF NOT EXISTS postgis;
-- CreateEnum
CREATE TYPE "OrgRole" AS ENUM ('ORG_ADMIN', 'MEMBER', 'VIEWER');
-- CreateEnum
CREATE TYPE "JobStatus" AS ENUM ('OPEN', 'IN_PROGRESS', 'COMPLETED', 'CANCELLED');
-- CreateEnum
CREATE TYPE "JobSource" AS ENUM ('WEB', 'DEVICE');
-- CreateEnum
CREATE TYPE "UtilityType" AS ENUM ('ELECTRIC', 'GAS', 'WATER', 'SEWER', 'TELECOM', 'CATV', 'FIBER', 'STEAM', 'UNKNOWN');
-- CreateEnum
CREATE TYPE "GpsFixType" AS ENUM ('NONE', 'AUTONOMOUS', 'DGPS', 'FLOAT_RTK', 'FIXED_RTK');
-- CreateTable
CREATE TABLE "organizations" (
"id" TEXT NOT NULL,
"name" TEXT NOT NULL,
"slug" TEXT NOT NULL,
"createdAt" TIMESTAMPTZ(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" TIMESTAMPTZ(6) NOT NULL,
CONSTRAINT "organizations_pkey" PRIMARY KEY ("id")
);
-- CreateTable
CREATE TABLE "users" (
"id" TEXT NOT NULL,
"email" TEXT NOT NULL,
"passwordHash" TEXT NOT NULL,
"name" TEXT NOT NULL,
"createdAt" TIMESTAMPTZ(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" TIMESTAMPTZ(6) NOT NULL,
CONSTRAINT "users_pkey" PRIMARY KEY ("id")
);
-- CreateTable
CREATE TABLE "org_memberships" (
"id" TEXT NOT NULL,
"orgId" TEXT NOT NULL,
"userId" TEXT NOT NULL,
"role" "OrgRole" NOT NULL DEFAULT 'MEMBER',
"createdAt" TIMESTAMPTZ(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "org_memberships_pkey" PRIMARY KEY ("id")
);
-- CreateTable
CREATE TABLE "jobs" (
"id" TEXT NOT NULL,
"orgId" TEXT NOT NULL,
"ticketNumber" TEXT NOT NULL,
"title" TEXT NOT NULL,
"description" TEXT,
"address" TEXT,
"status" "JobStatus" NOT NULL DEFAULT 'OPEN',
"source" "JobSource" NOT NULL DEFAULT 'WEB',
"assignedToId" TEXT,
"createdById" TEXT,
"createdByDeviceId" TEXT,
"dueAt" TIMESTAMPTZ(6),
"startedAt" TIMESTAMPTZ(6),
"completedAt" TIMESTAMPTZ(6),
"createdAt" TIMESTAMPTZ(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" TIMESTAMPTZ(6) NOT NULL,
CONSTRAINT "jobs_pkey" PRIMARY KEY ("id")
);
-- CreateTable
CREATE TABLE "devices" (
"id" TEXT NOT NULL,
"orgId" TEXT NOT NULL,
"name" TEXT NOT NULL,
"serialNumber" TEXT,
"mqttUsername" TEXT NOT NULL,
"isActive" BOOLEAN NOT NULL DEFAULT true,
"lastSeenAt" TIMESTAMPTZ(6),
"createdAt" TIMESTAMPTZ(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" TIMESTAMPTZ(6) NOT NULL,
CONSTRAINT "devices_pkey" PRIMARY KEY ("id")
);
-- CreateTable
CREATE TABLE "locate_points" (
"id" BIGSERIAL NOT NULL,
"jobId" TEXT NOT NULL,
"deviceId" TEXT,
"lat" DECIMAL(10,8) NOT NULL,
"lng" DECIMAL(11,8) NOT NULL,
"altitude" DECIMAL(8,3),
"fixType" "GpsFixType" NOT NULL DEFAULT 'NONE',
"hAccuracy" DECIMAL(7,3),
"depth" DECIMAL(6,3),
"utilityType" "UtilityType" NOT NULL DEFAULT 'UNKNOWN',
"sequence" INTEGER,
"recordedAt" TIMESTAMPTZ(6) NOT NULL,
"receivedAt" TIMESTAMPTZ(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"raw" JSONB,
-- geom is derived from lat/lng and can never drift; Prisma never writes it (Unsupported type)
"geom" geometry(Point, 4326) GENERATED ALWAYS AS (ST_SetSRID(ST_MakePoint("lng"::float8, "lat"::float8), 4326)) STORED,
CONSTRAINT "locate_points_pkey" PRIMARY KEY ("id")
);
-- CreateTable
CREATE TABLE "api_keys" (
"id" TEXT NOT NULL,
"orgId" TEXT NOT NULL,
"name" TEXT NOT NULL,
"keyPrefix" TEXT NOT NULL,
"keyHash" TEXT NOT NULL,
"scopes" TEXT[],
"createdById" TEXT,
"expiresAt" TIMESTAMPTZ(6),
"lastUsedAt" TIMESTAMPTZ(6),
"revokedAt" TIMESTAMPTZ(6),
"createdAt" TIMESTAMPTZ(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "api_keys_pkey" PRIMARY KEY ("id")
);
-- CreateTable
CREATE TABLE "device_events" (
"id" SERIAL NOT NULL,
"topic" TEXT NOT NULL,
"payload" TEXT NOT NULL,
"received_at" TIMESTAMPTZ(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "device_events_pkey" PRIMARY KEY ("id")
);
-- CreateIndex
CREATE UNIQUE INDEX "organizations_slug_key" ON "organizations"("slug");
-- CreateIndex
CREATE UNIQUE INDEX "users_email_key" ON "users"("email");
-- CreateIndex
CREATE UNIQUE INDEX "org_memberships_orgId_userId_key" ON "org_memberships"("orgId", "userId");
-- CreateIndex
CREATE INDEX "jobs_orgId_status_idx" ON "jobs"("orgId", "status");
-- CreateIndex
CREATE UNIQUE INDEX "jobs_orgId_ticketNumber_key" ON "jobs"("orgId", "ticketNumber");
-- CreateIndex
CREATE UNIQUE INDEX "devices_mqttUsername_key" ON "devices"("mqttUsername");
-- CreateIndex
CREATE INDEX "locate_points_jobId_recordedAt_idx" ON "locate_points"("jobId", "recordedAt");
-- CreateIndex (spatial)
CREATE INDEX "locate_points_geom_idx" ON "locate_points" USING GIST ("geom");
-- CreateIndex
CREATE UNIQUE INDEX "api_keys_keyHash_key" ON "api_keys"("keyHash");
-- CreateIndex
CREATE INDEX "api_keys_keyPrefix_idx" ON "api_keys"("keyPrefix");
-- AddForeignKey
ALTER TABLE "org_memberships" ADD CONSTRAINT "org_memberships_orgId_fkey" FOREIGN KEY ("orgId") REFERENCES "organizations"("id") ON DELETE CASCADE ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "org_memberships" ADD CONSTRAINT "org_memberships_userId_fkey" FOREIGN KEY ("userId") REFERENCES "users"("id") ON DELETE CASCADE ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "jobs" ADD CONSTRAINT "jobs_orgId_fkey" FOREIGN KEY ("orgId") REFERENCES "organizations"("id") ON DELETE CASCADE ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "jobs" ADD CONSTRAINT "jobs_assignedToId_fkey" FOREIGN KEY ("assignedToId") REFERENCES "users"("id") ON DELETE SET NULL ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "jobs" ADD CONSTRAINT "jobs_createdById_fkey" FOREIGN KEY ("createdById") REFERENCES "users"("id") ON DELETE SET NULL ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "devices" ADD CONSTRAINT "devices_orgId_fkey" FOREIGN KEY ("orgId") REFERENCES "organizations"("id") ON DELETE CASCADE ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "locate_points" ADD CONSTRAINT "locate_points_jobId_fkey" FOREIGN KEY ("jobId") REFERENCES "jobs"("id") ON DELETE CASCADE ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "locate_points" ADD CONSTRAINT "locate_points_deviceId_fkey" FOREIGN KEY ("deviceId") REFERENCES "devices"("id") ON DELETE SET NULL ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "api_keys" ADD CONSTRAINT "api_keys_orgId_fkey" FOREIGN KEY ("orgId") REFERENCES "organizations"("id") ON DELETE CASCADE ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "api_keys" ADD CONSTRAINT "api_keys_createdById_fkey" FOREIGN KEY ("createdById") REFERENCES "users"("id") ON DELETE SET NULL ON UPDATE CASCADE;

View File

@@ -0,0 +1,25 @@
-- Locator receiver telemetry + device identity by serial number.
-- NOTE: hand-edited — Prisma's diff wanted to drop the geom GIST index and the
-- generated-column expression on locate_points.geom; those statements were removed.
-- CreateEnum
CREATE TYPE "LocateMode" AS ENUM ('PEAK', 'NULL', 'BROAD_PEAK', 'SONDE');
-- AlterTable
ALTER TABLE "devices" ALTER COLUMN "mqttUsername" DROP NOT NULL;
-- AlterTable
ALTER TABLE "locate_points" ADD COLUMN "compassDeg" DECIMAL(5,2),
ADD COLUMN "currentMa" DECIMAL(9,3),
ADD COLUMN "distortionPct" DECIMAL(5,2),
ADD COLUMN "frequencyHz" INTEGER,
ADD COLUMN "gainDb" DECIMAL(6,2),
ADD COLUMN "hdop" DECIMAL(4,2),
ADD COLUMN "locateMode" "LocateMode",
ADD COLUMN "phaseDeg" DECIMAL(6,2),
ADD COLUMN "satellites" INTEGER,
ADD COLUMN "signalDb" DECIMAL(6,2),
ADD COLUMN "vAccuracy" DECIMAL(7,3);
-- CreateIndex
CREATE UNIQUE INDEX "devices_orgId_serialNumber_key" ON "devices"("orgId", "serialNumber");

View File

@@ -0,0 +1,8 @@
-- Locator serial numbers are globally unique (not per-org), so a bare
-- devices/<serial>/log MQTT topic can identify the device without org context.
-- DropIndex
DROP INDEX "devices_orgId_serialNumber_key";
-- CreateIndex
CREATE UNIQUE INDEX "devices_serialNumber_key" ON "devices"("serialNumber");

View File

@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "devices" ADD COLUMN "disabledReason" TEXT;

View File

@@ -0,0 +1,3 @@
-- AlterTable
ALTER TABLE "devices" ADD COLUMN "lastPosition" JSONB,
ADD COLUMN "lastPositionAt" TIMESTAMPTZ(6);

View File

@@ -0,0 +1,20 @@
-- CreateTable
CREATE TABLE "device_certificates" (
"id" TEXT NOT NULL,
"deviceId" TEXT NOT NULL,
"serialNumber" TEXT NOT NULL,
"commonName" TEXT NOT NULL,
"certificatePem" TEXT NOT NULL,
"privateKeyPem" TEXT NOT NULL,
"fingerprint" TEXT NOT NULL,
"issuedAt" TIMESTAMPTZ(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"expiresAt" TIMESTAMPTZ(6) NOT NULL,
CONSTRAINT "device_certificates_pkey" PRIMARY KEY ("id")
);
-- CreateIndex
CREATE UNIQUE INDEX "device_certificates_deviceId_key" ON "device_certificates"("deviceId");
-- AddForeignKey
ALTER TABLE "device_certificates" ADD CONSTRAINT "device_certificates_deviceId_fkey" FOREIGN KEY ("deviceId") REFERENCES "devices"("id") ON DELETE CASCADE ON UPDATE CASCADE;

View File

@@ -0,0 +1,21 @@
-- App-path ingest idempotency + provenance (Sprint 2, S2-a; SRS-SYN-2/7, §3.4.2).
-- Adds the UUIDv7 idempotency key and forward-compat provenance tags to locate_points.
-- CreateEnum
CREATE TYPE "PointOrigin" AS ENUM ('APP', 'LOCATOR', 'MAGLINK');
-- CreateEnum
CREATE TYPE "PointUploadPath" AS ENUM ('APP_MQTT', 'DEVICE_MQTT', 'REST_BATCH');
-- AlterTable
ALTER TABLE "locate_points" ADD COLUMN "pointId" TEXT,
ADD COLUMN "origin" "PointOrigin" NOT NULL DEFAULT 'LOCATOR',
ADD COLUMN "uploadPath" "PointUploadPath",
ADD COLUMN "originClientId" TEXT,
ADD COLUMN "createdAt" TIMESTAMPTZ(6);
-- CreateIndex
CREATE UNIQUE INDEX "locate_points_pointId_key" ON "locate_points"("pointId");
-- CreateIndex
CREATE INDEX "locate_points_jobId_createdAt_idx" ON "locate_points"("jobId", "createdAt");

View File

@@ -0,0 +1,3 @@
# Please do not edit this file manually
# It should be added in your version-control system (e.g., Git)
provider = "postgresql"

View File

@@ -0,0 +1,291 @@
generator client {
provider = "prisma-client-js"
}
datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
}
enum OrgRole {
ORG_ADMIN
MEMBER
VIEWER
}
enum JobStatus {
OPEN
IN_PROGRESS
COMPLETED
CANCELLED
}
enum JobSource {
WEB
DEVICE
}
enum UtilityType {
ELECTRIC
GAS
WATER
SEWER
TELECOM
CATV
FIBER
STEAM
UNKNOWN
}
enum GpsFixType {
NONE
AUTONOMOUS
DGPS
FLOAT_RTK
FIXED_RTK
}
// EM locator receiver antenna mode used when the point was captured
enum LocateMode {
PEAK
NULL
BROAD_PEAK
SONDE
}
// Which producer a point originated from (telemetry-schema.md `origin`). Tagged now
// for SYN-6 path arbitration; no arbitration LOGIC runs yet (deferred, sprint-2 scope).
enum PointOrigin {
APP
LOCATOR
MAGLINK
}
// Which transport carried the point to the cloud (telemetry-schema.md `uploadPath`).
// Same forward-compat tagging as PointOrigin.
enum PointUploadPath {
APP_MQTT
DEVICE_MQTT
REST_BATCH
}
model Organization {
id String @id @default(cuid())
name String
slug String @unique
createdAt DateTime @default(now()) @db.Timestamptz(6)
updatedAt DateTime @updatedAt @db.Timestamptz(6)
memberships OrgMembership[]
jobs Job[]
devices Device[]
apiKeys ApiKey[]
@@map("organizations")
}
model User {
id String @id @default(cuid())
email String @unique
passwordHash String
name String
createdAt DateTime @default(now()) @db.Timestamptz(6)
updatedAt DateTime @updatedAt @db.Timestamptz(6)
memberships OrgMembership[]
assignedJobs Job[] @relation("JobAssignee")
createdJobs Job[] @relation("JobCreator")
apiKeys ApiKey[]
@@map("users")
}
model OrgMembership {
id String @id @default(cuid())
orgId String
userId String
role OrgRole @default(MEMBER)
createdAt DateTime @default(now()) @db.Timestamptz(6)
org Organization @relation(fields: [orgId], references: [id], onDelete: Cascade)
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
@@unique([orgId, userId])
@@map("org_memberships")
}
model Job {
id String @id @default(cuid())
orgId String
ticketNumber String
title String
description String?
address String?
status JobStatus @default(OPEN)
source JobSource @default(WEB)
assignedToId String?
createdById String?
createdByDeviceId String?
dueAt DateTime? @db.Timestamptz(6)
startedAt DateTime? @db.Timestamptz(6)
completedAt DateTime? @db.Timestamptz(6)
createdAt DateTime @default(now()) @db.Timestamptz(6)
updatedAt DateTime @updatedAt @db.Timestamptz(6)
org Organization @relation(fields: [orgId], references: [id], onDelete: Cascade)
assignedTo User? @relation("JobAssignee", fields: [assignedToId], references: [id])
createdBy User? @relation("JobCreator", fields: [createdById], references: [id])
points LocatePoint[]
@@unique([orgId, ticketNumber])
@@index([orgId, status])
@@map("jobs")
}
// A device is either a locator receiver (identified by serialNumber, usually
// auto-registered from incoming data) or an MQTT publisher (app/gateway with
// broker credentials, identified by mqttUsername) — or both, when a locator
// connects to the broker directly.
model Device {
id String @id @default(cuid())
orgId String
name String
// Globally unique (manufacturer serials aren't org-scoped in the real world) —
// this lets a bare devices/<serial>/log MQTT topic identify the device without
// any org context in the topic itself.
serialNumber String? @unique
mqttUsername String? @unique
isActive Boolean @default(true)
// Set (and cleared on re-enable) via the devices admin page; a device
// fetches this via GET /api/devices/:serial/status to show on its own screen.
disabledReason String?
lastSeenAt DateTime? @db.Timestamptz(6)
// Most recent known position, from either a "status" ping (live-only, never
// persisted as a LocatePoint) or a "log" point — whichever is newest. Lets
// the devices page show current position without waiting for a log write.
lastPosition Json?
lastPositionAt DateTime? @db.Timestamptz(6)
createdAt DateTime @default(now()) @db.Timestamptz(6)
updatedAt DateTime @updatedAt @db.Timestamptz(6)
org Organization @relation(fields: [orgId], references: [id], onDelete: Cascade)
points LocatePoint[]
certificate DeviceCertificate?
@@map("devices")
}
// A client certificate issued to a device for mTLS auth on the broker's 8883
// listener; the cert's CN (= serialNumber) becomes the MQTT username. The CA
// key/server key never touch the DB (see PkiService) — only device
// certs/keys are stored here, mirroring how the reference implementation
// (MQTT_DEVICE_AUTH.md) does it. One active cert per device; issuing a new
// one requires deleting this row first. No revokedAt: deleting the row is
// the only "revoke" action there is, and it isn't broker-enforced either way
// (no CRL/OCSP), so a soft-delete flag would misleadingly imply otherwise.
model DeviceCertificate {
id String @id @default(cuid())
deviceId String @unique
serialNumber String
commonName String
certificatePem String @db.Text
privateKeyPem String @db.Text
fingerprint String
issuedAt DateTime @default(now()) @db.Timestamptz(6)
expiresAt DateTime @db.Timestamptz(6)
device Device @relation(fields: [deviceId], references: [id], onDelete: Cascade)
@@map("device_certificates")
}
model LocatePoint {
id BigInt @id @default(autoincrement())
jobId String
deviceId String?
// Client-generated UUIDv7, the idempotency key across every upload path
// (app MQTT, device-direct MQTT, REST batch — SRS §3.4.2 / telemetry-schema.md).
// A replayed UUID is ingested exactly once (unique constraint below). Nullable:
// the legacy device-direct path (LogIngest/PointsIngest) predates it and does not
// supply one yet; the app path (SYN-2) always does.
pointId String? @unique
// Provenance tags (telemetry-schema.md). Set on the app path now; carried for
// forward-compat with SYN-6 arbitration, which does not run yet.
origin PointOrigin @default(LOCATOR)
uploadPath PointUploadPath?
originClientId String?
// Event time asserted by the producer (record creation, telemetry-schema.md
// `createdAt`). Ingest orders and dedups by this, NOT by arrival (`receivedAt`),
// so replays and out-of-order delivery converge to the same stored ordering.
createdAt DateTime? @db.Timestamptz(6)
lat Decimal @db.Decimal(10, 8)
lng Decimal @db.Decimal(11, 8)
altitude Decimal? @db.Decimal(8, 3)
utilityType UtilityType @default(UNKNOWN)
sequence Int?
// GPS quality
fixType GpsFixType @default(NONE)
hAccuracy Decimal? @db.Decimal(7, 3) // meters
vAccuracy Decimal? @db.Decimal(7, 3) // meters
satellites Int?
hdop Decimal? @db.Decimal(4, 2)
// Locator receiver telemetry
depth Decimal? @db.Decimal(6, 3) // meters below grade
frequencyHz Int? // active/passive locate frequency
currentMa Decimal? @db.Decimal(9, 3) // signal current on the line
signalDb Decimal? @db.Decimal(6, 2) // signal strength
gainDb Decimal? @db.Decimal(6, 2) // receiver gain
locateMode LocateMode?
phaseDeg Decimal? @db.Decimal(6, 2)
compassDeg Decimal? @db.Decimal(5, 2) // line direction, 0-360
distortionPct Decimal? @db.Decimal(5, 2)
recordedAt DateTime @db.Timestamptz(6)
receivedAt DateTime @default(now()) @db.Timestamptz(6)
raw Json?
geom Unsupported("geometry(Point, 4326)")?
job Job @relation(fields: [jobId], references: [id], onDelete: Cascade)
device Device? @relation(fields: [deviceId], references: [id], onDelete: SetNull)
// (jobId, recordedAt) serves GPS-time reads; (jobId, createdAt) serves the
// event-time ordering the app path and portal read use (SRS-SYN-2). pointId is
// already uniquely indexed above — that index also backs the dedup lookup.
@@index([jobId, recordedAt])
@@index([jobId, createdAt])
@@map("locate_points")
}
model ApiKey {
id String @id @default(cuid())
orgId String
name String
keyPrefix String
keyHash String @unique
scopes String[]
createdById String?
expiresAt DateTime? @db.Timestamptz(6)
lastUsedAt DateTime? @db.Timestamptz(6)
revokedAt DateTime? @db.Timestamptz(6)
createdAt DateTime @default(now()) @db.Timestamptz(6)
org Organization @relation(fields: [orgId], references: [id], onDelete: Cascade)
createdBy User? @relation(fields: [createdById], references: [id])
@@index([keyPrefix])
@@map("api_keys")
}
// Raw MQTT ingest log (every message on devices/#, matched or not)
model DeviceEvent {
id Int @id @default(autoincrement())
topic String
payload String
receivedAt DateTime @default(now()) @map("received_at") @db.Timestamptz(6)
@@map("device_events")
}

98
backend/prisma/seed.ts Normal file
View File

@@ -0,0 +1,98 @@
import { PrismaClient, GpsFixType, JobStatus, LocateMode, UtilityType } from '@prisma/client';
import * as bcrypt from 'bcryptjs';
const prisma = new PrismaClient();
const SEED_ADMIN_EMAIL = 'brent.perteet@gmail.com';
const SEED_ADMIN_PASSWORD = 'changeme123';
async function main() {
const org = await prisma.organization.upsert({
where: { slug: 'umagul' },
update: {},
create: { name: 'Umagul', slug: 'umagul' },
});
const admin = await prisma.user.upsert({
where: { email: SEED_ADMIN_EMAIL },
update: {},
create: {
email: SEED_ADMIN_EMAIL,
name: 'Brent Perteet',
passwordHash: await bcrypt.hash(SEED_ADMIN_PASSWORD, 12),
},
});
await prisma.orgMembership.upsert({
where: { orgId_userId: { orgId: org.id, userId: admin.id } },
update: { role: 'ORG_ADMIN' },
create: { orgId: org.id, userId: admin.id, role: 'ORG_ADMIN' },
});
const device = await prisma.device.upsert({
where: { mqttUsername: 'testuser' },
update: {},
create: {
orgId: org.id,
name: 'Demo locator',
mqttUsername: 'testuser',
serialNumber: 'DEMO-0001',
},
});
const job = await prisma.job.upsert({
where: { orgId_ticketNumber: { orgId: org.id, ticketNumber: 'TKT-2026-0001' } },
update: {},
create: {
orgId: org.id,
ticketNumber: 'TKT-2026-0001',
title: 'Gas line locate - Main St demo',
description: 'Seeded demo job with sample RTK points along a gas line.',
address: '100 Main St',
status: JobStatus.IN_PROGRESS,
createdById: admin.id,
assignedToId: admin.id,
},
});
const existing = await prisma.locatePoint.count({ where: { jobId: job.id } });
if (existing === 0) {
// A short run of points along a line (roughly northeast, ~1.5m spacing)
const startLat = 33.15012345;
const startLng = -96.83512345;
const points = Array.from({ length: 12 }, (_, i) => ({
jobId: job.id,
deviceId: device.id,
lat: startLat + i * 0.0000135,
lng: startLng + i * 0.0000042,
altitude: 187.4 + i * 0.02,
fixType: GpsFixType.FIXED_RTK,
hAccuracy: 0.014,
vAccuracy: 0.021,
satellites: 22,
hdop: 0.7,
depth: 1.2,
frequencyHz: 33000,
currentMa: 48.5 - i * 0.4,
signalDb: 62.1 - i * 0.3,
gainDb: 40,
locateMode: LocateMode.PEAK,
compassDeg: 17.5,
distortionPct: 4.2,
utilityType: UtilityType.GAS,
sequence: i + 1,
recordedAt: new Date(Date.now() - (12 - i) * 5000),
}));
await prisma.locatePoint.createMany({ data: points });
}
console.log(`Seeded org=${org.slug} admin=${admin.email} device=${device.mqttUsername} job=${job.ticketNumber}`);
console.log(`Admin password: ${SEED_ADMIN_PASSWORD}`);
}
main()
.catch((e) => {
console.error(e);
process.exit(1);
})
.finally(() => prisma.$disconnect());

View File

@@ -0,0 +1,38 @@
import { Body, Controller, Delete, Get, Param, Post, UseGuards } from '@nestjs/common';
import { ApiBearerAuth, ApiTags } from '@nestjs/swagger';
import { CurrentPrincipal } from '../auth/decorators/current-user.decorator';
import { Roles } from '../auth/decorators/roles.decorator';
import { JwtAuthGuard } from '../auth/guards/auth.guard';
import { OrgRolesGuard } from '../auth/guards/org-roles.guard';
import { UserPrincipal } from '../auth/principal';
import { ApiKeysService } from './api-keys.service';
import { CreateApiKeyDto } from './dto/api-keys.dto';
// JWT-only by design: an API key must not be able to mint or revoke API keys
@ApiTags('api-keys')
@ApiBearerAuth('jwt')
@Controller('orgs/:orgId/api-keys')
@UseGuards(JwtAuthGuard, OrgRolesGuard)
@Roles('ORG_ADMIN')
export class ApiKeysController {
constructor(private readonly apiKeysService: ApiKeysService) {}
@Get()
list(@Param('orgId') orgId: string) {
return this.apiKeysService.list(orgId);
}
@Post()
create(
@Param('orgId') orgId: string,
@Body() dto: CreateApiKeyDto,
@CurrentPrincipal() principal: UserPrincipal,
) {
return this.apiKeysService.create(orgId, dto, principal.userId);
}
@Delete(':keyId')
revoke(@Param('orgId') orgId: string, @Param('keyId') keyId: string) {
return this.apiKeysService.revoke(orgId, keyId);
}
}

View File

@@ -0,0 +1,9 @@
import { Module } from '@nestjs/common';
import { ApiKeysController } from './api-keys.controller';
import { ApiKeysService } from './api-keys.service';
@Module({
controllers: [ApiKeysController],
providers: [ApiKeysService],
})
export class ApiKeysModule {}

View File

@@ -0,0 +1,71 @@
import { randomBytes } from 'crypto';
import { Injectable, NotFoundException } from '@nestjs/common';
import { PrismaService } from '../prisma/prisma.service';
import { API_KEY_PREFIX, hashApiKey } from '../auth/strategies/api-key.strategy';
import { CreateApiKeyDto } from './dto/api-keys.dto';
const KEY_SELECT = {
id: true,
name: true,
keyPrefix: true,
scopes: true,
expiresAt: true,
lastUsedAt: true,
revokedAt: true,
createdAt: true,
createdBy: { select: { id: true, name: true, email: true } },
};
function generateKey(): string {
// ulh_ + 40 chars base62
const alphabet = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';
const bytes = randomBytes(40);
let suffix = '';
for (let i = 0; i < 40; i++) {
suffix += alphabet[bytes[i] % alphabet.length];
}
return API_KEY_PREFIX + suffix;
}
@Injectable()
export class ApiKeysService {
constructor(private readonly prisma: PrismaService) {}
list(orgId: string) {
return this.prisma.apiKey.findMany({
where: { orgId },
select: KEY_SELECT,
orderBy: { createdAt: 'desc' },
});
}
async create(orgId: string, dto: CreateApiKeyDto, createdById: string) {
const key = generateKey();
const record = await this.prisma.apiKey.create({
data: {
orgId,
name: dto.name,
keyPrefix: key.slice(0, 12),
keyHash: hashApiKey(key),
scopes: dto.scopes,
expiresAt: dto.expiresAt ? new Date(dto.expiresAt) : undefined,
createdById,
},
select: KEY_SELECT,
});
// The plaintext key is returned exactly once; only the hash is stored.
return { ...record, key };
}
async revoke(orgId: string, keyId: string) {
const record = await this.prisma.apiKey.findFirst({ where: { id: keyId, orgId } });
if (!record) {
throw new NotFoundException('API key not found');
}
await this.prisma.apiKey.update({
where: { id: keyId },
data: { revokedAt: record.revokedAt ?? new Date() },
});
return { ok: true };
}
}

View File

@@ -0,0 +1,22 @@
import { ApiProperty } from '@nestjs/swagger';
import { ArrayMinSize, IsArray, IsDateString, IsIn, IsNotEmpty, IsOptional, IsString, MaxLength } from 'class-validator';
import { API_KEY_SCOPES, ApiKeyScope } from '../../auth/principal';
export class CreateApiKeyDto {
@ApiProperty({ example: 'GIS export' })
@IsString()
@IsNotEmpty()
@MaxLength(120)
name: string;
@ApiProperty({ enum: API_KEY_SCOPES, isArray: true })
@IsArray()
@ArrayMinSize(1)
@IsIn(API_KEY_SCOPES, { each: true })
scopes: ApiKeyScope[];
@ApiProperty({ required: false, format: 'date-time' })
@IsOptional()
@IsDateString()
expiresAt?: string;
}

View File

@@ -1,6 +1,8 @@
import { Controller, Get } from '@nestjs/common';
import { ApiTags } from '@nestjs/swagger';
import { AppService } from './app.service';
@ApiTags('app')
@Controller()
export class AppController {
constructor(private readonly appService: AppService) {}

View File

@@ -2,9 +2,36 @@ import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { StatusController } from './status.controller';
import { PrismaModule } from './prisma/prisma.module';
import { AuthModule } from './auth/auth.module';
import { OrgsModule } from './orgs/orgs.module';
import { JobsModule } from './jobs/jobs.module';
import { PointsModule } from './points/points.module';
import { DevicesModule } from './devices/devices.module';
import { IngestModule } from './ingest/ingest.module';
import { RealtimeModule } from './realtime/realtime.module';
import { ApiKeysModule } from './api-keys/api-keys.module';
import { SimModule } from './sim/sim.module';
import { DeviceStatusModule } from './device-status/device-status.module';
import { CertificatesModule } from './certificates/certificates.module';
import { DeviceMqttAuthModule } from './device-mqtt-auth/device-mqtt-auth.module';
@Module({
imports: [],
imports: [
PrismaModule,
AuthModule,
OrgsModule,
JobsModule,
PointsModule,
DevicesModule,
IngestModule,
RealtimeModule,
ApiKeysModule,
SimModule,
DeviceStatusModule,
CertificatesModule,
DeviceMqttAuthModule,
],
controllers: [AppController, StatusController],
providers: [AppService],
})

View File

@@ -0,0 +1,27 @@
import { Test, TestingModule } from '@nestjs/testing';
import { AppService } from './app.service';
// Smoke test for the QA gate (S1-f, trace: SRS §6 gate / NFR-8).
// Exercises Nest DI wiring end-to-end for a pure service so a green run proves the
// toolchain (ts-jest + @nestjs/testing) is functional. Real coverage grows from here.
describe('AppService (smoke)', () => {
let service: AppService;
beforeEach(async () => {
const moduleRef: TestingModule = await Test.createTestingModule({
providers: [AppService],
}).compile();
service = moduleRef.get<AppService>(AppService);
});
it('is resolvable from the DI container', () => {
expect(service).toBeDefined();
});
it('returns the API welcome payload', () => {
expect(service.getHello()).toEqual({
message: 'Welcome to UlHub API',
docs: 'GET /api',
});
});
});

View File

@@ -0,0 +1,60 @@
import { Body, Controller, Get, HttpCode, Post, Res, UseGuards } from '@nestjs/common';
import { ApiBearerAuth, ApiTags } from '@nestjs/swagger';
import type { Response } from 'express';
import { AuthService } from './auth.service';
import { CurrentPrincipal } from './decorators/current-user.decorator';
import { LoginDto } from './dto/login.dto';
import { RegisterDto } from './dto/register.dto';
import { JwtAuthGuard } from './guards/auth.guard';
import { UserPrincipal } from './principal';
import { AUTH_COOKIE } from './strategies/jwt.strategy';
const COOKIE_MAX_AGE_MS = 7 * 24 * 60 * 60 * 1000; // matches JWT expiry
function setAuthCookie(res: Response, token: string) {
res.cookie(AUTH_COOKIE, token, {
httpOnly: true,
sameSite: 'lax',
secure: process.env.NODE_ENV === 'production',
maxAge: COOKIE_MAX_AGE_MS,
path: '/',
});
}
@ApiTags('auth')
@Controller('auth')
export class AuthController {
constructor(private readonly authService: AuthService) {}
// Public: creates a new organization with the registrant as ORG_ADMIN.
@Post('register')
async register(@Body() dto: RegisterDto, @Res({ passthrough: true }) res: Response) {
const { token, user, memberships } = await this.authService.register(dto);
setAuthCookie(res, token);
return { user, memberships };
}
@Post('login')
@HttpCode(200)
async login(@Body() dto: LoginDto, @Res({ passthrough: true }) res: Response) {
const { token, user, memberships } = await this.authService.login(dto);
setAuthCookie(res, token);
return { user, memberships };
}
@Post('logout')
@HttpCode(200)
@UseGuards(JwtAuthGuard)
@ApiBearerAuth('jwt')
logout(@Res({ passthrough: true }) res: Response) {
res.clearCookie(AUTH_COOKIE, { path: '/' });
return { ok: true };
}
@Get('me')
@UseGuards(JwtAuthGuard)
@ApiBearerAuth('jwt')
me(@CurrentPrincipal() principal: UserPrincipal) {
return this.authService.me(principal.userId);
}
}

View File

@@ -0,0 +1,21 @@
import { Module } from '@nestjs/common';
import { JwtModule } from '@nestjs/jwt';
import { PassportModule } from '@nestjs/passport';
import { AuthController } from './auth.controller';
import { AuthService } from './auth.service';
import { ApiKeyStrategy } from './strategies/api-key.strategy';
import { JwtStrategy } from './strategies/jwt.strategy';
@Module({
imports: [
PassportModule,
JwtModule.register({
secret: process.env.JWT_SECRET || 'dev-only-insecure-secret',
signOptions: { expiresIn: '7d' },
}),
],
controllers: [AuthController],
providers: [AuthService, JwtStrategy, ApiKeyStrategy],
exports: [AuthService],
})
export class AuthModule {}

View File

@@ -0,0 +1,92 @@
import { ConflictException, Injectable, UnauthorizedException } from '@nestjs/common';
import { JwtService } from '@nestjs/jwt';
import * as bcrypt from 'bcryptjs';
import { PrismaService } from '../prisma/prisma.service';
import { LoginDto } from './dto/login.dto';
import { RegisterDto } from './dto/register.dto';
import { JwtPayload } from './strategies/jwt.strategy';
const BCRYPT_ROUNDS = 12;
function slugify(name: string): string {
return name
.toLowerCase()
.trim()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '')
.slice(0, 48) || 'org';
}
@Injectable()
export class AuthService {
constructor(
private readonly prisma: PrismaService,
private readonly jwtService: JwtService,
) {}
async register(dto: RegisterDto) {
const existing = await this.prisma.user.findUnique({ where: { email: dto.email } });
if (existing) {
throw new ConflictException('An account with this email already exists');
}
const passwordHash = await bcrypt.hash(dto.password, BCRYPT_ROUNDS);
const baseSlug = slugify(dto.orgName);
const { user } = await this.prisma.$transaction(async (tx) => {
let slug = baseSlug;
if (await tx.organization.findUnique({ where: { slug } })) {
slug = `${baseSlug}-${Math.random().toString(36).slice(2, 8)}`;
}
const org = await tx.organization.create({ data: { name: dto.orgName, slug } });
const user = await tx.user.create({
data: { email: dto.email, name: dto.name, passwordHash },
});
await tx.orgMembership.create({
data: { orgId: org.id, userId: user.id, role: 'ORG_ADMIN' },
});
return { user, org };
});
return this.sessionFor(user.id);
}
async login(dto: LoginDto) {
const user = await this.prisma.user.findUnique({ where: { email: dto.email } });
if (!user || !(await bcrypt.compare(dto.password, user.passwordHash))) {
throw new UnauthorizedException('Invalid email or password');
}
return this.sessionFor(user.id);
}
async me(userId: string) {
const user = await this.prisma.user.findUniqueOrThrow({
where: { id: userId },
select: { id: true, email: true, name: true, createdAt: true },
});
const memberships = await this.membershipsOf(userId);
return { user, memberships };
}
private async sessionFor(userId: string) {
const user = await this.prisma.user.findUniqueOrThrow({
where: { id: userId },
select: { id: true, email: true, name: true },
});
const payload: JwtPayload = { sub: user.id, email: user.email };
const token = this.jwtService.sign(payload);
const memberships = await this.membershipsOf(userId);
return { token, user, memberships };
}
private membershipsOf(userId: string) {
return this.prisma.orgMembership.findMany({
where: { userId },
select: {
role: true,
org: { select: { id: true, name: true, slug: true } },
},
orderBy: { createdAt: 'asc' },
});
}
}

View File

@@ -0,0 +1,6 @@
import { createParamDecorator, ExecutionContext } from '@nestjs/common';
import { Principal } from '../principal';
export const CurrentPrincipal = createParamDecorator(
(_data: unknown, ctx: ExecutionContext): Principal => ctx.switchToHttp().getRequest().user,
);

View File

@@ -0,0 +1,7 @@
import { SetMetadata } from '@nestjs/common';
import { OrgRole } from '@prisma/client';
export const ROLES_KEY = 'org_role';
// Requires the JWT user to hold AT LEAST this role in the org from :orgId.
export const Roles = (role: OrgRole) => SetMetadata(ROLES_KEY, role);

View File

@@ -0,0 +1,8 @@
import { SetMetadata } from '@nestjs/common';
import { ApiKeyScope } from '../principal';
export const SCOPES_KEY = 'api_key_scopes';
// Requires API-key principals to hold ALL listed scopes. JWT users are unaffected
// (their access is governed by org role instead).
export const RequireScopes = (...scopes: ApiKeyScope[]) => SetMetadata(SCOPES_KEY, scopes);

View File

@@ -0,0 +1,13 @@
import { ApiProperty } from '@nestjs/swagger';
import { IsEmail, IsString, MinLength } from 'class-validator';
export class LoginDto {
@ApiProperty()
@IsEmail()
email: string;
@ApiProperty()
@IsString()
@MinLength(1)
password: string;
}

View File

@@ -0,0 +1,26 @@
import { ApiProperty } from '@nestjs/swagger';
import { IsEmail, IsNotEmpty, IsString, MaxLength, MinLength } from 'class-validator';
export class RegisterDto {
@ApiProperty({ example: 'brent@example.com' })
@IsEmail()
email: string;
@ApiProperty({ minLength: 8, maxLength: 72 })
@IsString()
@MinLength(8)
@MaxLength(72)
password: string;
@ApiProperty({ example: 'Brent Perteet' })
@IsString()
@IsNotEmpty()
@MaxLength(120)
name: string;
@ApiProperty({ description: 'A new organization is created with you as its admin', example: 'Umagul' })
@IsString()
@IsNotEmpty()
@MaxLength(120)
orgName: string;
}

View File

@@ -0,0 +1,10 @@
import { Injectable } from '@nestjs/common';
import { AuthGuard } from '@nestjs/passport';
// JWT-only (auth/me, logout — endpoints that make no sense for API keys)
@Injectable()
export class JwtAuthGuard extends AuthGuard('jwt') {}
// Accepts either a logged-in user (cookie/Bearer JWT) or an X-API-Key.
@Injectable()
export class UserOrApiKeyGuard extends AuthGuard(['jwt', 'api-key']) {}

View File

@@ -0,0 +1,51 @@
import { CanActivate, ExecutionContext, ForbiddenException, Injectable } from '@nestjs/common';
import { Reflector } from '@nestjs/core';
import { OrgRole } from '@prisma/client';
import { PrismaService } from '../../prisma/prisma.service';
import { ROLES_KEY } from '../decorators/roles.decorator';
import { Principal, ROLE_RANK } from '../principal';
// Runs after UserOrApiKeyGuard on org-scoped routes (/orgs/:orgId/...).
// Users: must be a member of the org, with at least the @Roles() role if present.
// API keys: must belong to the org (scopes are checked by ScopesGuard).
@Injectable()
export class OrgRolesGuard implements CanActivate {
constructor(
private readonly reflector: Reflector,
private readonly prisma: PrismaService,
) {}
async canActivate(context: ExecutionContext): Promise<boolean> {
const req = context.switchToHttp().getRequest();
const principal: Principal | undefined = req.user;
const orgId: string | undefined = req.params?.orgId;
if (!principal || !orgId) {
throw new ForbiddenException('Organization scope required');
}
if (principal.type === 'apiKey') {
if (principal.orgId !== orgId) {
throw new ForbiddenException('API key does not belong to this organization');
}
return true;
}
const membership = await this.prisma.orgMembership.findUnique({
where: { orgId_userId: { orgId, userId: principal.userId } },
});
if (!membership) {
throw new ForbiddenException('Not a member of this organization');
}
const required = this.reflector.getAllAndOverride<OrgRole | undefined>(ROLES_KEY, [
context.getHandler(),
context.getClass(),
]);
if (required && ROLE_RANK[membership.role] < ROLE_RANK[required]) {
throw new ForbiddenException(`Requires ${required} role`);
}
req.membership = membership;
return true;
}
}

View File

@@ -0,0 +1,26 @@
import { CanActivate, ExecutionContext, ForbiddenException, Injectable } from '@nestjs/common';
import { Reflector } from '@nestjs/core';
import { SCOPES_KEY } from '../decorators/scopes.decorator';
import { ApiKeyScope, Principal } from '../principal';
@Injectable()
export class ScopesGuard implements CanActivate {
constructor(private readonly reflector: Reflector) {}
canActivate(context: ExecutionContext): boolean {
const principal: Principal | undefined = context.switchToHttp().getRequest().user;
if (!principal || principal.type !== 'apiKey') {
return true;
}
const required = this.reflector.getAllAndOverride<ApiKeyScope[] | undefined>(SCOPES_KEY, [
context.getHandler(),
context.getClass(),
]);
const missing = (required ?? []).filter((s) => !principal.scopes.includes(s));
if (missing.length > 0) {
throw new ForbiddenException(`API key missing scope(s): ${missing.join(', ')}`);
}
return true;
}
}

View File

@@ -0,0 +1,32 @@
import { OrgRole } from '@prisma/client';
export interface UserPrincipal {
type: 'user';
userId: string;
email: string;
}
export interface ApiKeyPrincipal {
type: 'apiKey';
apiKeyId: string;
orgId: string;
scopes: string[];
}
export type Principal = UserPrincipal | ApiKeyPrincipal;
// Rank order used by OrgRolesGuard: a required role means "at least this role".
export const ROLE_RANK: Record<OrgRole, number> = {
VIEWER: 0,
MEMBER: 1,
ORG_ADMIN: 2,
};
export const API_KEY_SCOPES = [
'jobs:read',
'jobs:write',
'points:read',
'points:write',
'devices:read',
] as const;
export type ApiKeyScope = (typeof API_KEY_SCOPES)[number];

View File

@@ -0,0 +1,40 @@
import { createHash } from 'crypto';
import { Injectable, UnauthorizedException } from '@nestjs/common';
import { PassportStrategy } from '@nestjs/passport';
import { Strategy } from 'passport-custom';
import type { Request } from 'express';
import { PrismaService } from '../../prisma/prisma.service';
import { ApiKeyPrincipal } from '../principal';
export const API_KEY_HEADER = 'x-api-key';
export const API_KEY_PREFIX = 'ulh_';
export function hashApiKey(key: string): string {
return createHash('sha256').update(key).digest('hex');
}
@Injectable()
export class ApiKeyStrategy extends PassportStrategy(Strategy, 'api-key') {
constructor(private readonly prisma: PrismaService) {
super();
}
async validate(req: Request): Promise<ApiKeyPrincipal> {
const key = req.headers[API_KEY_HEADER];
if (typeof key !== 'string' || !key.startsWith(API_KEY_PREFIX)) {
throw new UnauthorizedException('Missing or malformed API key');
}
const record = await this.prisma.apiKey.findUnique({ where: { keyHash: hashApiKey(key) } });
if (!record || record.revokedAt || (record.expiresAt && record.expiresAt < new Date())) {
throw new UnauthorizedException('Invalid API key');
}
// fire-and-forget usage stamp; never block the request on it
this.prisma.apiKey
.update({ where: { id: record.id }, data: { lastUsedAt: new Date() } })
.catch(() => undefined);
return { type: 'apiKey', apiKeyId: record.id, orgId: record.orgId, scopes: record.scopes };
}
}

View File

@@ -0,0 +1,29 @@
import { Injectable } from '@nestjs/common';
import { PassportStrategy } from '@nestjs/passport';
import { ExtractJwt, Strategy } from 'passport-jwt';
import { UserPrincipal } from '../principal';
export const AUTH_COOKIE = 'ulhub_token';
export interface JwtPayload {
sub: string;
email: string;
}
@Injectable()
export class JwtStrategy extends PassportStrategy(Strategy, 'jwt') {
constructor() {
super({
jwtFromRequest: ExtractJwt.fromExtractors([
(req) => req?.cookies?.[AUTH_COOKIE] ?? null,
ExtractJwt.fromAuthHeaderAsBearerToken(),
]),
ignoreExpiration: false,
secretOrKey: process.env.JWT_SECRET || 'dev-only-insecure-secret',
});
}
validate(payload: JwtPayload): UserPrincipal {
return { type: 'user', userId: payload.sub, email: payload.email };
}
}

View File

@@ -0,0 +1,82 @@
import { Body, Controller, Delete, Get, Header, Param, Post, UseGuards } from '@nestjs/common';
import { ApiBearerAuth, ApiTags } from '@nestjs/swagger';
import { JwtAuthGuard } from '../auth/guards/auth.guard';
import { OrgRolesGuard } from '../auth/guards/org-roles.guard';
import { Roles } from '../auth/decorators/roles.decorator';
import { CertificatesService } from './certificates.service';
import { ProvisionMqttCertDto } from './dto/certificates.dto';
import { RequireAnyOrgAdminGuard } from './guards/require-any-org-admin.guard';
// Global broker/CA management — not org-nested, since the CA is broker-wide
// (mirrors serialNumber already being globally unique across orgs). JWT-only,
// same precedent as api-keys: credential-minting endpoints never accept an
// API key.
@ApiTags('certificates')
@ApiBearerAuth('jwt')
@Controller('certificates')
@UseGuards(JwtAuthGuard, RequireAnyOrgAdminGuard)
export class CertificatesController {
constructor(private readonly certificatesService: CertificatesService) {}
@Get('ca')
caStatus() {
return this.certificatesService.getCaStatus();
}
@Post('ca/init')
initCa() {
return this.certificatesService.initCa();
}
@Post('mqtt/provision')
provisionMqttCert(@Body() dto: ProvisionMqttCertDto) {
return this.certificatesService.provisionMqttCert(dto);
}
@Get('ca/download')
@Header('Content-Type', 'application/x-pem-file')
@Header('Content-Disposition', 'attachment; filename="ca.crt"')
downloadCa() {
return this.certificatesService.downloadCaCert();
}
}
// Per-device certificate issuance, nested under the org's devices like the
// rest of the domain. Same guard stack as devices create/disable.
@ApiTags('certificates')
@ApiBearerAuth('jwt')
@Controller('orgs/:orgId/devices/:deviceId/certificate')
@UseGuards(JwtAuthGuard, OrgRolesGuard)
@Roles('ORG_ADMIN')
export class DeviceCertificatesController {
constructor(private readonly certificatesService: CertificatesService) {}
@Post()
issue(@Param('orgId') orgId: string, @Param('deviceId') deviceId: string) {
return this.certificatesService.issueDeviceCertificate(orgId, deviceId);
}
@Get()
get(@Param('orgId') orgId: string, @Param('deviceId') deviceId: string) {
return this.certificatesService.getDeviceCertificate(orgId, deviceId);
}
@Get('cert')
@Header('Content-Type', 'application/x-pem-file')
@Header('Content-Disposition', 'attachment; filename="device.crt"')
downloadCert(@Param('orgId') orgId: string, @Param('deviceId') deviceId: string) {
return this.certificatesService.downloadDeviceCert(orgId, deviceId);
}
@Get('key')
@Header('Content-Type', 'application/x-pem-file')
@Header('Content-Disposition', 'attachment; filename="device.key"')
downloadKey(@Param('orgId') orgId: string, @Param('deviceId') deviceId: string) {
return this.certificatesService.downloadDeviceKey(orgId, deviceId);
}
@Delete()
revoke(@Param('orgId') orgId: string, @Param('deviceId') deviceId: string) {
return this.certificatesService.revokeDeviceCertificate(orgId, deviceId);
}
}

View File

@@ -0,0 +1,12 @@
import { Module } from '@nestjs/common';
import { CertificatesController, DeviceCertificatesController } from './certificates.controller';
import { CertificatesService } from './certificates.service';
import { PkiService } from './pki.service';
import { RequireAnyOrgAdminGuard } from './guards/require-any-org-admin.guard';
@Module({
controllers: [CertificatesController, DeviceCertificatesController],
providers: [CertificatesService, PkiService, RequireAnyOrgAdminGuard],
exports: [PkiService],
})
export class CertificatesModule {}

View File

@@ -0,0 +1,113 @@
import { BadRequestException, ConflictException, Injectable, NotFoundException } from '@nestjs/common';
import { PrismaService } from '../prisma/prisma.service';
import { PkiService } from './pki.service';
import { ProvisionMqttCertDto } from './dto/certificates.dto';
const CERT_SELECT = {
id: true,
serialNumber: true,
commonName: true,
fingerprint: true,
issuedAt: true,
expiresAt: true,
};
@Injectable()
export class CertificatesService {
constructor(
private readonly prisma: PrismaService,
private readonly pki: PkiService,
) {}
getCaStatus() {
return this.pki.caStatus();
}
async initCa() {
await this.pki.initCa();
return this.pki.caStatus();
}
async provisionMqttCert(dto: ProvisionMqttCertDto) {
await this.pki.provisionServerCert(dto.hostname);
return { ok: true };
}
async downloadCaCert() {
return this.pki.caCertPem();
}
async issueDeviceCertificate(orgId: string, deviceId: string) {
const device = await this.getDevice(orgId, deviceId);
if (!device.serialNumber) {
throw new BadRequestException('Device needs a serial number before a certificate can be issued');
}
const existing = await this.prisma.deviceCertificate.findUnique({ where: { deviceId } });
if (existing) {
throw new ConflictException('Device already has a certificate — revoke it before issuing a new one');
}
const issued = await this.pki.issueDeviceCert(device.serialNumber);
const record = await this.prisma.deviceCertificate.create({
data: {
deviceId,
serialNumber: device.serialNumber,
commonName: device.serialNumber.toUpperCase(),
certificatePem: issued.certificatePem,
privateKeyPem: issued.privateKeyPem,
fingerprint: issued.fingerprint,
expiresAt: issued.expiresAt,
},
select: CERT_SELECT,
});
return record;
}
async getDeviceCertificate(orgId: string, deviceId: string) {
await this.getDevice(orgId, deviceId);
const cert = await this.prisma.deviceCertificate.findUnique({ where: { deviceId }, select: CERT_SELECT });
if (!cert) {
throw new NotFoundException('Device has no certificate');
}
return cert;
}
async downloadDeviceCert(orgId: string, deviceId: string): Promise<string> {
await this.getDevice(orgId, deviceId);
const cert = await this.prisma.deviceCertificate.findUnique({ where: { deviceId } });
if (!cert) {
throw new NotFoundException('Device has no certificate');
}
return cert.certificatePem;
}
async downloadDeviceKey(orgId: string, deviceId: string): Promise<string> {
await this.getDevice(orgId, deviceId);
const cert = await this.prisma.deviceCertificate.findUnique({ where: { deviceId } });
if (!cert) {
throw new NotFoundException('Device has no certificate');
}
return cert.privateKeyPem;
}
async revokeDeviceCertificate(orgId: string, deviceId: string) {
await this.getDevice(orgId, deviceId);
const cert = await this.prisma.deviceCertificate.findUnique({ where: { deviceId } });
if (!cert) {
throw new NotFoundException('Device has no certificate');
}
// Hard delete: this is DB bookkeeping only, not broker-enforced (no
// CRL/OCSP) — the device's existing cert still authenticates until it
// expires. See README known gaps.
await this.prisma.deviceCertificate.delete({ where: { deviceId } });
return { ok: true };
}
private async getDevice(orgId: string, deviceId: string) {
const device = await this.prisma.device.findFirst({ where: { id: deviceId, orgId } });
if (!device) {
throw new NotFoundException('Device not found');
}
return device;
}
}

View File

@@ -0,0 +1,12 @@
import { ApiProperty } from '@nestjs/swagger';
import { IsNotEmpty, IsString, Matches } from 'class-validator';
export class ProvisionMqttCertDto {
@ApiProperty({ example: 'mqtt.example.com', description: 'Hostname/CN for the broker server certificate' })
@IsString()
@IsNotEmpty()
@Matches(/^[a-zA-Z0-9.-]{1,253}$/, {
message: 'hostname must be 1-253 chars of letters, digits, dot, or dash',
})
hostname: string;
}

View File

@@ -0,0 +1,27 @@
import { CanActivate, ExecutionContext, ForbiddenException, Injectable } from '@nestjs/common';
import { PrismaService } from '../../prisma/prisma.service';
import { Principal } from '../../auth/principal';
// The CA/broker-cert routes are global (not nested under /orgs/:orgId/...),
// so OrgRolesGuard doesn't apply — it hard-requires an :orgId route param.
// Gate: any authenticated user with an ORG_ADMIN membership in *some* org.
@Injectable()
export class RequireAnyOrgAdminGuard implements CanActivate {
constructor(private readonly prisma: PrismaService) {}
async canActivate(context: ExecutionContext): Promise<boolean> {
const req = context.switchToHttp().getRequest();
const principal: Principal | undefined = req.user;
if (!principal || principal.type !== 'user') {
throw new ForbiddenException('Organization admin required');
}
const membership = await this.prisma.orgMembership.findFirst({
where: { userId: principal.userId, role: 'ORG_ADMIN' },
});
if (!membership) {
throw new ForbiddenException('Requires ORG_ADMIN in at least one organization');
}
return true;
}
}

View File

@@ -0,0 +1,265 @@
import { execFile } from 'child_process';
import { existsSync } from 'fs';
import { chmod, mkdir, mkdtemp, readFile, rm, writeFile } from 'fs/promises';
import { tmpdir } from 'os';
import { join } from 'path';
import { promisify } from 'util';
import { BadRequestException, ConflictException, Injectable, InternalServerErrorException, NotFoundException } from '@nestjs/common';
const execFileAsync = promisify(execFile);
const CA_KEY = 'ca.key';
const CA_CERT = 'ca.crt';
const SERVER_KEY = 'server.key';
const SERVER_CERT = 'server.crt';
const CA_DAYS = 3650;
const DEVICE_CERT_DAYS = 3650;
// CN charsets are validated here (not just at the DTO boundary) because they
// are interpolated into openssl's "-subj" string, where a stray "/" or "="
// could inject extra subject fields (e.g. "/CN=x/OU=admin") even though
// execFile's argv-array form already rules out shell injection.
const HOSTNAME_PATTERN = /^[a-zA-Z0-9.-]{1,253}$/;
const SERIAL_PATTERN = /^[A-Z0-9-]{1,64}$/;
export interface CaStatus {
initialized: boolean;
fingerprint?: string;
expiresAt?: Date;
}
export interface IssuedCert {
certificatePem: string;
privateKeyPem: string;
fingerprint: string;
expiresAt: Date;
}
// Acts as the root CA for MQTT device client certificates: shells out to the
// openssl CLI (added to backend/Dockerfile via apk) rather than a JS crypto
// library. The CA/server key pair live only on disk (bind-mounted into the
// mosquitto container at /mosquitto/certs) and are never written to the DB.
@Injectable()
export class PkiService {
private readonly certsDir = process.env.MQTT_CERTS_DIR || '/mosquitto-certs';
private path(name: string): string {
return join(this.certsDir, name);
}
private caExists(): boolean {
return existsSync(this.path(CA_KEY)) && existsSync(this.path(CA_CERT));
}
async caStatus(): Promise<CaStatus> {
if (!this.caExists()) {
return { initialized: false };
}
const [fingerprint, expiresAt] = await Promise.all([
this.fingerprintOf(this.path(CA_CERT)),
this.expiryOf(this.path(CA_CERT)),
]);
return { initialized: true, fingerprint, expiresAt };
}
async initCa(): Promise<void> {
if (this.caExists()) {
throw new ConflictException('CA already initialized');
}
await mkdir(this.certsDir, { recursive: true });
await this.run([
'req',
'-x509',
'-newkey',
'rsa:4096',
'-sha256',
'-days',
String(CA_DAYS),
'-nodes',
'-keyout',
this.path(CA_KEY),
'-out',
this.path(CA_CERT),
'-subj',
'/CN=UlHub Device CA',
]);
}
async provisionServerCert(hostname: string): Promise<void> {
if (!this.caExists()) {
throw new NotFoundException('Initialize the CA before provisioning a broker certificate');
}
if (!HOSTNAME_PATTERN.test(hostname)) {
throw new BadRequestException('Invalid hostname');
}
const dir = await mkdtemp(join(tmpdir(), 'ulhub-mqtt-server-'));
try {
const csrPath = join(dir, 'server.csr');
await this.run([
'req',
'-new',
'-newkey',
'rsa:2048',
'-nodes',
'-keyout',
this.path(SERVER_KEY),
'-out',
csrPath,
'-subj',
`/CN=${hostname}`,
]);
await this.run([
'x509',
'-req',
'-in',
csrPath,
'-CA',
this.path(CA_CERT),
'-CAkey',
this.path(CA_KEY),
'-CAcreateserial',
'-out',
this.path(SERVER_CERT),
'-days',
String(CA_DAYS),
'-sha256',
]);
// Mosquitto runs as its own non-root "mosquitto" user in-container;
// openssl writes -keyout as 0600 (owner-only), which that user can't
// read. The CA key itself stays 0600 (only this service ever reads
// it) — only the broker's own key needs to be world-readable, mirroring
// the reference implementation's documented reasoning exactly.
await chmod(this.path(SERVER_KEY), 0o644);
} finally {
await rm(dir, { recursive: true, force: true });
}
}
async caCertPem(): Promise<string> {
if (!this.caExists()) {
throw new NotFoundException('CA not initialized');
}
return readFile(this.path(CA_CERT), 'utf8');
}
// `days` defaults to the permanent 10-year validity used by the
// ORG_ADMIN-issued device cert; callers minting short-lived session certs
// (see DeviceMqttAuthService) pass a much shorter override.
async issueDeviceCert(serialNumber: string, days: number = DEVICE_CERT_DAYS): Promise<IssuedCert> {
if (!this.caExists()) {
throw new NotFoundException('Initialize the CA before issuing device certificates');
}
const cn = serialNumber.toUpperCase();
if (!SERIAL_PATTERN.test(cn)) {
throw new BadRequestException(
'Device serial number must be 1-64 chars of A-Z, 0-9, or hyphen to be used as a certificate CN',
);
}
const dir = await mkdtemp(join(tmpdir(), 'ulhub-device-cert-'));
try {
const keyPath = join(dir, 'device.key');
const csrPath = join(dir, 'device.csr');
const certPath = join(dir, 'device.crt');
await this.run(['req', '-new', '-newkey', 'rsa:2048', '-nodes', '-keyout', keyPath, '-out', csrPath, '-subj', `/CN=${cn}`]);
await this.run([
'x509',
'-req',
'-in',
csrPath,
'-CA',
this.path(CA_CERT),
'-CAkey',
this.path(CA_KEY),
'-CAcreateserial',
'-out',
certPath,
'-days',
String(Math.max(1, Math.round(days))),
'-sha256',
]);
const [certificatePem, privateKeyPem, fingerprint, expiresAt] = await Promise.all([
readFile(certPath, 'utf8'),
readFile(keyPath, 'utf8'),
this.fingerprintOf(certPath),
this.expiryOf(certPath),
]);
return { certificatePem, privateKeyPem, fingerprint, expiresAt };
} finally {
await rm(dir, { recursive: true, force: true });
}
}
// Verifies that `signatureBase64` is a valid RSA-SHA256 signature over
// `payload`, produced by the private key matching `certificatePem`'s public
// key — i.e. proof of possession without ever seeing the private key
// itself. A non-zero openssl exit here means "signature didn't verify",
// which is an expected outcome, not a server error — so this deliberately
// doesn't go through the shared `run()` helper, which treats any non-zero
// exit as a 500.
async verifySignature(certificatePem: string, payload: string, signatureBase64: string): Promise<boolean> {
const dir = await mkdtemp(join(tmpdir(), 'ulhub-verify-'));
try {
const certPath = join(dir, 'cert.pem');
const pubkeyPath = join(dir, 'pubkey.pem');
const payloadPath = join(dir, 'payload.txt');
const sigPath = join(dir, 'signature.bin');
await writeFile(certPath, certificatePem, 'utf8');
await writeFile(payloadPath, payload, 'utf8');
let signature: Buffer;
try {
signature = Buffer.from(signatureBase64, 'base64');
} catch {
return false;
}
await writeFile(sigPath, signature);
try {
await execFileAsync('openssl', ['x509', '-in', certPath, '-pubkey', '-noout', '-out', pubkeyPath]);
} catch (err: any) {
throw new InternalServerErrorException(`openssl x509 failed: ${err.stderr || err.message}`);
}
try {
await execFileAsync('openssl', ['dgst', '-sha256', '-verify', pubkeyPath, '-signature', sigPath, payloadPath]);
return true;
} catch (err: any) {
// Exit code 1 with "Verification Failure" / "Verification failure" is
// the expected shape of a bad signature. Anything else (malformed
// signature bytes openssl can't even parse, etc.) still resolves to
// "not verified" from the caller's perspective — a forged or garbled
// signature is not verified either way.
return false;
}
} finally {
await rm(dir, { recursive: true, force: true });
}
}
private async fingerprintOf(certPath: string): Promise<string> {
const { stdout } = await this.run(['x509', '-in', certPath, '-noout', '-fingerprint', '-sha256']);
const match = stdout.match(/Fingerprint=([0-9A-Fa-f:]+)/);
return match ? match[1] : stdout.trim();
}
private async expiryOf(certPath: string): Promise<Date> {
const { stdout } = await this.run(['x509', '-in', certPath, '-noout', '-enddate']);
const match = stdout.match(/notAfter=(.+)/);
if (!match) {
throw new InternalServerErrorException('Could not read certificate expiry');
}
return new Date(match[1].trim());
}
private async run(args: string[]): Promise<{ stdout: string; stderr: string }> {
try {
return await execFileAsync('openssl', args);
} catch (err: any) {
throw new InternalServerErrorException(`openssl ${args[0]} failed: ${err.stderr || err.message}`);
}
}
}

View File

@@ -0,0 +1,27 @@
import { Body, Controller, Param, Post, UseGuards } from '@nestjs/common';
import { ApiTags } from '@nestjs/swagger';
import { ThrottlerGuard } from '@nestjs/throttler';
import { RedeemMqttChallengeDto } from './dto/device-mqtt-auth.dto';
import { DeviceMqttAuthService } from './device-mqtt-auth.service';
// Public and unauthenticated by design, same trust model as
// DeviceStatusController: a BLE-only locator has no login of its own, only
// its permanent certificate. Throttled (unlike DeviceStatusController) since
// each request here does real work — an openssl signature verification and,
// on success, a fresh cert issuance — rather than a single indexed read.
@ApiTags('device-mqtt-auth (public)')
@Controller('devices/:serial/mqtt-session')
@UseGuards(ThrottlerGuard)
export class DeviceMqttAuthController {
constructor(private readonly deviceMqttAuth: DeviceMqttAuthService) {}
@Post('challenge')
challenge(@Param('serial') serial: string) {
return this.deviceMqttAuth.issueChallenge(serial);
}
@Post()
redeem(@Param('serial') serial: string, @Body() dto: RedeemMqttChallengeDto) {
return this.deviceMqttAuth.redeemChallenge(serial, dto.nonce, dto.signature);
}
}

View File

@@ -0,0 +1,12 @@
import { Module } from '@nestjs/common';
import { ThrottlerModule } from '@nestjs/throttler';
import { CertificatesModule } from '../certificates/certificates.module';
import { DeviceMqttAuthController } from './device-mqtt-auth.controller';
import { DeviceMqttAuthService } from './device-mqtt-auth.service';
@Module({
imports: [CertificatesModule, ThrottlerModule.forRoot([{ ttl: 60_000, limit: 10 }])],
controllers: [DeviceMqttAuthController],
providers: [DeviceMqttAuthService],
})
export class DeviceMqttAuthModule {}

View File

@@ -0,0 +1,104 @@
import { randomBytes } from 'crypto';
import {
BadRequestException,
ForbiddenException,
Injectable,
NotFoundException,
OnModuleDestroy,
UnauthorizedException,
} from '@nestjs/common';
import { PkiService } from '../certificates/pki.service';
import { PrismaService } from '../prisma/prisma.service';
const CHALLENGE_TTL_MS = Number(process.env.MQTT_CHALLENGE_TTL_SECONDS || 120) * 1000;
const SESSION_CERT_DAYS = Number(process.env.MQTT_SESSION_CERT_HOURS || 24) / 24;
const SWEEP_INTERVAL_MS = 60_000;
interface PendingChallenge {
serial: string;
expiresAt: number;
}
// Lets a BLE-only locator prove possession of its permanent private key
// (issued via PkiService.issueDeviceCert, never leaves the device) through a
// phone relay, without ever exporting that permanent key. The phone gets a
// short-lived session cert instead: mint-on-demand, never persisted, since
// Mosquitto validates any CA-signed cert at connect time regardless of
// whether this service remembers issuing it.
@Injectable()
export class DeviceMqttAuthService implements OnModuleDestroy {
private readonly pending = new Map<string, PendingChallenge>();
private readonly sweep = setInterval(() => this.sweepExpired(), SWEEP_INTERVAL_MS);
constructor(
private readonly prisma: PrismaService,
private readonly pki: PkiService,
) {}
async issueChallenge(serial: string) {
const device = await this.prisma.device.findUnique({ where: { serialNumber: serial } });
if (!device) {
throw new NotFoundException(`No device with serial "${serial}"`);
}
const nonce = randomBytes(32).toString('hex');
const expiresAt = Date.now() + CHALLENGE_TTL_MS;
this.pending.set(nonce, { serial, expiresAt });
return { nonce, payload: this.payloadFor(serial, nonce), expiresAt: new Date(expiresAt) };
}
async redeemChallenge(serial: string, nonce: string, signature: string) {
const entry = this.pending.get(nonce);
// Single-use: consumed here regardless of outcome, so a nonce can never
// be replayed even after a failed attempt.
this.pending.delete(nonce);
if (!entry || entry.expiresAt < Date.now() || entry.serial !== serial) {
throw new BadRequestException('Challenge is invalid, expired, or already used — request a new one');
}
const device = await this.prisma.device.findUnique({ where: { serialNumber: serial } });
if (!device) {
throw new NotFoundException(`No device with serial "${serial}"`);
}
if (!device.isActive) {
throw new ForbiddenException(`Device "${serial}" is disabled`);
}
const cert = await this.prisma.deviceCertificate.findUnique({ where: { deviceId: device.id } });
if (!cert) {
throw new NotFoundException(`Device "${serial}" has no permanent certificate issued yet`);
}
const payload = this.payloadFor(serial, nonce);
const verified = await this.pki.verifySignature(cert.certificatePem, payload, signature);
if (!verified) {
throw new UnauthorizedException('Signature does not match this device\'s certificate');
}
const [issued, caCertPem] = await Promise.all([
this.pki.issueDeviceCert(serial, SESSION_CERT_DAYS),
this.pki.caCertPem(),
]);
return {
certificatePem: issued.certificatePem,
privateKeyPem: issued.privateKeyPem,
caCertPem,
expiresAt: issued.expiresAt,
};
}
private payloadFor(serial: string, nonce: string): string {
return `ulhub-mqtt-auth-v1:${serial}:${nonce}`;
}
private sweepExpired(): void {
const now = Date.now();
for (const [nonce, entry] of this.pending) {
if (entry.expiresAt < now) {
this.pending.delete(nonce);
}
}
}
onModuleDestroy(): void {
clearInterval(this.sweep);
}
}

View File

@@ -0,0 +1,15 @@
import { IsString, Length, Matches } from 'class-validator';
import { ApiProperty } from '@nestjs/swagger';
export class RedeemMqttChallengeDto {
@ApiProperty({ description: 'The nonce returned by POST .../mqtt-session/challenge' })
@IsString()
@Length(64, 64)
@Matches(/^[0-9a-f]{64}$/)
nonce: string;
@ApiProperty({ description: 'Base64-encoded RSA-SHA256 signature over the challenge payload' })
@IsString()
@Length(1, 2048)
signature: string;
}

View File

@@ -0,0 +1,18 @@
import { Controller, Get, Param } from '@nestjs/common';
import { ApiTags } from '@nestjs/swagger';
import { DeviceStatusService } from './device-status.service';
// Public and unauthenticated by design: a field device checks in by serial
// number alone (the same trust model already used for devices/<serial>/log
// MQTT ingestion) before any human has logged it into an org. The response
// only ever reveals a boolean + a short admin-written reason string.
@ApiTags('device-status (public)')
@Controller('devices')
export class DeviceStatusController {
constructor(private readonly deviceStatusService: DeviceStatusService) {}
@Get(':serial/status')
getStatus(@Param('serial') serial: string) {
return this.deviceStatusService.getStatus(serial);
}
}

View File

@@ -0,0 +1,9 @@
import { Module } from '@nestjs/common';
import { DeviceStatusController } from './device-status.controller';
import { DeviceStatusService } from './device-status.service';
@Module({
controllers: [DeviceStatusController],
providers: [DeviceStatusService],
})
export class DeviceStatusModule {}

View File

@@ -0,0 +1,22 @@
import { Injectable } from '@nestjs/common';
import { PrismaService } from '../prisma/prisma.service';
@Injectable()
export class DeviceStatusService {
constructor(private readonly prisma: PrismaService) {}
// A device that has never registered (never sent data, never added in the
// UI) isn't disabled by anyone — treat it as active so first contact works.
async getStatus(serial: string) {
const device = await this.prisma.device.findUnique({
where: { serialNumber: serial },
select: { isActive: true, disabledReason: true },
});
return {
serial,
registered: device !== null,
disabled: device ? !device.isActive : false,
reason: device?.disabledReason ?? null,
};
}
}

View File

@@ -0,0 +1,27 @@
// Most recent known position for a device, from whichever source is newest —
// a live "status" ping (never persisted as a LocatePoint) or a persisted
// "log" point. Stored as JSON on Device.lastPosition.
export interface DevicePositionSnapshot {
lat: number;
lng: number;
altitude: number | null;
utilityType: string;
fixType: string;
hAccuracy: number | null;
vAccuracy: number | null;
satellites: number | null;
hdop: number | null;
depth: number | null;
frequencyHz: number | null;
currentMa: number | null;
signalDb: number | null;
gainDb: number | null;
locateMode: string | null;
phaseDeg: number | null;
compassDeg: number | null;
distortionPct: number | null;
recordedAt: string;
jobId: string;
jobTicketNumber: string;
jobTitle: string;
}

View File

@@ -0,0 +1,53 @@
import { Body, Controller, Delete, Get, Param, Patch, Post, UseGuards } from '@nestjs/common';
import { ApiBearerAuth, ApiSecurity, ApiTags } from '@nestjs/swagger';
import { Roles } from '../auth/decorators/roles.decorator';
import { RequireScopes } from '../auth/decorators/scopes.decorator';
import { UserOrApiKeyGuard } from '../auth/guards/auth.guard';
import { OrgRolesGuard } from '../auth/guards/org-roles.guard';
import { ScopesGuard } from '../auth/guards/scopes.guard';
import { DevicesService } from './devices.service';
import { CreateDeviceDto, UpdateDeviceDto } from './dto/devices.dto';
@ApiTags('devices')
@ApiBearerAuth('jwt')
@ApiSecurity('apiKey')
@Controller('orgs/:orgId/devices')
@UseGuards(UserOrApiKeyGuard, OrgRolesGuard, ScopesGuard)
export class DevicesController {
constructor(private readonly devicesService: DevicesService) {}
@Get()
@RequireScopes('devices:read')
list(@Param('orgId') orgId: string) {
return this.devicesService.list(orgId);
}
@Post()
@Roles('ORG_ADMIN')
@RequireScopes('devices:read') // API keys cannot create devices; role gate handles users
create(@Param('orgId') orgId: string, @Body() dto: CreateDeviceDto) {
return this.devicesService.create(orgId, dto);
}
@Get(':deviceId/location')
@RequireScopes('devices:read')
getLocation(@Param('orgId') orgId: string, @Param('deviceId') deviceId: string) {
return this.devicesService.getLocation(orgId, deviceId);
}
@Patch(':deviceId')
@Roles('ORG_ADMIN')
update(
@Param('orgId') orgId: string,
@Param('deviceId') deviceId: string,
@Body() dto: UpdateDeviceDto,
) {
return this.devicesService.update(orgId, deviceId, dto);
}
@Delete(':deviceId')
@Roles('ORG_ADMIN')
remove(@Param('orgId') orgId: string, @Param('deviceId') deviceId: string) {
return this.devicesService.remove(orgId, deviceId);
}
}

View File

@@ -0,0 +1,12 @@
import { Module } from '@nestjs/common';
import { RealtimeModule } from '../realtime/realtime.module';
import { DevicesController } from './devices.controller';
import { DevicesService } from './devices.service';
@Module({
imports: [RealtimeModule],
controllers: [DevicesController],
providers: [DevicesService],
exports: [DevicesService],
})
export class DevicesModule {}

View File

@@ -0,0 +1,144 @@
import { BadRequestException, ConflictException, Injectable, NotFoundException } from '@nestjs/common';
import { PrismaService } from '../prisma/prisma.service';
import { toPointDto } from '../points/points.service';
import { RealtimeService } from '../realtime/realtime.service';
import { DevicePositionSnapshot } from './device-position';
import { CreateDeviceDto, UpdateDeviceDto } from './dto/devices.dto';
@Injectable()
export class DevicesService {
constructor(
private readonly prisma: PrismaService,
private readonly realtime: RealtimeService,
) {}
list(orgId: string) {
return this.prisma.device.findMany({
where: { orgId },
orderBy: { createdAt: 'asc' },
});
}
async create(orgId: string, dto: CreateDeviceDto) {
if (!dto.mqttUsername && !dto.serialNumber) {
throw new BadRequestException('A device needs a serial number, an MQTT username, or both');
}
if (dto.mqttUsername) {
const existing = await this.prisma.device.findUnique({
where: { mqttUsername: dto.mqttUsername },
});
if (existing) {
throw new ConflictException('A device with this MQTT username already exists');
}
}
if (dto.serialNumber) {
const existing = await this.prisma.device.findUnique({
where: { serialNumber: dto.serialNumber },
});
if (existing) {
throw new ConflictException('A device with this serial number already exists');
}
}
const device = await this.prisma.device.create({
data: { orgId, name: dto.name, mqttUsername: dto.mqttUsername, serialNumber: dto.serialNumber },
});
return {
...device,
provisioning: device.mqttUsername
? {
mqttUsername: device.mqttUsername,
pointsTopic: `devices/${device.mqttUsername}/points`,
jobsTopic: `devices/${device.mqttUsername}/jobs`,
note: 'Broker credentials must be created separately (mosquitto_passwd) until dynamic broker auth lands.',
}
: {
logTopic: device.serialNumber ? `devices/${device.serialNumber}/log` : undefined,
note: `Relayed locator: points published by a gateway should carry "serial": "${device.serialNumber}", or a direct-connect locator can publish single readings to devices/${device.serialNumber}/log.`,
},
};
}
async update(orgId: string, deviceId: string, dto: UpdateDeviceDto) {
await this.get(orgId, deviceId);
const { disabledReason, ...rest } = dto;
const device = await this.prisma.device.update({
where: { id: deviceId },
data: {
...rest,
// A reason only makes sense while disabled; re-enabling always clears it.
...(dto.isActive === true && { disabledReason: null }),
...(dto.isActive === false && { disabledReason: disabledReason ?? null }),
},
});
this.realtime.publish(`org:${orgId}:devices`, {
type: 'device',
orgId,
deviceId,
isActive: device.isActive,
disabledReason: device.disabledReason,
});
return device;
}
async remove(orgId: string, deviceId: string) {
await this.get(orgId, deviceId);
await this.prisma.device.delete({ where: { id: deviceId } });
return { ok: true };
}
async getLocation(orgId: string, deviceId: string) {
const device = await this.get(orgId, deviceId);
const point = await this.prisma.locatePoint.findFirst({
where: { deviceId },
orderBy: { recordedAt: 'desc' },
include: { job: { select: { id: true, ticketNumber: true, title: true } } },
});
// A live "status" ping is never persisted as a LocatePoint, so the most
// recent position may only exist on the device's lastPosition snapshot —
// compare timestamps and use whichever source is actually newer.
const live = device.lastPosition as unknown as DevicePositionSnapshot | null;
if (live && device.lastPositionAt && (!point || device.lastPositionAt > point.recordedAt)) {
return {
point: {
id: `live-${deviceId}`,
lat: live.lat,
lng: live.lng,
altitude: live.altitude,
utilityType: live.utilityType,
fixType: live.fixType,
sequence: null,
recordedAt: live.recordedAt,
hAccuracy: live.hAccuracy,
vAccuracy: live.vAccuracy,
satellites: live.satellites,
hdop: live.hdop,
depth: live.depth,
frequencyHz: live.frequencyHz,
currentMa: live.currentMa,
signalDb: live.signalDb,
gainDb: live.gainDb,
locateMode: live.locateMode,
phaseDeg: live.phaseDeg,
compassDeg: live.compassDeg,
distortionPct: live.distortionPct,
},
job: { id: live.jobId, ticketNumber: live.jobTicketNumber, title: live.jobTitle },
};
}
if (!point) {
return { point: null, job: null };
}
const { job, ...rest } = point;
return { point: toPointDto(rest), job };
}
private async get(orgId: string, deviceId: string) {
const device = await this.prisma.device.findFirst({ where: { id: deviceId, orgId } });
if (!device) {
throw new NotFoundException('Device not found');
}
return device;
}
}

View File

@@ -0,0 +1,57 @@
import { ApiPropertyOptional, ApiProperty } from '@nestjs/swagger';
import { IsBoolean, IsNotEmpty, IsOptional, IsString, Matches, MaxLength } from 'class-validator';
export class CreateDeviceDto {
@ApiProperty()
@IsString()
@IsNotEmpty()
@MaxLength(120)
name: string;
// Broker username (or future TLS cert CN) if this device connects to MQTT
// itself; locators relayed by a gateway need only a serial number.
@ApiPropertyOptional({
description: 'Broker username, only if this device connects to MQTT itself',
pattern: '^[a-zA-Z0-9._-]{3,64}$',
})
@IsOptional()
@IsString()
@Matches(/^[a-zA-Z0-9._-]{3,64}$/, {
message: 'mqttUsername must be 3-64 chars of letters, digits, dot, dash, underscore',
})
mqttUsername?: string;
@ApiPropertyOptional({ description: 'Locator serial number, globally unique' })
@IsOptional()
@IsString()
@MaxLength(64)
serialNumber?: string;
}
export class UpdateDeviceDto {
@ApiPropertyOptional()
@IsOptional()
@IsString()
@IsNotEmpty()
@MaxLength(120)
name?: string;
@ApiPropertyOptional()
@IsOptional()
@IsString()
@MaxLength(120)
serialNumber?: string;
@ApiPropertyOptional({ description: 'Set false to remotely disable the device' })
@IsOptional()
@IsBoolean()
isActive?: boolean;
// Only meaningful when disabling (isActive: false); cleared automatically
// on re-enable regardless of what's passed here.
@ApiPropertyOptional({ description: 'Only used when isActive: false; cleared automatically on re-enable' })
@IsOptional()
@IsString()
@MaxLength(500)
disabledReason?: string;
}

View File

@@ -0,0 +1,315 @@
import { Test } from '@nestjs/testing';
import { readFileSync } from 'fs';
import { resolve } from 'path';
import { PrismaService } from '../prisma/prisma.service';
import { RealtimeService } from '../realtime/realtime.service';
import { AppLogAck } from './dto/app-messages.dto';
import { AppLogIngestService } from './app-log-ingest.service';
import { MqttClientService } from './mqtt-client.service';
// Unit coverage for the durable app-log ingest path (S2-a, SRS-SYN-2/7, §3.4.2).
// Prisma is an in-memory fake keyed on pointId so idempotency is exercised for real
// (a replayed UUID must be stored exactly once), without a database. The broker is a
// spy so we can assert the application-level ack shape and topic.
describe('AppLogIngestService', () => {
const ORG = 'org_alpha';
const CLIENT = 'app_client_1';
const JOB = { id: 'job_1', orgId: ORG, ticketNumber: 'T-1' };
let service: AppLogIngestService;
let published: { topic: string; payload: AppLogAck }[];
let store: Map<string, { pointId: string; jobId: string; createdAt: Date }>;
const point = (
pointId: string,
createdAt: string,
extra: Record<string, unknown> = {},
) => ({
pointId,
createdAt,
origin: 'APP',
uploadPath: 'APP_MQTT',
lat: 40,
lng: -80,
ts: createdAt,
qualityFlag: 'IN_SPEC',
...extra,
});
const msg = (
points: object[],
job: { jobId?: string; ticket?: string } = { jobId: JOB.id },
) => JSON.stringify({ schemaVersion: '1', ...job, points });
const lastAck = () => published[published.length - 1].payload;
beforeEach(async () => {
published = [];
store = new Map();
const prisma = {
organization: {
findUnique: jest.fn(({ where }: { where: { id: string } }) =>
Promise.resolve(where.id === ORG ? { id: ORG } : null),
),
},
job: {
findFirst: jest.fn(
({ where }: { where: { id: string; orgId: string } }) =>
Promise.resolve(
where.id === JOB.id && where.orgId === ORG ? JOB : null,
),
),
findUnique: jest.fn(() => Promise.resolve(null)),
create: jest.fn(() => Promise.resolve(JOB)),
},
locatePoint: {
findMany: jest.fn(
({ where }: { where: { pointId: { in: string[] } } }) =>
Promise.resolve(
where.pointId.in
.filter((id) => store.has(id))
.map((pointId) => ({ pointId })),
),
),
createManyAndReturn: jest.fn(
({
data,
}: {
data: { pointId: string; jobId: string; createdAt: Date }[];
}) => {
const inserted: unknown[] = [];
for (const row of data) {
if (store.has(row.pointId)) {
continue; // skipDuplicates
}
store.set(row.pointId, row);
inserted.push({
...row,
id: BigInt(store.size),
lat: 40,
lng: -80,
altitude: null,
utilityType: 'UNKNOWN',
sequence: null,
fixType: 'NONE',
hAccuracy: null,
vAccuracy: null,
satellites: null,
hdop: null,
depth: null,
frequencyHz: null,
currentMa: null,
signalDb: null,
gainDb: null,
locateMode: null,
phaseDeg: null,
compassDeg: null,
distortionPct: null,
recordedAt: row.createdAt,
receivedAt: row.createdAt,
origin: 'APP',
uploadPath: 'APP_MQTT',
originClientId: CLIENT,
deviceId: null,
});
}
return Promise.resolve(inserted);
},
),
},
};
const mqtt = {
publish: jest.fn((topic: string, payload: AppLogAck) =>
published.push({ topic, payload }),
),
};
const moduleRef = await Test.createTestingModule({
providers: [
AppLogIngestService,
{ provide: PrismaService, useValue: prisma },
{ provide: RealtimeService, useValue: { publish: jest.fn() } },
{ provide: MqttClientService, useValue: mqtt },
],
}).compile();
service = moduleRef.get(AppLogIngestService);
});
it('ingests a fresh point once and acks it accepted on the client namespace', async () => {
await service.handle(
ORG,
CLIENT,
msg([
point('018f1a00-0000-7000-8000-000000000001', '2026-08-21T10:00:00Z'),
]),
);
expect(store.size).toBe(1);
expect(published[0].topic).toBe(`ul/${ORG}/app/${CLIENT}/ack`);
const ack = lastAck();
expect(ack.results).toEqual([
{ pointId: '018f1a00-0000-7000-8000-000000000001', outcome: 'ACCEPTED' },
]);
});
it('consumes the shared app fixture and emits the shared ack fixture', async () => {
const fixtureRoot = resolve(
__dirname,
'../../../../meta/contracts/fixtures',
);
const publishFixture = readFileSync(
resolve(fixtureRoot, 'app-log-points-v1.json'),
'utf8',
);
const expectedAck = JSON.parse(
readFileSync(resolve(fixtureRoot, 'app-log-ack-v1.json'), 'utf8'),
);
await service.handle(ORG, CLIENT, publishFixture);
expect(store.size).toBe(1);
expect(lastAck()).toEqual(expectedAck);
});
it('ingests a replayed UUID exactly once (idempotency) and acks it as duplicate', async () => {
const p = point(
'018f1a00-0000-7000-8000-000000000002',
'2026-08-21T10:00:00Z',
);
await service.handle(ORG, CLIENT, msg([p]));
await service.handle(ORG, CLIENT, msg([p])); // replay
expect(store.size).toBe(1); // stored exactly once
const ack = lastAck();
expect(ack.results).toEqual([
{ pointId: '018f1a00-0000-7000-8000-000000000002', outcome: 'DUPLICATE' },
]);
});
it('orders a batch by event time, not array order', async () => {
const later = point(
'018f1a00-0000-7000-8000-00000000000a',
'2026-08-21T10:05:00Z',
);
const earlier = point(
'018f1a00-0000-7000-8000-00000000000b',
'2026-08-21T10:01:00Z',
);
await service.handle(ORG, CLIENT, msg([later, earlier]));
const rows = [...store.values()];
expect(rows.map((r) => r.pointId)).toEqual([
earlier.pointId,
later.pointId,
]);
});
it('rejects a point whose jobId belongs to another org (cross-namespace backstop)', async () => {
await service.handle(
ORG,
CLIENT,
msg(
[point('018f1a00-0000-7000-8000-00000000000c', '2026-08-21T10:00:00Z')],
{ jobId: 'job_of_org_beta' },
),
);
expect(store.size).toBe(0);
const ack = lastAck();
expect(ack.results).toEqual([
{
pointId: '018f1a00-0000-7000-8000-00000000000c',
outcome: 'REJECTED',
reasonCode: 'UNKNOWN_JOB_OR_WRONG_ORG',
},
]);
});
it('drops (no ack) a message on an unknown org namespace', async () => {
await service.handle(
'org_ghost',
CLIENT,
msg([
point('018f1a00-0000-7000-8000-00000000000d', '2026-08-21T10:00:00Z'),
]),
);
expect(published).toHaveLength(0);
expect(store.size).toBe(0);
});
it('does not emit an ambiguous ack when malformed JSON has no usable pointId', async () => {
await service.handle(ORG, CLIENT, '{ not json');
expect(published).toHaveLength(0);
});
it('rejects a schema-invalid point but names its UUID in the ack', async () => {
// Missing required createdAt + bad lat → validation fails; pointId still surfaced.
await service.handle(
ORG,
CLIENT,
msg([
{
pointId: '018f1a00-0000-7000-8000-00000000000e',
lat: 999,
lng: 0,
ts: '2026-08-21T10:00:00Z',
},
]),
);
const ack = lastAck();
expect(store.size).toBe(0);
expect(ack.results).toEqual([
{
pointId: '018f1a00-0000-7000-8000-00000000000e',
outcome: 'REJECTED',
reasonCode: 'VALIDATION_ERROR',
},
]);
});
it('accepts a valid point while rejecting a malformed neighbor in the same batch', async () => {
const bad = point(
'018f1a00-0000-7000-8000-00000000000f',
'2026-08-21T10:00:00Z',
{ lat: 999 },
);
const good = point(
'018f1a00-0000-7000-8000-000000000010',
'2026-08-21T10:00:01Z',
);
await service.handle(ORG, CLIENT, msg([bad, good]));
expect(store.size).toBe(1);
expect(lastAck().results).toEqual(
expect.arrayContaining([
{
pointId: bad.pointId,
outcome: 'REJECTED',
reasonCode: 'VALIDATION_ERROR',
},
{ pointId: good.pointId, outcome: 'ACCEPTED' },
]),
);
});
it('rejects a non-v7 UUID even when it is otherwise a valid UUID', async () => {
const uuidV4 = '550e8400-e29b-41d4-a716-446655440000';
await service.handle(
ORG,
CLIENT,
msg([point(uuidV4, '2026-08-21T10:00:00Z')]),
);
expect(store.size).toBe(0);
expect(lastAck().results).toEqual([
{
pointId: uuidV4,
outcome: 'REJECTED',
reasonCode: 'POINT_ID_NOT_UUIDV7',
},
]);
});
});

View File

@@ -0,0 +1,277 @@
import { Injectable, Logger } from '@nestjs/common';
import { GpsFixType, Prisma } from '@prisma/client';
import { plainToInstance } from 'class-transformer';
import { validate } from 'class-validator';
import { toPointDto } from '../points/points.service';
import { PrismaService } from '../prisma/prisma.service';
import { RealtimeService } from '../realtime/realtime.service';
import {
AppFixType,
AppLogAck,
AppLogAckResult,
AppLogPointDto,
AppLogPointsMessageDto,
} from './dto/app-messages.dto';
import { MqttClientService } from './mqtt-client.service';
const SCHEMA_VERSION = '1';
const UUID_V7 =
/^[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
// Durable app-originated ingest for ul/{orgId}/app/{clientId}/log/points. The exact v1
// publish/ack envelopes are frozen in meta/contracts/telemetry-schema.md.
@Injectable()
export class AppLogIngestService {
private readonly logger = new Logger(AppLogIngestService.name);
constructor(
private readonly prisma: PrismaService,
private readonly realtime: RealtimeService,
private readonly mqttClient: MqttClientService,
) {}
async handle(
orgId: string,
clientId: string,
rawPayload: string,
): Promise<void> {
const org = await this.prisma.organization.findUnique({
where: { id: orgId },
select: { id: true },
});
if (!org) {
this.logger.warn(
`app-log for unknown org "${orgId}" (client ${clientId}) — dropped`,
);
return;
}
let parsed: unknown;
try {
parsed = JSON.parse(rawPayload);
} catch {
// There is no trustworthy pointId to acknowledge. Raw app payloads are redacted by the
// router; retain/log on the app side rather than sending a null/ambiguous acknowledgement.
this.logger.warn(
`invalid JSON from app ${orgId}/${clientId} — no acknowledgement emitted`,
);
return;
}
const msg = plainToInstance(AppLogPointsMessageDto, parsed as object);
const rawPoints = Array.isArray((parsed as { points?: unknown[] })?.points)
? (parsed as { points: unknown[] }).points
: [];
const usableIds = rawPoints
.map((value) =>
typeof (value as { pointId?: unknown })?.pointId === 'string'
? (value as { pointId: string }).pointId
: null,
)
.filter((value): value is string => value !== null);
const hasJobId = typeof msg.jobId === 'string' && msg.jobId.length > 0;
const hasTicket = typeof msg.ticket === 'string' && msg.ticket.length > 0;
const envelopeValid =
msg.schemaVersion === SCHEMA_VERSION &&
rawPoints.length > 0 &&
rawPoints.length <= 500 &&
hasJobId !== hasTicket;
if (!envelopeValid) {
this.publishAck(
orgId,
clientId,
usableIds.map((pointId) => ({
pointId,
outcome: 'REJECTED',
reasonCode: 'ENVELOPE_VALIDATION_ERROR',
})),
);
return;
}
// Validate independently so one malformed capture does not discard valid captures in the
// same batch (LOG-7). Invalid UUID strings can still be named in a rejection; missing IDs
// cannot be safely correlated and therefore remain unacknowledged.
const results: AppLogAckResult[] = [];
const candidates: AppLogPointDto[] = [];
for (const rawPoint of rawPoints) {
const point = plainToInstance(AppLogPointDto, rawPoint as object);
const pointId = typeof point.pointId === 'string' ? point.pointId : null;
const errors = await validate(point, { whitelist: true });
if (!pointId) {
continue;
}
if (errors.length > 0 || !UUID_V7.test(pointId)) {
results.push({
pointId,
outcome: 'REJECTED',
reasonCode: UUID_V7.test(pointId)
? 'VALIDATION_ERROR'
: 'POINT_ID_NOT_UUIDV7',
});
continue;
}
candidates.push(point);
}
const job = await this.resolveJob(orgId, clientId, msg);
if (!job) {
results.push(
...candidates.map((point) => ({
pointId: point.pointId,
outcome: 'REJECTED' as const,
reasonCode: 'UNKNOWN_JOB_OR_WRONG_ORG',
})),
);
this.publishAck(orgId, clientId, results);
return;
}
// Collapse duplicates inside this delivery, then use producer event time rather than
// arrival/array order. The first valid occurrence wins; repeated IDs are safe to release.
const byPointId = new Map<string, AppLogPointDto>();
for (const point of candidates) {
if (byPointId.has(point.pointId)) {
results.push({
pointId: point.pointId,
outcome: 'DUPLICATE',
reasonCode: 'DUPLICATE_IN_BATCH',
});
} else {
byPointId.set(point.pointId, point);
}
}
const ordered = [...byPointId.values()].sort(
(a, b) => +new Date(a.createdAt) - +new Date(b.createdAt),
);
const seen = ordered.length
? await this.prisma.locatePoint.findMany({
where: { pointId: { in: ordered.map((point) => point.pointId) } },
select: { pointId: true },
})
: [];
const seenIds = new Set(seen.map((row) => row.pointId));
const fresh = ordered.filter((point) => !seenIds.has(point.pointId));
for (const point of ordered.filter((candidate) =>
seenIds.has(candidate.pointId),
)) {
results.push({ pointId: point.pointId, outcome: 'DUPLICATE' });
}
let stored: Awaited<
ReturnType<typeof this.prisma.locatePoint.createManyAndReturn>
> = [];
if (fresh.length > 0) {
stored = await this.prisma.locatePoint.createManyAndReturn({
data: fresh.map((point) => ({
jobId: job.id,
deviceId: null,
pointId: point.pointId,
origin: 'APP' as const,
uploadPath: 'APP_MQTT' as const,
originClientId: clientId,
createdAt: new Date(point.createdAt),
lat: point.lat,
lng: point.lng,
altitude: point.alt,
utilityType: point.utility,
sequence: point.seq,
fixType: this.toStoredFix(point.fix),
hAccuracy: point.hAcc,
vAccuracy: point.vAcc,
satellites: point.sats,
hdop: point.hdop,
depth: point.depth,
frequencyHz: point.freqHz,
currentMa: point.currentMa,
signalDb: point.signalDb,
gainDb: point.gainDb,
locateMode: point.mode,
phaseDeg: point.phaseDeg,
compassDeg: point.compassDeg,
distortionPct: point.distortionPct,
recordedAt: new Date(point.ts),
raw: point as unknown as Prisma.InputJsonValue,
})),
skipDuplicates: true,
});
}
const storedIds = new Set(stored.map((row) => row.pointId));
for (const point of fresh) {
results.push({
pointId: point.pointId,
outcome: storedIds.has(point.pointId) ? 'ACCEPTED' : 'DUPLICATE',
});
}
if (stored.length > 0) {
this.realtime.publish(`job:${job.id}`, {
type: 'points',
jobId: job.id,
points: stored.map(toPointDto),
});
}
this.publishAck(orgId, clientId, results);
}
private async resolveJob(
orgId: string,
clientId: string,
msg: AppLogPointsMessageDto,
) {
if (msg.jobId) {
const job = await this.prisma.job.findFirst({
where: { id: msg.jobId, orgId },
});
if (!job)
this.logger.warn(
`app-log ${orgId}/${clientId} referenced a foreign/unknown job ${msg.jobId}`,
);
return job;
}
const existing = await this.prisma.job.findUnique({
where: { orgId_ticketNumber: { orgId, ticketNumber: msg.ticket! } },
});
if (existing) return existing;
return this.prisma.job.create({
data: {
orgId,
ticketNumber: msg.ticket!,
title: `Ticket ${msg.ticket} (app-created)`,
status: 'IN_PROGRESS',
source: 'WEB',
startedAt: new Date(),
},
});
}
private toStoredFix(fix?: AppFixType): GpsFixType | undefined {
switch (fix) {
case 'FLOAT':
return GpsFixType.FLOAT_RTK;
case 'FIXED':
return GpsFixType.FIXED_RTK;
case 'NO_FIX':
return GpsFixType.NONE;
case 'AUTONOMOUS':
return GpsFixType.AUTONOMOUS;
case 'DGPS':
return GpsFixType.DGPS;
default:
return undefined;
}
}
private publishAck(
orgId: string,
clientId: string,
results: AppLogAckResult[],
): void {
if (results.length === 0) return;
const ack: AppLogAck = { schemaVersion: SCHEMA_VERSION, results };
this.mqttClient.publish(`ul/${orgId}/app/${clientId}/ack`, ack);
}
}

View File

@@ -0,0 +1,189 @@
import { LocateMode, UtilityType } from '@prisma/client';
import { ApiProperty, ApiPropertyOptional } from '@nestjs/swagger';
import { Type } from 'class-transformer';
import {
ArrayMaxSize,
ArrayMinSize,
IsArray,
IsDateString,
IsEnum,
IsIn,
IsInt,
IsNumber,
IsOptional,
IsString,
IsUUID,
Max,
MaxLength,
Min,
ValidateNested,
} from 'class-validator';
export const APP_FIX_TYPES = [
'AUTONOMOUS',
'DGPS',
'FLOAT',
'FIXED',
'NO_FIX',
] as const;
export type AppFixType = (typeof APP_FIX_TYPES)[number];
// Frozen app MQTT wire profile v1 from meta/contracts/telemetry-schema.md. This is deliberately
// separate from MqttPointDto: the app wire uses the normative fix tokens FLOAT/FIXED/NO_FIX,
// while the legacy device path uses Prisma's FLOAT_RTK/FIXED_RTK/NONE storage tokens.
export class AppLogPointDto {
@ApiProperty({
format: 'uuid',
description: 'Client-generated UUIDv7 idempotency key.',
})
@IsUUID()
pointId: string;
@ApiProperty({ format: 'date-time' })
@IsDateString()
createdAt: string;
@IsIn(['APP'])
origin: 'APP';
@IsIn(['APP_MQTT'])
uploadPath: 'APP_MQTT';
@IsNumber()
@Min(-90)
@Max(90)
lat: number;
@IsNumber()
@Min(-180)
@Max(180)
lng: number;
@IsOptional()
@IsNumber()
alt?: number;
@IsDateString()
ts: string;
@IsOptional()
@IsIn(APP_FIX_TYPES)
fix?: AppFixType;
@IsOptional()
@IsNumber()
@Min(0)
hAcc?: number;
@IsOptional()
@IsNumber()
@Min(0)
vAcc?: number;
@IsOptional()
@IsInt()
@Min(0)
sats?: number;
@IsOptional()
@IsNumber()
@Min(0)
hdop?: number;
@IsOptional()
@IsNumber()
@Min(0)
depth?: number;
@IsOptional()
@IsInt()
@Min(0)
freqHz?: number;
@IsOptional()
@IsNumber()
@Min(0)
currentMa?: number;
@IsOptional()
@IsNumber()
signalDb?: number;
@IsOptional()
@IsNumber()
gainDb?: number;
@IsOptional()
@IsEnum(LocateMode)
mode?: LocateMode;
@IsOptional()
@IsNumber()
phaseDeg?: number;
@IsOptional()
@IsNumber()
@Min(0)
@Max(360)
compassDeg?: number;
@IsOptional()
@IsNumber()
@Min(0)
@Max(100)
distortionPct?: number;
@IsOptional()
@IsEnum(UtilityType)
utility?: UtilityType;
@IsOptional()
@IsInt()
seq?: number;
@IsIn(['IN_SPEC', 'OUT_OF_SPEC', 'MANUAL', 'PPK_CORRECTED', 'WAIVED', 'NONCOMPLIANT'])
qualityFlag: 'IN_SPEC' | 'OUT_OF_SPEC' | 'MANUAL' | 'PPK_CORRECTED' | 'WAIVED' | 'NONCOMPLIANT';
}
export class AppLogPointsMessageDto {
@ApiPropertyOptional({ description: 'MQTT app-log wire profile version.' })
@IsString()
@MaxLength(16)
schemaVersion: string;
@ApiPropertyOptional({
description: 'Job id; exactly one of jobId or ticket is required.',
})
@IsOptional()
@IsString()
jobId?: string;
@ApiPropertyOptional({
description: 'Ticket number; exactly one of jobId or ticket is required.',
})
@IsOptional()
@IsString()
@MaxLength(64)
ticket?: string;
@ApiProperty({ type: [AppLogPointDto] })
@IsArray()
@ArrayMinSize(1)
@ArrayMaxSize(500)
@ValidateNested({ each: true })
@Type(() => AppLogPointDto)
points: AppLogPointDto[];
}
export type AppLogAckOutcome = 'ACCEPTED' | 'DUPLICATE' | 'REJECTED';
export interface AppLogAckResult {
pointId: string;
outcome: AppLogAckOutcome;
reasonCode?: string;
}
export interface AppLogAck {
schemaVersion: string;
results: AppLogAckResult[];
}

View File

@@ -0,0 +1,205 @@
import { GpsFixType, LocateMode, UtilityType } from '@prisma/client';
import { ApiPropertyOptional, ApiProperty } from '@nestjs/swagger';
import { Type } from 'class-transformer';
import {
ArrayMaxSize,
ArrayMinSize,
IsArray,
IsDateString,
IsEnum,
IsIn,
IsInt,
IsNotEmpty,
IsNumber,
IsOptional,
IsString,
Max,
MaxLength,
Min,
ValidateNested,
} from 'class-validator';
// Field names here are terse (lat/lng/alt/hAcc/...) to keep MQTT payloads
// small; this same shape is reused as the sim tool's REST request body.
export class MqttPointDto {
@ApiProperty({ minimum: -90, maximum: 90 })
@IsNumber()
@Min(-90)
@Max(90)
lat: number;
@ApiProperty({ minimum: -180, maximum: 180 })
@IsNumber()
@Min(-180)
@Max(180)
lng: number;
@ApiPropertyOptional({ description: 'Altitude, meters' })
@IsOptional()
@IsNumber()
alt?: number;
@ApiPropertyOptional({ enum: GpsFixType })
@IsOptional()
@IsEnum(GpsFixType)
fix?: GpsFixType;
@ApiPropertyOptional({ description: 'Horizontal accuracy, meters', minimum: 0 })
@IsOptional()
@IsNumber()
@Min(0)
hAcc?: number;
@ApiPropertyOptional({ description: 'Meters below grade', minimum: 0 })
@IsOptional()
@IsNumber()
@Min(0)
depth?: number;
@ApiPropertyOptional({ enum: UtilityType })
@IsOptional()
@IsEnum(UtilityType)
utility?: UtilityType;
@ApiPropertyOptional({ description: 'Ordering within a locate run' })
@IsOptional()
@IsInt()
seq?: number;
// GPS quality
@ApiPropertyOptional({ description: 'Vertical accuracy, meters', minimum: 0 })
@IsOptional()
@IsNumber()
@Min(0)
vAcc?: number;
@ApiPropertyOptional({ minimum: 0 })
@IsOptional()
@IsInt()
@Min(0)
sats?: number;
@ApiPropertyOptional({ minimum: 0 })
@IsOptional()
@IsNumber()
@Min(0)
hdop?: number;
// Locator receiver telemetry
@ApiPropertyOptional({ description: 'Locate frequency, Hz', minimum: 0 })
@IsOptional()
@IsInt()
@Min(0)
freqHz?: number;
@ApiPropertyOptional({ description: 'Signal current on the line, mA', minimum: 0 })
@IsOptional()
@IsNumber()
@Min(0)
currentMa?: number;
@ApiPropertyOptional({ description: 'Signal strength, dB' })
@IsOptional()
@IsNumber()
signalDb?: number;
@ApiPropertyOptional({ description: 'Receiver gain, dB' })
@IsOptional()
@IsNumber()
gainDb?: number;
@ApiPropertyOptional({ enum: LocateMode })
@IsOptional()
@IsEnum(LocateMode)
mode?: LocateMode;
@ApiPropertyOptional({ description: 'Degrees' })
@IsOptional()
@IsNumber()
phaseDeg?: number;
@ApiPropertyOptional({ description: 'Line direction, 0-360', minimum: 0, maximum: 360 })
@IsOptional()
@IsNumber()
@Min(0)
@Max(360)
compassDeg?: number;
@ApiPropertyOptional({ minimum: 0, maximum: 100 })
@IsOptional()
@IsNumber()
@Min(0)
@Max(100)
distortionPct?: number;
@ApiProperty({ format: 'date-time', description: 'Device GPS timestamp' })
@IsDateString()
ts: string;
}
export class MqttPointsMessageDto {
// Job is resolved by jobId when present, else by (device org, ticket)
@IsOptional()
@IsString()
jobId?: string;
@IsOptional()
@IsString()
@MaxLength(64)
ticket?: string;
// Locator receiver serial number; the publisher (MQTT credential) may be a
// phone/gateway relaying for one or more locators. Auto-registered on first sight.
@IsOptional()
@IsString()
@MaxLength(64)
serial?: string;
@IsArray()
@ArrayMinSize(1)
@ArrayMaxSize(500)
@ValidateNested({ each: true })
@Type(() => MqttPointDto)
points: MqttPointDto[];
}
export const MQTT_LOG_MESSAGE_TYPES = ['log', 'status'] as const;
export type MqttLogMessageType = (typeof MQTT_LOG_MESSAGE_TYPES)[number];
// Single reading for a locator that publishes directly (or is relayed) to
// devices/<serial>/log — the serial comes from the topic itself, so identity
// is job-anchored rather than publisher-credential-anchored. `type` decides
// what happens to it: "log" persists a LocatePoint; "status" is an ephemeral
// current-position update, broadcast live but never written to the DB.
export class MqttLogMessageDto extends MqttPointDto {
@ApiProperty({ enum: MQTT_LOG_MESSAGE_TYPES, description: '"log" persists a point; "status" is live-only' })
@IsIn(MQTT_LOG_MESSAGE_TYPES)
type: MqttLogMessageType;
@ApiProperty({ description: 'Job this reading belongs to; also supplies the organization' })
@IsString()
@IsNotEmpty()
jobId: string;
}
export class MqttJobMessageDto {
@IsString()
@IsNotEmpty()
@MaxLength(64)
ticket: string;
@IsOptional()
@IsString()
@MaxLength(200)
title?: string;
@IsOptional()
@IsString()
@MaxLength(4000)
description?: string;
@IsOptional()
@IsString()
@MaxLength(400)
address?: string;
}

View File

@@ -0,0 +1,127 @@
import { Injectable, Logger, OnModuleInit } from '@nestjs/common';
import { PrismaService } from '../prisma/prisma.service';
import { AppLogIngestService } from './app-log-ingest.service';
import { JobsIngestService } from './jobs-ingest.service';
import { LogIngestService } from './log-ingest.service';
import { MqttClientService } from './mqtt-client.service';
import { PointsIngestService } from './points-ingest.service';
@Injectable()
export class IngestRouterService implements OnModuleInit {
private readonly logger = new Logger(IngestRouterService.name);
constructor(
private readonly prisma: PrismaService,
private readonly mqttClient: MqttClientService,
private readonly pointsIngest: PointsIngestService,
private readonly jobsIngest: JobsIngestService,
private readonly logIngest: LogIngestService,
private readonly appLogIngest: AppLogIngestService,
) {}
onModuleInit() {
this.mqttClient.onMessage((topic, payload) => {
this.route(topic, payload.toString()).catch((err) =>
this.logger.error(`Failed to process ${topic}: ${err.message}`),
);
});
}
private async route(topic: string, payload: string) {
const segments = topic.split('/');
const root = segments[0];
// App point payloads contain personal/location data. Preserve routing evidence without
// creating an unmanaged second copy of the telemetry in device_events (S2 security H3).
const auditPayload =
root === 'ul' &&
segments[2] === 'app' &&
segments.slice(4).join('/') === 'log/points'
? JSON.stringify({
redacted: true,
bytes: Buffer.byteLength(payload, 'utf8'),
})
: payload;
await this.prisma.deviceEvent.create({
data: { topic, payload: auditPayload },
});
// App / device MQTT namespace: ul/{orgId}/{clientClass}/{clientId}/{subtopic...}
// (SRS §3.4.2). Sprint 2 handles the durable app-log path; other clientClasses and
// subtopics are represented but not yet ingested here.
if (root === 'ul') {
return this.routeUl(segments, payload);
}
const [, idSegment, ...rest] = segments;
const subtopic = rest.join('/');
if (root !== 'devices' || !idSegment || subtopic === 'jobs/ack') {
return; // not device traffic, or our own ack echoed back
}
// devices/<serial>/log identifies the locator by serial in the topic
// itself (job in the payload supplies org) — no publisher device lookup.
if (subtopic === 'log') {
await this.logIngest.handle(idSegment, payload);
return;
}
const device = await this.prisma.device.findUnique({
where: { mqttUsername: idSegment },
});
if (!device) {
this.logger.warn(
`Message from unregistered device username "${idSegment}" (raw-logged only)`,
);
return;
}
if (!device.isActive) {
this.logger.warn(
`Message from deactivated device "${idSegment}" ignored`,
);
return;
}
this.prisma.device
.update({ where: { id: device.id }, data: { lastSeenAt: new Date() } })
.catch(() => undefined);
switch (subtopic) {
case 'points':
await this.pointsIngest.handle(device, payload);
break;
case 'jobs':
await this.jobsIngest.handle(device, payload);
break;
case 'status':
break; // lastSeenAt already stamped above
default:
this.logger.debug(`Unhandled subtopic "${subtopic}" from ${idSegment}`);
}
}
// ul/{orgId}/{clientClass}/{clientId}/{subtopic...} — SRS §3.4.2 topic scheme.
// The namespace itself carries tenant (orgId) + publisher (clientId) identity; the
// broker ACL confines a publisher to its own namespace, and AppLogIngest re-checks
// tenant scope on the payload's job as a backstop. Only the durable app-log path is
// ingested this sprint.
private async routeUl(segments: string[], payload: string) {
const [, orgId, clientClass, clientId, ...rest] = segments;
const subtopic = rest.join('/');
if (!orgId || !clientClass || !clientId) {
this.logger.warn(`Malformed ul topic "${segments.join('/')}" — ignored`);
return;
}
// Our own cloud→publisher acks are echoed back to us on the same subscription; skip.
if (subtopic === 'ack') {
return;
}
if (clientClass === 'app' && subtopic === 'log/points') {
await this.appLogIngest.handle(orgId, clientId, payload);
return;
}
this.logger.debug(
`Unhandled ul path "${clientClass}/${subtopic}" (org ${orgId}, client ${clientId})`,
);
}
}

View File

@@ -0,0 +1,24 @@
import { Module } from '@nestjs/common';
import { RealtimeModule } from '../realtime/realtime.module';
import { AppLogIngestService } from './app-log-ingest.service';
import { IngestRouterService } from './ingest-router.service';
import { JobsIngestService } from './jobs-ingest.service';
import { LocatorRegistryService } from './locator-registry.service';
import { LogIngestService } from './log-ingest.service';
import { MqttClientService } from './mqtt-client.service';
import { PointsIngestService } from './points-ingest.service';
@Module({
imports: [RealtimeModule],
providers: [
MqttClientService,
IngestRouterService,
PointsIngestService,
JobsIngestService,
LogIngestService,
AppLogIngestService,
LocatorRegistryService,
],
exports: [MqttClientService],
})
export class IngestModule {}

View File

@@ -0,0 +1,60 @@
import { Injectable, Logger } from '@nestjs/common';
import { Device } from '@prisma/client';
import { plainToInstance } from 'class-transformer';
import { validate } from 'class-validator';
import { PrismaService } from '../prisma/prisma.service';
import { MqttJobMessageDto } from './dto/mqtt-messages.dto';
import { MqttClientService } from './mqtt-client.service';
@Injectable()
export class JobsIngestService {
private readonly logger = new Logger(JobsIngestService.name);
constructor(
private readonly prisma: PrismaService,
private readonly mqttClient: MqttClientService,
) {}
async handle(device: Device, rawPayload: string) {
let msg: MqttJobMessageDto;
try {
msg = plainToInstance(MqttJobMessageDto, JSON.parse(rawPayload) as object);
} catch {
this.ack(device, { ticket: null, status: 'error', reason: 'invalid JSON' });
return;
}
const errors = await validate(msg, { whitelist: true });
if (errors.length > 0) {
this.ack(device, { ticket: msg.ticket ?? null, status: 'error', reason: 'validation failed' });
this.logger.warn(`Invalid job message from ${device.mqttUsername}: ${errors}`);
return;
}
const existing = await this.prisma.job.findUnique({
where: { orgId_ticketNumber: { orgId: device.orgId, ticketNumber: msg.ticket } },
});
if (existing) {
this.ack(device, { ticket: msg.ticket, jobId: existing.id, status: 'exists' });
return;
}
const job = await this.prisma.job.create({
data: {
orgId: device.orgId,
ticketNumber: msg.ticket,
title: msg.title || `Ticket ${msg.ticket}`,
description: msg.description,
address: msg.address,
source: 'DEVICE',
createdByDeviceId: device.id,
},
});
this.ack(device, { ticket: msg.ticket, jobId: job.id, status: 'created' });
this.logger.log(`Device ${device.mqttUsername} created job ${job.ticketNumber}`);
}
private ack(device: Device, payload: object) {
this.mqttClient.publish(`devices/${device.mqttUsername}/jobs/ack`, payload);
}
}

View File

@@ -0,0 +1,90 @@
import { Injectable, Logger } from '@nestjs/common';
import { Device, Prisma } from '@prisma/client';
import { DevicePositionSnapshot } from '../devices/device-position';
import { PrismaService } from '../prisma/prisma.service';
import { RealtimeService } from '../realtime/realtime.service';
@Injectable()
export class LocatorRegistryService {
private readonly logger = new Logger(LocatorRegistryService.name);
constructor(
private readonly prisma: PrismaService,
private readonly realtime: RealtimeService,
) {}
// Serial numbers are globally unique, so a bare serial identifies a locator
// regardless of which org's data pipeline saw it first. Unknown serials are
// auto-registered under the given org so field data is never dropped.
async resolve(orgId: string, serial: string): Promise<Device> {
const existing = await this.prisma.device.findUnique({ where: { serialNumber: serial } });
if (existing) {
this.prisma.device
.update({ where: { id: existing.id }, data: { lastSeenAt: new Date() } })
.catch(() => undefined);
return existing;
}
this.logger.log(`Auto-registering locator serial ${serial} under org ${orgId}`);
try {
return await this.prisma.device.create({
data: { orgId, name: `Locator ${serial}`, serialNumber: serial, lastSeenAt: new Date() },
});
} catch {
// lost a concurrent-registration race; the row exists now
const raced = await this.prisma.device.findUnique({ where: { serialNumber: serial } });
if (!raced) {
throw new Error(`Failed to resolve or create locator ${serial}`);
}
return raced;
}
}
// Records the freshest known position for a device (whether from a live
// "status" ping or a persisted "log" point) and broadcasts it to anyone
// watching the org's devices page, so it updates without a manual refresh.
async recordPosition(orgId: string, deviceId: string, snapshot: DevicePositionSnapshot) {
const lastSeenAt = new Date();
await this.prisma.device
.update({
where: { id: deviceId },
data: {
lastPosition: snapshot as unknown as Prisma.InputJsonValue,
lastPositionAt: new Date(snapshot.recordedAt),
lastSeenAt,
},
})
.catch(() => undefined);
this.realtime.publish(`org:${orgId}:devices`, {
type: 'device',
orgId,
deviceId,
lastSeenAt: lastSeenAt.toISOString(),
position: {
id: `live-${deviceId}`,
lat: snapshot.lat,
lng: snapshot.lng,
altitude: snapshot.altitude,
utilityType: snapshot.utilityType,
fixType: snapshot.fixType,
sequence: null,
recordedAt: snapshot.recordedAt,
hAccuracy: snapshot.hAccuracy,
vAccuracy: snapshot.vAccuracy,
satellites: snapshot.satellites,
hdop: snapshot.hdop,
depth: snapshot.depth,
frequencyHz: snapshot.frequencyHz,
currentMa: snapshot.currentMa,
signalDb: snapshot.signalDb,
gainDb: snapshot.gainDb,
locateMode: snapshot.locateMode,
phaseDeg: snapshot.phaseDeg,
compassDeg: snapshot.compassDeg,
distortionPct: snapshot.distortionPct,
},
job: { id: snapshot.jobId, ticketNumber: snapshot.jobTicketNumber, title: snapshot.jobTitle },
});
}
}

View File

@@ -0,0 +1,137 @@
import { Injectable, Logger } from '@nestjs/common';
import { plainToInstance } from 'class-transformer';
import { validate } from 'class-validator';
import { DevicePositionSnapshot } from '../devices/device-position';
import { toPointDto } from '../points/points.service';
import { PrismaService } from '../prisma/prisma.service';
import { RealtimeService } from '../realtime/realtime.service';
import { MqttLogMessageDto } from './dto/mqtt-messages.dto';
import { LocatorRegistryService } from './locator-registry.service';
@Injectable()
export class LogIngestService {
private readonly logger = new Logger(LogIngestService.name);
constructor(
private readonly prisma: PrismaService,
private readonly realtime: RealtimeService,
private readonly locatorRegistry: LocatorRegistryService,
) {}
// devices/<serial>/log carries the locator's identity in the topic itself
// (no publisher device needs to be pre-registered); the job named in the
// payload supplies the org, since the topic alone doesn't identify one.
// `type` decides what happens to the reading: "log" persists a LocatePoint;
// "status" is an ephemeral current-position update, broadcast live but
// never written to locate_points.
async handle(serial: string, rawPayload: string) {
const msg = plainToInstance(MqttLogMessageDto, JSON.parse(rawPayload) as object);
const errors = await validate(msg, { whitelist: true });
if (errors.length > 0) {
this.logger.warn(`Invalid log message from serial ${serial}: ${errors}`);
return;
}
const job = await this.prisma.job.findUnique({ where: { id: msg.jobId } });
if (!job) {
this.logger.warn(`Unknown jobId ${msg.jobId} in log message from serial ${serial}`);
return;
}
const locator = await this.locatorRegistry.resolve(job.orgId, serial);
if (!locator.isActive) {
this.logger.warn(`Message from disabled device "${serial}" ignored`);
return;
}
const snapshot: DevicePositionSnapshot = {
lat: msg.lat,
lng: msg.lng,
altitude: msg.alt ?? null,
fixType: msg.fix ?? 'NONE',
utilityType: msg.utility ?? 'UNKNOWN',
hAccuracy: msg.hAcc ?? null,
vAccuracy: msg.vAcc ?? null,
satellites: msg.sats ?? null,
hdop: msg.hdop ?? null,
depth: msg.depth ?? null,
frequencyHz: msg.freqHz ?? null,
currentMa: msg.currentMa ?? null,
signalDb: msg.signalDb ?? null,
gainDb: msg.gainDb ?? null,
locateMode: msg.mode ?? null,
phaseDeg: msg.phaseDeg ?? null,
compassDeg: msg.compassDeg ?? null,
distortionPct: msg.distortionPct ?? null,
recordedAt: msg.ts,
jobId: job.id,
jobTicketNumber: job.ticketNumber,
jobTitle: job.title,
};
await this.locatorRegistry.recordPosition(job.orgId, locator.id, snapshot);
if (msg.type === 'status') {
this.realtime.publish(`job:${job.id}`, {
type: 'status',
jobId: job.id,
deviceId: locator.id,
serial,
lat: msg.lat,
lng: msg.lng,
altitude: msg.alt ?? null,
fixType: msg.fix ?? 'NONE',
utilityType: msg.utility ?? 'UNKNOWN',
hAccuracy: msg.hAcc ?? null,
vAccuracy: msg.vAcc ?? null,
satellites: msg.sats ?? null,
hdop: msg.hdop ?? null,
depth: msg.depth ?? null,
frequencyHz: msg.freqHz ?? null,
currentMa: msg.currentMa ?? null,
signalDb: msg.signalDb ?? null,
gainDb: msg.gainDb ?? null,
locateMode: msg.mode ?? null,
phaseDeg: msg.phaseDeg ?? null,
compassDeg: msg.compassDeg ?? null,
distortionPct: msg.distortionPct ?? null,
recordedAt: msg.ts,
});
return;
}
const point = await this.prisma.locatePoint.create({
data: {
jobId: job.id,
deviceId: locator.id,
lat: msg.lat,
lng: msg.lng,
altitude: msg.alt,
utilityType: msg.utility,
sequence: msg.seq,
fixType: msg.fix,
hAccuracy: msg.hAcc,
vAccuracy: msg.vAcc,
satellites: msg.sats,
hdop: msg.hdop,
depth: msg.depth,
frequencyHz: msg.freqHz,
currentMa: msg.currentMa,
signalDb: msg.signalDb,
gainDb: msg.gainDb,
locateMode: msg.mode,
phaseDeg: msg.phaseDeg,
compassDeg: msg.compassDeg,
distortionPct: msg.distortionPct,
recordedAt: new Date(msg.ts),
raw: msg as object,
},
});
this.realtime.publish(`job:${job.id}`, {
type: 'points',
jobId: job.id,
points: [toPointDto(point)],
});
this.logger.debug(`Stored 1 point from serial ${serial} for job ${job.ticketNumber}`);
}
}

View File

@@ -0,0 +1,74 @@
import {
Injectable,
Logger,
OnModuleDestroy,
OnModuleInit,
} from '@nestjs/common';
import { connect, MqttClient } from 'mqtt';
export type MqttMessageListener = (topic: string, payload: Buffer) => void;
@Injectable()
export class MqttClientService implements OnModuleInit, OnModuleDestroy {
private readonly logger = new Logger(MqttClientService.name);
private readonly listeners = new Set<MqttMessageListener>();
private client: MqttClient | null = null;
onModuleInit() {
const host = process.env.MQTT_HOST || 'mosquitto';
const port = Number(process.env.MQTT_PORT || 1883);
const username = process.env.MQTT_USERNAME;
const password = process.env.MQTT_PASSWORD;
if (!username || !password) {
throw new Error(
'MQTT_USERNAME and MQTT_PASSWORD are required; refusing to use a default broker credential.',
);
}
this.client = connect(`mqtt://${host}:${port}`, {
username,
password,
clientId: `ulhub-backend-${Math.random().toString(16).slice(2)}`,
});
this.client.on('connect', () => {
this.logger.log(`Connected to MQTT broker at ${host}:${port}`);
// devices/# = legacy device-direct namespace; ul/# = SRS §3.4.2 topic scheme
// (app + device-direct). QoS 1 so the broker redelivers durable log records the
// backend missed while disconnected (broker loss ≠ capture loss, SRS-SYN-1/7).
this.client!.subscribe(['devices/#', 'ul/#'], { qos: 1 }, (err) => {
if (err) {
this.logger.error('Failed to subscribe to devices/# + ul/#', err);
} else {
this.logger.log('Subscribed to devices/# and ul/#');
}
});
});
this.client.on('message', (topic, payload) => {
for (const listener of this.listeners) {
listener(topic, payload);
}
});
this.client.on('error', (error) => {
this.logger.error(`MQTT client error: ${error.message}`);
});
}
async onModuleDestroy() {
await this.client?.endAsync();
}
onMessage(listener: MqttMessageListener): () => void {
this.listeners.add(listener);
return () => this.listeners.delete(listener);
}
publish(topic: string, payload: object) {
this.client?.publish(topic, JSON.stringify(payload), { qos: 1 }, (err) => {
if (err) {
this.logger.error(`Failed to publish to ${topic}: ${err.message}`);
}
});
}
}

View File

@@ -0,0 +1,141 @@
import { Injectable, Logger } from '@nestjs/common';
import { Device, Job } from '@prisma/client';
import { plainToInstance } from 'class-transformer';
import { validate } from 'class-validator';
import { DevicePositionSnapshot } from '../devices/device-position';
import { toPointDto } from '../points/points.service';
import { PrismaService } from '../prisma/prisma.service';
import { RealtimeService } from '../realtime/realtime.service';
import { MqttPointsMessageDto } from './dto/mqtt-messages.dto';
import { LocatorRegistryService } from './locator-registry.service';
@Injectable()
export class PointsIngestService {
private readonly logger = new Logger(PointsIngestService.name);
constructor(
private readonly prisma: PrismaService,
private readonly realtime: RealtimeService,
private readonly locatorRegistry: LocatorRegistryService,
) {}
async handle(device: Device, rawPayload: string) {
const msg = plainToInstance(MqttPointsMessageDto, JSON.parse(rawPayload) as object);
const errors = await validate(msg, { whitelist: true });
if (errors.length > 0) {
this.logger.warn(`Invalid points message from ${device.mqttUsername}: ${errors}`);
return;
}
if (!msg.jobId && !msg.ticket) {
this.logger.warn(`Points message from ${device.mqttUsername} has neither jobId nor ticket`);
return;
}
const job = await this.resolveJob(device, msg);
if (!job) {
return;
}
const locator = !msg.serial || msg.serial === device.serialNumber
? device
: await this.locatorRegistry.resolve(device.orgId, msg.serial);
const points = await this.prisma.locatePoint.createManyAndReturn({
data: msg.points.map((p) => ({
jobId: job.id,
deviceId: locator.id,
lat: p.lat,
lng: p.lng,
altitude: p.alt,
utilityType: p.utility,
sequence: p.seq,
fixType: p.fix,
hAccuracy: p.hAcc,
vAccuracy: p.vAcc,
satellites: p.sats,
hdop: p.hdop,
depth: p.depth,
frequencyHz: p.freqHz,
currentMa: p.currentMa,
signalDb: p.signalDb,
gainDb: p.gainDb,
locateMode: p.mode,
phaseDeg: p.phaseDeg,
compassDeg: p.compassDeg,
distortionPct: p.distortionPct,
recordedAt: new Date(p.ts),
raw: p as object,
})),
});
this.realtime.publish(`job:${job.id}`, {
type: 'points',
jobId: job.id,
points: points.map(toPointDto),
});
const latest = msg.points.reduce((a, b) => (new Date(b.ts) > new Date(a.ts) ? b : a));
const snapshot: DevicePositionSnapshot = {
lat: latest.lat,
lng: latest.lng,
altitude: latest.alt ?? null,
fixType: latest.fix ?? 'NONE',
utilityType: latest.utility ?? 'UNKNOWN',
hAccuracy: latest.hAcc ?? null,
vAccuracy: latest.vAcc ?? null,
satellites: latest.sats ?? null,
hdop: latest.hdop ?? null,
depth: latest.depth ?? null,
frequencyHz: latest.freqHz ?? null,
currentMa: latest.currentMa ?? null,
signalDb: latest.signalDb ?? null,
gainDb: latest.gainDb ?? null,
locateMode: latest.mode ?? null,
phaseDeg: latest.phaseDeg ?? null,
compassDeg: latest.compassDeg ?? null,
distortionPct: latest.distortionPct ?? null,
recordedAt: latest.ts,
jobId: job.id,
jobTicketNumber: job.ticketNumber,
jobTitle: job.title,
};
await this.locatorRegistry.recordPosition(device.orgId, locator.id, snapshot);
this.logger.debug(`Stored ${points.length} points for job ${job.ticketNumber}`);
}
private async resolveJob(device: Device, msg: MqttPointsMessageDto): Promise<Job | null> {
if (msg.jobId) {
const job = await this.prisma.job.findFirst({
where: { id: msg.jobId, orgId: device.orgId },
});
if (!job) {
this.logger.warn(`Unknown jobId ${msg.jobId} from device ${device.mqttUsername}`);
}
return job;
}
const ticket = msg.ticket!;
const existing = await this.prisma.job.findUnique({
where: { orgId_ticketNumber: { orgId: device.orgId, ticketNumber: ticket } },
});
if (existing) {
return existing;
}
// Field-first workflow: points arrive before anyone opened the ticket in the web
// app. Create a stub so the data is never lost; metadata gets filled in later.
this.logger.log(`Auto-creating stub job for unknown ticket ${ticket} (device ${device.mqttUsername})`);
return this.prisma.job.create({
data: {
orgId: device.orgId,
ticketNumber: ticket,
title: `Ticket ${ticket} (device-created)`,
status: 'IN_PROGRESS',
source: 'DEVICE',
createdByDeviceId: device.id,
startedAt: new Date(),
},
});
}
}

View File

@@ -0,0 +1,123 @@
import { JobStatus } from '@prisma/client';
import { ApiProperty, ApiPropertyOptional } from '@nestjs/swagger';
import { Type } from 'class-transformer';
import {
IsDateString,
IsEnum,
IsInt,
IsNotEmpty,
IsOptional,
IsString,
Max,
MaxLength,
Min,
} from 'class-validator';
export class CreateJobDto {
@ApiProperty({ example: 'TKT-2026-0001', description: 'Unique within the org' })
@IsString()
@IsNotEmpty()
@MaxLength(64)
ticketNumber: string;
@ApiProperty({ example: 'Gas line locate - Main St' })
@IsString()
@IsNotEmpty()
@MaxLength(200)
title: string;
@ApiPropertyOptional()
@IsOptional()
@IsString()
@MaxLength(4000)
description?: string;
@ApiPropertyOptional({ example: '100 Main St' })
@IsOptional()
@IsString()
@MaxLength(400)
address?: string;
@ApiPropertyOptional({ enum: JobStatus })
@IsOptional()
@IsEnum(JobStatus)
status?: JobStatus;
@ApiPropertyOptional({ description: 'User id to assign the job to' })
@IsOptional()
@IsString()
assignedToId?: string;
@ApiPropertyOptional({ format: 'date-time' })
@IsOptional()
@IsDateString()
dueAt?: string;
}
export class UpdateJobDto {
@ApiPropertyOptional()
@IsOptional()
@IsString()
@IsNotEmpty()
@MaxLength(200)
title?: string;
@ApiPropertyOptional()
@IsOptional()
@IsString()
@MaxLength(4000)
description?: string;
@ApiPropertyOptional()
@IsOptional()
@IsString()
@MaxLength(400)
address?: string;
@ApiPropertyOptional({ enum: JobStatus })
@IsOptional()
@IsEnum(JobStatus)
status?: JobStatus;
@ApiPropertyOptional({ nullable: true })
@IsOptional()
@IsString()
assignedToId?: string | null;
@ApiPropertyOptional({ format: 'date-time', nullable: true })
@IsOptional()
@IsDateString()
dueAt?: string | null;
}
export class QueryJobsDto {
@ApiPropertyOptional({ enum: JobStatus })
@IsOptional()
@IsEnum(JobStatus)
status?: JobStatus;
@ApiPropertyOptional()
@IsOptional()
@IsString()
assignedToId?: string;
@ApiPropertyOptional({ description: 'Search ticket number, title, and address' })
@IsOptional()
@IsString()
q?: string;
@ApiPropertyOptional({ minimum: 1, maximum: 200, default: 50 })
@IsOptional()
@Type(() => Number)
@IsInt()
@Min(1)
@Max(200)
limit?: number;
@ApiPropertyOptional({ minimum: 0, default: 0 })
@IsOptional()
@Type(() => Number)
@IsInt()
@Min(0)
offset?: number;
}

View File

@@ -0,0 +1,57 @@
import { Body, Controller, Delete, Get, Param, Patch, Post, Query, UseGuards } from '@nestjs/common';
import { ApiBearerAuth, ApiSecurity, ApiTags } from '@nestjs/swagger';
import { CurrentPrincipal } from '../auth/decorators/current-user.decorator';
import { Roles } from '../auth/decorators/roles.decorator';
import { RequireScopes } from '../auth/decorators/scopes.decorator';
import { UserOrApiKeyGuard } from '../auth/guards/auth.guard';
import { OrgRolesGuard } from '../auth/guards/org-roles.guard';
import { ScopesGuard } from '../auth/guards/scopes.guard';
import { Principal } from '../auth/principal';
import { CreateJobDto, QueryJobsDto, UpdateJobDto } from './dto/jobs.dto';
import { JobsService } from './jobs.service';
@ApiTags('jobs')
@ApiBearerAuth('jwt')
@ApiSecurity('apiKey')
@Controller('orgs/:orgId/jobs')
@UseGuards(UserOrApiKeyGuard, OrgRolesGuard, ScopesGuard)
export class JobsController {
constructor(private readonly jobsService: JobsService) {}
@Get()
@RequireScopes('jobs:read')
list(@Param('orgId') orgId: string, @Query() query: QueryJobsDto) {
return this.jobsService.list(orgId, query);
}
@Get(':jobId')
@RequireScopes('jobs:read')
get(@Param('orgId') orgId: string, @Param('jobId') jobId: string) {
return this.jobsService.get(orgId, jobId);
}
@Post()
@Roles('MEMBER')
@RequireScopes('jobs:write')
create(
@Param('orgId') orgId: string,
@Body() dto: CreateJobDto,
@CurrentPrincipal() principal: Principal,
) {
return this.jobsService.create(orgId, dto, principal.type === 'user' ? principal.userId : null);
}
@Patch(':jobId')
@Roles('MEMBER')
@RequireScopes('jobs:write')
update(@Param('orgId') orgId: string, @Param('jobId') jobId: string, @Body() dto: UpdateJobDto) {
return this.jobsService.update(orgId, jobId, dto);
}
@Delete(':jobId')
@Roles('ORG_ADMIN')
@RequireScopes('jobs:write')
remove(@Param('orgId') orgId: string, @Param('jobId') jobId: string) {
return this.jobsService.remove(orgId, jobId);
}
}

View File

@@ -0,0 +1,10 @@
import { Module } from '@nestjs/common';
import { JobsController } from './jobs.controller';
import { JobsService } from './jobs.service';
@Module({
controllers: [JobsController],
providers: [JobsService],
exports: [JobsService],
})
export class JobsModule {}

View File

@@ -0,0 +1,117 @@
import { ConflictException, Injectable, NotFoundException } from '@nestjs/common';
import { Prisma } from '@prisma/client';
import { PrismaService } from '../prisma/prisma.service';
import { CreateJobDto, QueryJobsDto, UpdateJobDto } from './dto/jobs.dto';
const JOB_SELECT = {
id: true,
ticketNumber: true,
title: true,
description: true,
address: true,
status: true,
source: true,
dueAt: true,
startedAt: true,
completedAt: true,
createdAt: true,
updatedAt: true,
assignedTo: { select: { id: true, name: true, email: true } },
createdBy: { select: { id: true, name: true, email: true } },
createdByDeviceId: true,
_count: { select: { points: true } },
} satisfies Prisma.JobSelect;
@Injectable()
export class JobsService {
constructor(private readonly prisma: PrismaService) {}
async list(orgId: string, query: QueryJobsDto) {
const where: Prisma.JobWhereInput = {
orgId,
...(query.status && { status: query.status }),
...(query.assignedToId && { assignedToId: query.assignedToId }),
...(query.q && {
OR: [
{ ticketNumber: { contains: query.q, mode: 'insensitive' } },
{ title: { contains: query.q, mode: 'insensitive' } },
{ address: { contains: query.q, mode: 'insensitive' } },
],
}),
};
const [jobs, total] = await this.prisma.$transaction([
this.prisma.job.findMany({
where,
select: JOB_SELECT,
orderBy: { createdAt: 'desc' },
take: query.limit ?? 50,
skip: query.offset ?? 0,
}),
this.prisma.job.count({ where }),
]);
return { jobs, total };
}
async get(orgId: string, jobId: string) {
const job = await this.prisma.job.findFirst({
where: { id: jobId, orgId },
select: JOB_SELECT,
});
if (!job) {
throw new NotFoundException('Job not found');
}
return job;
}
async create(orgId: string, dto: CreateJobDto, createdById: string | null) {
const existing = await this.prisma.job.findUnique({
where: { orgId_ticketNumber: { orgId, ticketNumber: dto.ticketNumber } },
});
if (existing) {
throw new ConflictException(`Ticket ${dto.ticketNumber} already exists in this organization`);
}
return this.prisma.job.create({
data: {
orgId,
ticketNumber: dto.ticketNumber,
title: dto.title,
description: dto.description,
address: dto.address,
status: dto.status,
assignedToId: dto.assignedToId,
dueAt: dto.dueAt ? new Date(dto.dueAt) : undefined,
createdById,
},
select: JOB_SELECT,
});
}
async update(orgId: string, jobId: string, dto: UpdateJobDto) {
await this.get(orgId, jobId);
const statusTimestamps: Prisma.JobUncheckedUpdateInput =
dto.status === 'IN_PROGRESS'
? { startedAt: new Date() }
: dto.status === 'COMPLETED'
? { completedAt: new Date() }
: {};
return this.prisma.job.update({
where: { id: jobId },
data: {
...(dto.title !== undefined && { title: dto.title }),
...(dto.description !== undefined && { description: dto.description }),
...(dto.address !== undefined && { address: dto.address }),
...(dto.status !== undefined && { status: dto.status }),
...(dto.assignedToId !== undefined && { assignedToId: dto.assignedToId }),
...(dto.dueAt !== undefined && { dueAt: dto.dueAt ? new Date(dto.dueAt) : null }),
...statusTimestamps,
},
select: JOB_SELECT,
});
}
async remove(orgId: string, jobId: string) {
await this.get(orgId, jobId);
await this.prisma.job.delete({ where: { id: jobId } });
return { ok: true };
}
}

View File

@@ -1,9 +1,35 @@
import { ValidationPipe } from '@nestjs/common';
import { NestFactory } from '@nestjs/core';
import { WsAdapter } from '@nestjs/platform-ws';
import { DocumentBuilder, SwaggerModule } from '@nestjs/swagger';
import * as cookieParser from 'cookie-parser';
import { AppModule } from './app.module';
async function bootstrap() {
const app = await NestFactory.create(AppModule);
app.setGlobalPrefix('api');
app.use(cookieParser());
app.useGlobalPipes(new ValidationPipe({ whitelist: true, transform: true }));
app.useWebSocketAdapter(new WsAdapter(app));
const swaggerConfig = new DocumentBuilder()
.setTitle('UlHub API')
.setDescription(
'Utility-locating platform API: organizations, jobs, locate points, devices, and realtime data. ' +
'Click Authorize and supply a Bearer token (from POST /auth/login) or an org-scoped X-API-Key to try requests.',
)
.setVersion('1.0')
.addBearerAuth({ type: 'http', scheme: 'bearer', bearerFormat: 'JWT' }, 'jwt')
.addApiKey({ type: 'apiKey', name: 'X-API-Key', in: 'header' }, 'apiKey')
.build();
const swaggerDocument = SwaggerModule.createDocument(app, swaggerConfig);
SwaggerModule.setup('api/docs', app, swaggerDocument, {
// "Try it out" must not silently ride on the browser's ulhub_token login
// cookie — force it to only use what's explicitly entered via Authorize
// (Bearer token or X-API-Key), same as any other API client.
swaggerOptions: { withCredentials: false },
});
await app.listen(3001, '0.0.0.0');
}

View File

@@ -0,0 +1,27 @@
import { OrgRole } from '@prisma/client';
import { ApiProperty } from '@nestjs/swagger';
import { IsEmail, IsEnum, IsNotEmpty, IsString, MaxLength } from 'class-validator';
export class UpdateOrgDto {
@ApiProperty()
@IsString()
@IsNotEmpty()
@MaxLength(120)
name: string;
}
export class AddMemberDto {
@ApiProperty({ description: 'Must belong to an existing user (they must have registered already)' })
@IsEmail()
email: string;
@ApiProperty({ enum: OrgRole })
@IsEnum(OrgRole)
role: OrgRole;
}
export class UpdateMemberDto {
@ApiProperty({ enum: OrgRole })
@IsEnum(OrgRole)
role: OrgRole;
}

View File

@@ -0,0 +1,67 @@
import { Body, Controller, Delete, Get, Param, Patch, Post, UseGuards } from '@nestjs/common';
import { ApiBearerAuth, ApiSecurity, ApiTags } from '@nestjs/swagger';
import { CurrentPrincipal } from '../auth/decorators/current-user.decorator';
import { Roles } from '../auth/decorators/roles.decorator';
import { JwtAuthGuard, UserOrApiKeyGuard } from '../auth/guards/auth.guard';
import { OrgRolesGuard } from '../auth/guards/org-roles.guard';
import { ScopesGuard } from '../auth/guards/scopes.guard';
import { UserPrincipal } from '../auth/principal';
import { AddMemberDto, UpdateMemberDto, UpdateOrgDto } from './dto/orgs.dto';
import { OrgsService } from './orgs.service';
@ApiTags('orgs')
@Controller('orgs')
export class OrgsController {
constructor(private readonly orgsService: OrgsService) {}
@Get()
@UseGuards(JwtAuthGuard)
@ApiBearerAuth('jwt')
listMine(@CurrentPrincipal() principal: UserPrincipal) {
return this.orgsService.listForUser(principal.userId);
}
@Patch(':orgId')
@UseGuards(JwtAuthGuard, OrgRolesGuard)
@Roles('ORG_ADMIN')
@ApiBearerAuth('jwt')
rename(@Param('orgId') orgId: string, @Body() dto: UpdateOrgDto) {
return this.orgsService.rename(orgId, dto.name);
}
@Get(':orgId/members')
@UseGuards(UserOrApiKeyGuard, OrgRolesGuard, ScopesGuard)
@ApiBearerAuth('jwt')
@ApiSecurity('apiKey')
listMembers(@Param('orgId') orgId: string) {
return this.orgsService.listMembers(orgId);
}
@Post(':orgId/members')
@UseGuards(JwtAuthGuard, OrgRolesGuard)
@Roles('ORG_ADMIN')
@ApiBearerAuth('jwt')
addMember(@Param('orgId') orgId: string, @Body() dto: AddMemberDto) {
return this.orgsService.addMember(orgId, dto.email, dto.role);
}
@Patch(':orgId/members/:userId')
@UseGuards(JwtAuthGuard, OrgRolesGuard)
@Roles('ORG_ADMIN')
@ApiBearerAuth('jwt')
updateMember(
@Param('orgId') orgId: string,
@Param('userId') userId: string,
@Body() dto: UpdateMemberDto,
) {
return this.orgsService.updateMember(orgId, userId, dto.role);
}
@Delete(':orgId/members/:userId')
@UseGuards(JwtAuthGuard, OrgRolesGuard)
@Roles('ORG_ADMIN')
@ApiBearerAuth('jwt')
removeMember(@Param('orgId') orgId: string, @Param('userId') userId: string) {
return this.orgsService.removeMember(orgId, userId);
}
}

View File

@@ -0,0 +1,9 @@
import { Module } from '@nestjs/common';
import { OrgsController } from './orgs.controller';
import { OrgsService } from './orgs.service';
@Module({
controllers: [OrgsController],
providers: [OrgsService],
})
export class OrgsModule {}

View File

@@ -0,0 +1,92 @@
import { BadRequestException, Injectable, NotFoundException } from '@nestjs/common';
import { OrgRole } from '@prisma/client';
import { PrismaService } from '../prisma/prisma.service';
@Injectable()
export class OrgsService {
constructor(private readonly prisma: PrismaService) {}
listForUser(userId: string) {
return this.prisma.organization.findMany({
where: { memberships: { some: { userId } } },
select: {
id: true,
name: true,
slug: true,
createdAt: true,
memberships: { where: { userId }, select: { role: true } },
_count: { select: { jobs: true, devices: true } },
},
orderBy: { createdAt: 'asc' },
});
}
rename(orgId: string, name: string) {
return this.prisma.organization.update({ where: { id: orgId }, data: { name } });
}
listMembers(orgId: string) {
return this.prisma.orgMembership.findMany({
where: { orgId },
select: {
role: true,
createdAt: true,
user: { select: { id: true, email: true, name: true } },
},
orderBy: { createdAt: 'asc' },
});
}
async addMember(orgId: string, email: string, role: OrgRole) {
const user = await this.prisma.user.findUnique({ where: { email } });
if (!user) {
throw new NotFoundException('No account exists with that email (they must register first)');
}
const existing = await this.prisma.orgMembership.findUnique({
where: { orgId_userId: { orgId, userId: user.id } },
});
if (existing) {
throw new BadRequestException('Already a member of this organization');
}
return this.prisma.orgMembership.create({
data: { orgId, userId: user.id, role },
select: { role: true, user: { select: { id: true, email: true, name: true } } },
});
}
async updateMember(orgId: string, userId: string, role: OrgRole) {
await this.assertNotLastAdmin(orgId, userId);
return this.prisma.orgMembership.update({
where: { orgId_userId: { orgId, userId } },
data: { role },
select: { role: true, user: { select: { id: true, email: true, name: true } } },
});
}
async removeMember(orgId: string, userId: string) {
await this.assertNotLastAdmin(orgId, userId);
await this.prisma.orgMembership.delete({
where: { orgId_userId: { orgId, userId } },
});
return { ok: true };
}
// Refuse to demote/remove the only remaining admin so the org can't be orphaned.
private async assertNotLastAdmin(orgId: string, userId: string) {
const target = await this.prisma.orgMembership.findUnique({
where: { orgId_userId: { orgId, userId } },
});
if (!target) {
throw new NotFoundException('Membership not found');
}
if (target.role !== 'ORG_ADMIN') {
return;
}
const admins = await this.prisma.orgMembership.count({
where: { orgId, role: 'ORG_ADMIN' },
});
if (admins <= 1) {
throw new BadRequestException('Cannot demote or remove the last organization admin');
}
}
}

View File

@@ -0,0 +1,163 @@
import { GpsFixType, LocateMode, UtilityType } from '@prisma/client';
import { ApiPropertyOptional, ApiProperty } from '@nestjs/swagger';
import { Type } from 'class-transformer';
import {
IsDateString,
IsEnum,
IsInt,
IsNumber,
IsOptional,
IsString,
Matches,
Max,
Min,
} from 'class-validator';
export class CreatePointDto {
@ApiProperty({ minimum: -90, maximum: 90 })
@IsNumber()
@Min(-90)
@Max(90)
lat: number;
@ApiProperty({ minimum: -180, maximum: 180 })
@IsNumber()
@Min(-180)
@Max(180)
lng: number;
@ApiPropertyOptional({ description: 'Meters' })
@IsOptional()
@IsNumber()
altitude?: number;
@ApiPropertyOptional({ enum: GpsFixType })
@IsOptional()
@IsEnum(GpsFixType)
fixType?: GpsFixType;
@ApiPropertyOptional({ description: 'Horizontal accuracy, meters', minimum: 0 })
@IsOptional()
@IsNumber()
@Min(0)
hAccuracy?: number;
@ApiPropertyOptional({ description: 'Meters below grade', minimum: 0 })
@IsOptional()
@IsNumber()
@Min(0)
depth?: number;
@ApiPropertyOptional({ enum: UtilityType })
@IsOptional()
@IsEnum(UtilityType)
utilityType?: UtilityType;
@ApiPropertyOptional({ description: 'Ordering within a locate run' })
@IsOptional()
@IsInt()
sequence?: number;
// GPS quality
@ApiPropertyOptional({ description: 'Vertical accuracy, meters', minimum: 0 })
@IsOptional()
@IsNumber()
@Min(0)
vAccuracy?: number;
@ApiPropertyOptional({ minimum: 0 })
@IsOptional()
@IsInt()
@Min(0)
satellites?: number;
@ApiPropertyOptional({ minimum: 0 })
@IsOptional()
@IsNumber()
@Min(0)
hdop?: number;
// Locator receiver telemetry
@ApiPropertyOptional({ description: 'Locate frequency, Hz', minimum: 0 })
@IsOptional()
@IsInt()
@Min(0)
frequencyHz?: number;
@ApiPropertyOptional({ description: 'Signal current on the line, mA', minimum: 0 })
@IsOptional()
@IsNumber()
@Min(0)
currentMa?: number;
@ApiPropertyOptional({ description: 'Signal strength, dB' })
@IsOptional()
@IsNumber()
signalDb?: number;
@ApiPropertyOptional({ description: 'Receiver gain, dB' })
@IsOptional()
@IsNumber()
gainDb?: number;
@ApiPropertyOptional({ enum: LocateMode })
@IsOptional()
@IsEnum(LocateMode)
locateMode?: LocateMode;
@ApiPropertyOptional({ description: 'Degrees' })
@IsOptional()
@IsNumber()
phaseDeg?: number;
@ApiPropertyOptional({ description: 'Line direction, 0-360', minimum: 0, maximum: 360 })
@IsOptional()
@IsNumber()
@Min(0)
@Max(360)
compassDeg?: number;
@ApiPropertyOptional({ minimum: 0, maximum: 100 })
@IsOptional()
@IsNumber()
@Min(0)
@Max(100)
distortionPct?: number;
@ApiProperty({ format: 'date-time', description: 'Device GPS timestamp' })
@IsDateString()
recordedAt: string;
}
export class QueryPointsDto {
@ApiPropertyOptional({ format: 'date-time', description: 'Return points recorded strictly after this instant' })
@IsOptional()
@IsDateString()
after?: string;
@ApiPropertyOptional({ format: 'date-time' })
@IsOptional()
@IsDateString()
from?: string;
@ApiPropertyOptional({ format: 'date-time' })
@IsOptional()
@IsDateString()
to?: string;
@ApiPropertyOptional({ description: 'minLng,minLat,maxLng,maxLat', example: '-96.85,33.14,-96.82,33.16' })
@IsOptional()
@IsString()
@Matches(/^-?\d+(\.\d+)?,-?\d+(\.\d+)?,-?\d+(\.\d+)?,-?\d+(\.\d+)?$/, {
message: 'bbox must be "minLng,minLat,maxLng,maxLat"',
})
bbox?: string;
@ApiPropertyOptional({ minimum: 1, maximum: 10000, default: 5000 })
@IsOptional()
@Type(() => Number)
@IsInt()
@Min(1)
@Max(10000)
limit?: number;
}

View File

@@ -0,0 +1,45 @@
import { Body, Controller, Get, Param, Post, Query, UseGuards } from '@nestjs/common';
import { ApiBearerAuth, ApiSecurity, ApiTags } from '@nestjs/swagger';
import { Roles } from '../auth/decorators/roles.decorator';
import { RequireScopes } from '../auth/decorators/scopes.decorator';
import { UserOrApiKeyGuard } from '../auth/guards/auth.guard';
import { OrgRolesGuard } from '../auth/guards/org-roles.guard';
import { ScopesGuard } from '../auth/guards/scopes.guard';
import { CreatePointDto, QueryPointsDto } from './dto/points.dto';
import { PointsService } from './points.service';
@ApiTags('points')
@ApiBearerAuth('jwt')
@ApiSecurity('apiKey')
@Controller('orgs/:orgId')
@UseGuards(UserOrApiKeyGuard, OrgRolesGuard, ScopesGuard)
export class PointsController {
constructor(private readonly pointsService: PointsService) {}
@Get('jobs/:jobId/points')
@RequireScopes('points:read')
listForJob(
@Param('orgId') orgId: string,
@Param('jobId') jobId: string,
@Query() query: QueryPointsDto,
) {
return this.pointsService.listForJob(orgId, jobId, query);
}
@Post('jobs/:jobId/points')
@Roles('MEMBER')
@RequireScopes('points:write')
create(
@Param('orgId') orgId: string,
@Param('jobId') jobId: string,
@Body() dto: CreatePointDto,
) {
return this.pointsService.createForJob(orgId, jobId, dto);
}
@Get('points')
@RequireScopes('points:read')
listForOrg(@Param('orgId') orgId: string, @Query() query: QueryPointsDto) {
return this.pointsService.listForOrg(orgId, query);
}
}

View File

@@ -0,0 +1,10 @@
import { Module } from '@nestjs/common';
import { PointsController } from './points.controller';
import { PointsService } from './points.service';
@Module({
controllers: [PointsController],
providers: [PointsService],
exports: [PointsService],
})
export class PointsModule {}

View File

@@ -0,0 +1,146 @@
import { Injectable, NotFoundException } from '@nestjs/common';
import { LocatePoint, Prisma } from '@prisma/client';
import { PrismaService } from '../prisma/prisma.service';
import { CreatePointDto, QueryPointsDto } from './dto/points.dto';
// JSON-safe shape: BigInt id -> string, Decimal -> number
export interface PointDto {
id: string;
pointId: string | null;
jobId: string;
deviceId: string | null;
origin: string;
uploadPath: string | null;
originClientId: string | null;
createdAt: Date | null;
lat: number;
lng: number;
altitude: number | null;
utilityType: string;
sequence: number | null;
fixType: string;
hAccuracy: number | null;
vAccuracy: number | null;
satellites: number | null;
hdop: number | null;
depth: number | null;
frequencyHz: number | null;
currentMa: number | null;
signalDb: number | null;
gainDb: number | null;
locateMode: string | null;
phaseDeg: number | null;
compassDeg: number | null;
distortionPct: number | null;
recordedAt: Date;
receivedAt: Date;
}
function num(value: unknown): number | null {
return value === null || value === undefined ? null : Number(value);
}
export function toPointDto(p: LocatePoint): PointDto {
return {
id: p.id.toString(),
pointId: p.pointId,
jobId: p.jobId,
deviceId: p.deviceId,
origin: p.origin,
uploadPath: p.uploadPath,
originClientId: p.originClientId,
createdAt: p.createdAt,
lat: Number(p.lat),
lng: Number(p.lng),
altitude: num(p.altitude),
utilityType: p.utilityType,
sequence: p.sequence,
fixType: p.fixType,
hAccuracy: num(p.hAccuracy),
vAccuracy: num(p.vAccuracy),
satellites: p.satellites,
hdop: num(p.hdop),
depth: num(p.depth),
frequencyHz: p.frequencyHz,
currentMa: num(p.currentMa),
signalDb: num(p.signalDb),
gainDb: num(p.gainDb),
locateMode: p.locateMode,
phaseDeg: num(p.phaseDeg),
compassDeg: num(p.compassDeg),
distortionPct: num(p.distortionPct),
recordedAt: p.recordedAt,
receivedAt: p.receivedAt,
};
}
@Injectable()
export class PointsService {
constructor(private readonly prisma: PrismaService) {}
async listForJob(orgId: string, jobId: string, query: QueryPointsDto): Promise<{ points: PointDto[] }> {
const job = await this.prisma.job.findFirst({ where: { id: jobId, orgId }, select: { id: true } });
if (!job) {
throw new NotFoundException('Job not found');
}
return { points: await this.query({ jobId }, query) };
}
async listForOrg(orgId: string, query: QueryPointsDto): Promise<{ points: PointDto[] }> {
return { points: await this.query({ job: { orgId } }, query) };
}
async createForJob(
orgId: string,
jobId: string,
dto: CreatePointDto,
deviceId: string | null = null,
): Promise<PointDto> {
const job = await this.prisma.job.findFirst({ where: { id: jobId, orgId }, select: { id: true } });
if (!job) {
throw new NotFoundException('Job not found');
}
const { recordedAt, ...fields } = dto;
const point = await this.prisma.locatePoint.create({
data: {
jobId,
deviceId,
...fields,
recordedAt: new Date(recordedAt),
},
});
return toPointDto(point);
}
private async query(scope: Prisma.LocatePointWhereInput, query: QueryPointsDto): Promise<PointDto[]> {
const where: Prisma.LocatePointWhereInput = {
...scope,
...(query.after && { recordedAt: { gt: new Date(query.after) } }),
...((query.from || query.to) && {
recordedAt: {
...(query.after && { gt: new Date(query.after) }),
...(query.from && { gte: new Date(query.from) }),
...(query.to && { lte: new Date(query.to) }),
},
}),
};
if (query.bbox) {
const [minLng, minLat, maxLng, maxLat] = query.bbox.split(',').map(Number);
// Spatial filter runs on the generated geom column (GIST-indexed); the id list
// is then fed back through Prisma so scope/time filters and typing stay uniform.
const rows = await this.prisma.$queryRaw<{ id: bigint }[]>`
SELECT id FROM locate_points
WHERE geom && ST_MakeEnvelope(${minLng}, ${minLat}, ${maxLng}, ${maxLat}, 4326)
`;
where.id = { in: rows.map((r) => r.id) };
}
const points = await this.prisma.locatePoint.findMany({
where,
orderBy: [{ recordedAt: 'asc' }, { id: 'asc' }],
take: query.limit ?? 5000,
});
return points.map(toPointDto);
}
}

View File

@@ -0,0 +1,9 @@
import { Global, Module } from '@nestjs/common';
import { PrismaService } from './prisma.service';
@Global()
@Module({
providers: [PrismaService],
exports: [PrismaService],
})
export class PrismaModule {}

View File

@@ -0,0 +1,13 @@
import { Injectable, OnModuleDestroy, OnModuleInit } from '@nestjs/common';
import { PrismaClient } from '@prisma/client';
@Injectable()
export class PrismaService extends PrismaClient implements OnModuleInit, OnModuleDestroy {
async onModuleInit() {
await this.$connect();
}
async onModuleDestroy() {
await this.$disconnect();
}
}

View File

@@ -0,0 +1,117 @@
import { Logger } from '@nestjs/common';
import { JwtService } from '@nestjs/jwt';
import { OnGatewayConnection, OnGatewayDisconnect, WebSocketGateway } from '@nestjs/websockets';
import type { IncomingMessage } from 'http';
import type { WebSocket } from 'ws';
import { JwtPayload, AUTH_COOKIE } from '../auth/strategies/jwt.strategy';
import { PrismaService } from '../prisma/prisma.service';
import { RealtimeService } from './realtime.service';
interface ClientMessage {
type: 'subscribe' | 'unsubscribe';
channel: string;
}
@WebSocketGateway({ path: '/api/ws' })
export class RealtimeGateway implements OnGatewayConnection, OnGatewayDisconnect {
private readonly logger = new Logger(RealtimeGateway.name);
private readonly users = new WeakMap<WebSocket, string>();
constructor(
private readonly jwtService: JwtService,
private readonly prisma: PrismaService,
private readonly realtime: RealtimeService,
) {}
handleConnection(client: WebSocket, req: IncomingMessage) {
const userId = this.authenticate(req);
if (!userId) {
client.close(4401, 'unauthorized');
return;
}
this.users.set(client, userId);
client.send(JSON.stringify({ type: 'connected' }));
client.on('message', (data) => {
this.onClientMessage(client, data.toString()).catch((err) =>
this.logger.error(`WS message error: ${err.message}`),
);
});
}
handleDisconnect(client: WebSocket) {
this.realtime.removeSocket(client);
}
private authenticate(req: IncomingMessage): string | null {
// Same-origin httpOnly cookie rides along on the WS upgrade request
const cookies = req.headers.cookie ?? '';
const token = cookies
.split(';')
.map((c) => c.trim())
.find((c) => c.startsWith(`${AUTH_COOKIE}=`))
?.slice(AUTH_COOKIE.length + 1);
if (!token) {
return null;
}
try {
const payload = this.jwtService.verify<JwtPayload>(token);
return payload.sub;
} catch {
return null;
}
}
private async onClientMessage(client: WebSocket, raw: string) {
let msg: ClientMessage;
try {
msg = JSON.parse(raw);
} catch {
client.send(JSON.stringify({ type: 'error', reason: 'invalid JSON' }));
return;
}
if (msg.type === 'unsubscribe' && typeof msg.channel === 'string') {
this.realtime.unsubscribe(msg.channel, client);
client.send(JSON.stringify({ type: 'unsubscribed', channel: msg.channel }));
return;
}
if (msg.type !== 'subscribe' || typeof msg.channel !== 'string') {
client.send(JSON.stringify({ type: 'error', reason: 'expected {type: subscribe|unsubscribe, channel}' }));
return;
}
const allowed = await this.canAccessChannel(this.users.get(client), msg.channel);
if (!allowed) {
client.send(JSON.stringify({ type: 'error', reason: `cannot subscribe to ${msg.channel}` }));
return;
}
this.realtime.subscribe(msg.channel, client);
client.send(JSON.stringify({ type: 'subscribed', channel: msg.channel }));
}
private async canAccessChannel(userId: string | undefined, channel: string): Promise<boolean> {
if (!userId) {
return false;
}
if (channel.startsWith('job:')) {
const jobId = channel.slice(4);
const job = await this.prisma.job.findUnique({ where: { id: jobId }, select: { orgId: true } });
return job ? this.isMember(job.orgId, userId) : false;
}
const devicesMatch = /^org:(.+):devices$/.exec(channel);
if (devicesMatch) {
return this.isMember(devicesMatch[1], userId);
}
return false;
}
private async isMember(orgId: string, userId: string): Promise<boolean> {
const membership = await this.prisma.orgMembership.findUnique({
where: { orgId_userId: { orgId, userId } },
});
return membership !== null;
}
}

View File

@@ -0,0 +1,16 @@
import { Module } from '@nestjs/common';
import { JwtModule } from '@nestjs/jwt';
import { RealtimeGateway } from './realtime.gateway';
import { RealtimeService } from './realtime.service';
@Module({
imports: [
JwtModule.register({
secret: process.env.JWT_SECRET || 'dev-only-insecure-secret',
signOptions: { expiresIn: '7d' },
}),
],
providers: [RealtimeGateway, RealtimeService],
exports: [RealtimeService],
})
export class RealtimeModule {}

View File

@@ -0,0 +1,46 @@
import { Injectable } from '@nestjs/common';
import type { WebSocket } from 'ws';
@Injectable()
export class RealtimeService {
private readonly channels = new Map<string, Set<WebSocket>>();
subscribe(channel: string, socket: WebSocket) {
let sockets = this.channels.get(channel);
if (!sockets) {
sockets = new Set();
this.channels.set(channel, sockets);
}
sockets.add(socket);
}
unsubscribe(channel: string, socket: WebSocket) {
const sockets = this.channels.get(channel);
sockets?.delete(socket);
if (sockets?.size === 0) {
this.channels.delete(channel);
}
}
removeSocket(socket: WebSocket) {
for (const [channel, sockets] of this.channels) {
sockets.delete(socket);
if (sockets.size === 0) {
this.channels.delete(channel);
}
}
}
publish(channel: string, message: object) {
const sockets = this.channels.get(channel);
if (!sockets) {
return;
}
const data = JSON.stringify(message);
for (const socket of sockets) {
if (socket.readyState === socket.OPEN) {
socket.send(data);
}
}
}
}

View File

@@ -0,0 +1,27 @@
import { ApiProperty, ApiPropertyOptional } from '@nestjs/swagger';
import { IsIn, IsNotEmpty, IsOptional, IsString, MaxLength } from 'class-validator';
import { MqttLogMessageDto } from '../../ingest/dto/mqtt-messages.dto';
export type SimTransport = 'relay' | 'mqtts';
// What the simulator UI sends the backend: a devices/<serial>/log payload
// (type: "log" persists a point, "status" is a live-only position update)
// plus "serial", which lives in the topic rather than the wire payload.
export class SimPublishPointDto extends MqttLogMessageDto {
@ApiProperty({ description: 'Locator serial number; identifies the device via devices/<serial>/log' })
@IsString()
@IsNotEmpty()
@MaxLength(64)
serial: string;
@ApiPropertyOptional({
description:
'"relay" (default) publishes via the backend\'s own privileged broker connection. ' +
'"mqtts" instead connects to port 8883 and authenticates as this serial\'s own issued ' +
'client certificate, exercising the real device-auth + ACL path.',
enum: ['relay', 'mqtts'],
})
@IsOptional()
@IsIn(['relay', 'mqtts'])
transport?: SimTransport;
}

View File

@@ -0,0 +1,97 @@
import { BadRequestException, Injectable, Logger, NotFoundException, OnModuleDestroy } from '@nestjs/common';
import { connect, MqttClient } from 'mqtt';
import { PkiService } from '../certificates/pki.service';
import { PrismaService } from '../prisma/prisma.service';
// Publishes as a real field device would: connects to the 8883 TLS listener
// and authenticates with the device's own issued client certificate (CN =
// serial number), rather than going through the backend's own privileged
// broker connection (see MqttClientService). This is what actually exercises
// the mTLS handshake + devices/%u/# ACL scoping end-to-end, so a simulated
// device is only ever as trusted as a real one.
@Injectable()
export class SimMqttsService implements OnModuleDestroy {
private readonly logger = new Logger(SimMqttsService.name);
private readonly clients = new Map<string, Promise<MqttClient>>();
constructor(
private readonly prisma: PrismaService,
private readonly pki: PkiService,
) {}
async publish(orgId: string, serial: string, topic: string, payload: object): Promise<void> {
const client = await this.clientFor(orgId, serial);
await new Promise<void>((resolve, reject) => {
client.publish(topic, JSON.stringify(payload), { qos: 1 }, (err) => (err ? reject(err) : resolve()));
});
}
private clientFor(orgId: string, serial: string): Promise<MqttClient> {
const existing = this.clients.get(serial);
if (existing) {
return existing;
}
const created = this.connectAsDevice(orgId, serial).catch((err) => {
this.clients.delete(serial);
throw err;
});
this.clients.set(serial, created);
return created;
}
private async connectAsDevice(orgId: string, serial: string): Promise<MqttClient> {
const device = await this.prisma.device.findFirst({ where: { orgId, serialNumber: serial } });
if (!device) {
throw new NotFoundException(
`No device with serial "${serial}" in this organization — create one on the Devices page first`,
);
}
const cert = await this.prisma.deviceCertificate.findUnique({ where: { deviceId: device.id } });
if (!cert) {
throw new BadRequestException(
`Device "${serial}" has no certificate issued — issue one from the Devices page, then retry`,
);
}
const ca = await this.pki.caCertPem();
const host = process.env.MQTT_HOST || 'mosquitto';
const port = Number(process.env.MQTT_TLS_PORT || 8883);
// The broker's server cert is issued for whatever hostname an admin chose
// when provisioning it (PkiService.provisionServerCert), which may not
// match this container's docker-network hostname. Overriding just the TLS
// servername lets us verify the chain + name against what it was actually
// issued for, rather than disabling verification.
const servername = process.env.MQTT_TLS_SERVERNAME || 'localhost';
return new Promise<MqttClient>((resolve, reject) => {
const client = connect({
host,
port,
protocol: 'mqtts',
servername,
ca,
cert: cert.certificatePem,
key: cert.privateKeyPem,
rejectUnauthorized: true,
connectTimeout: 8000,
clientId: `ulhub-sim-${serial}-${Math.random().toString(16).slice(2)}`,
});
const onError = (err: Error) => {
client.end(true);
reject(new BadRequestException(`MQTTS connection failed for device "${serial}": ${err.message}`));
};
client.once('error', onError);
client.once('connect', () => {
client.removeListener('error', onError);
client.on('error', (err) => this.logger.warn(`Simulated device ${serial} MQTTS error: ${err.message}`));
resolve(client);
});
});
}
async onModuleDestroy(): Promise<void> {
for (const pending of this.clients.values()) {
await pending.then((client) => client.endAsync()).catch(() => undefined);
}
}
}

View File

@@ -0,0 +1,22 @@
import { Body, Controller, Param, Post, UseGuards } from '@nestjs/common';
import { ApiBearerAuth, ApiTags } from '@nestjs/swagger';
import { Roles } from '../auth/decorators/roles.decorator';
import { JwtAuthGuard } from '../auth/guards/auth.guard';
import { OrgRolesGuard } from '../auth/guards/org-roles.guard';
import { SimPublishPointDto } from './dto/sim.dto';
import { SimService } from './sim.service';
// JWT-only: this is a UI-driven testing tool, not a public integration surface.
@ApiTags('simulator')
@ApiBearerAuth('jwt')
@Controller('orgs/:orgId/sim')
@UseGuards(JwtAuthGuard, OrgRolesGuard)
@Roles('MEMBER')
export class SimController {
constructor(private readonly simService: SimService) {}
@Post('publish')
publish(@Param('orgId') orgId: string, @Body() dto: SimPublishPointDto) {
return this.simService.publish(orgId, dto);
}
}

View File

@@ -0,0 +1,13 @@
import { Module } from '@nestjs/common';
import { CertificatesModule } from '../certificates/certificates.module';
import { IngestModule } from '../ingest/ingest.module';
import { SimMqttsService } from './sim-mqtts.service';
import { SimController } from './sim.controller';
import { SimService } from './sim.service';
@Module({
imports: [IngestModule, CertificatesModule],
controllers: [SimController],
providers: [SimService, SimMqttsService],
})
export class SimModule {}

View File

@@ -0,0 +1,36 @@
import { Injectable, NotFoundException } from '@nestjs/common';
import { MqttClientService } from '../ingest/mqtt-client.service';
import { PrismaService } from '../prisma/prisma.service';
import { SimPublishPointDto } from './dto/sim.dto';
import { SimMqttsService } from './sim-mqtts.service';
@Injectable()
export class SimService {
constructor(
private readonly prisma: PrismaService,
private readonly mqttClient: MqttClientService,
private readonly simMqtts: SimMqttsService,
) {}
// Publishes onto the real broker rather than writing to the DB directly, so
// the simulator exercises the exact same ingest path a real locator would.
// "relay" (default) reuses the backend's own privileged connection; "mqtts"
// instead opens a real TLS connection authenticated with the device's own
// client certificate, so it's subject to the same mTLS handshake and
// devices/%u/# ACL a real device would be.
async publish(orgId: string, dto: SimPublishPointDto) {
const job = await this.prisma.job.findFirst({ where: { id: dto.jobId, orgId }, select: { id: true } });
if (!job) {
throw new NotFoundException('Job not found in this organization');
}
const { serial, transport, ...payload } = dto;
const topic = `devices/${serial}/log`;
if (transport === 'mqtts') {
await this.simMqtts.publish(orgId, serial, topic, payload);
} else {
this.mqttClient.publish(topic, payload);
}
return { ok: true, topic, transport: transport ?? 'relay' };
}
}

View File

@@ -1,5 +1,7 @@
import { Controller, Get } from '@nestjs/common';
import { ApiTags } from '@nestjs/swagger';
@ApiTags('app')
@Controller('status')
export class StatusController {
@Get()

View File

@@ -1,6 +1,6 @@
services:
postgres:
image: postgres:17
image: postgis/postgis:17-3.5
environment:
POSTGRES_DB: ulhub
POSTGRES_USER: ulhub
@@ -11,11 +11,13 @@ services:
mosquitto:
image: eclipse-mosquitto:2
ports:
- "1883:1883"
- "8883:8883"
- "9001:9001"
- "8884:8884"
- "127.0.0.1:9001:9001"
volumes:
- ./mosquitto:/mosquitto
- /home/ubuntu/.config/ul-platform/mosquitto.passwd:/run/secrets/mosquitto_passwd:ro
- /home/ubuntu/.config/ul-platform/mosquitto.acl:/run/secrets/mosquitto_acl:ro
backend:
build:
@@ -25,18 +27,32 @@ services:
- "3001:3001"
depends_on:
- postgres
- mosquitto
# Development mounts: mount source for hot-reload and keep container node_modules
volumes:
- ./backend:/usr/src/app:delegated
- /usr/src/app/node_modules
- ./mosquitto/certs:/mosquitto-certs
environment:
DATABASE_HOST: postgres
DATABASE_PORT: 5432
DATABASE_USER: ulhub
DATABASE_PASSWORD: development
DATABASE_NAME: ulhub
DATABASE_URL: postgresql://ulhub:development@postgres:5432/ulhub
JWT_SECRET: ${JWT_SECRET:-dev-only-insecure-secret}
MQTT_HOST: mosquitto
MQTT_PORT: 1883
MQTT_USERNAME: ${MQTT_BACKEND_USERNAME:-backend}
MQTT_PASSWORD: ${MQTT_BACKEND_PASSWORD:?set MQTT_BACKEND_PASSWORD}
MQTT_CERTS_DIR: /mosquitto-certs
MQTT_TLS_PORT: 8883
# Must match the CN the broker's server cert was provisioned for
# (Settings → MQTT Certs), not this container's docker-network hostname —
# used by the simulator's MQTTS transport to verify the broker's identity.
MQTT_TLS_SERVERNAME: ${MQTT_TLS_SERVERNAME:-localhost}
# BLE challenge-response session certs (backend/src/device-mqtt-auth):
# how long a nonce is redeemable for, and how long the short-lived
# session cert it produces is valid before a phone must re-challenge.
MQTT_CHALLENGE_TTL_SECONDS: ${MQTT_CHALLENGE_TTL_SECONDS:-120}
MQTT_SESSION_CERT_HOURS: ${MQTT_SESSION_CERT_HOURS:-24}
NODE_ENV: development
command: npm run start:dev
command: sh -c "npx prisma migrate deploy && npm run start:dev"
web:
build:
@@ -52,8 +68,19 @@ services:
- /usr/src/app/node_modules
environment:
BACKEND_HOST: http://backend:3001
NEXT_PUBLIC_ARCGIS_API_KEY: ${NEXT_PUBLIC_ARCGIS_API_KEY:-}
NODE_ENV: development
command: npm run dev
pgadmin:
image: dpage/pgadmin4:8
environment:
PGADMIN_DEFAULT_EMAIL: admin@example.com
PGADMIN_DEFAULT_PASSWORD: admin
ports:
- "5050:80"
depends_on:
- postgres
volumes:
postgres-data:

View File

@@ -11,6 +11,23 @@ user admin
topic readwrite #
topic readwrite $SYS/#
user testuser
topic readwrite #
topic readwrite $SYS/#
# Backend service: reads all device traffic, writes job acks back to devices,
# and publishes on behalf of the /sim simulator tool (devices/<serial>/log).
# Also ingests the SRS §3.4.2 ul/# namespace (app + device-direct) and writes the
# application-level acks back on ul/{orgId}/{clientClass}/{clientId}/ack.
user backend
topic read devices/#
topic write devices/+/jobs/ack
topic write devices/+/log
topic read ul/#
topic write ul/+/+/+/ack
# --- App-client namespace confinement (S2-a / SRS §3.4.2 Topic scheme) --------------
# The interim app credential uses username == orgId, so %u confines it to that tenant.
# The '+' clientId wildcard is the accepted Sprint 2 tradeoff: clients within one org share
# the credential and can see that org's ack topics until OIDC-derived per-client identity lands.
# Publish/read permissions are narrower than readwrite '#': only durable point input and acks.
pattern write ul/%u/app/+/log/points
pattern read ul/%u/app/+/ack
# testuser is a demo *device*: only the per-device pattern rule above applies

View File

@@ -2,34 +2,39 @@ per_listener_settings true
# Plain MQTT — internal services and clients authenticate with username/password on port 1883
listener 1883 0.0.0.0
password_file /mosquitto/config/passwd
acl_file /mosquitto/config/devices.acl
password_file /run/secrets/mosquitto_passwd
acl_file /run/secrets/mosquitto_acl
allow_anonymous false
# WebSocket — browser clients, no authentication required
# Authenticated MQTT over WebSocket for app clients. Docker binds this listener only to
# host loopback; nginx supplies the public WSS/TLS endpoint at /mqtt on port 443.
listener 9001 0.0.0.0
protocol websockets
allow_anonymous true
password_file /run/secrets/mosquitto_passwd
acl_file /run/secrets/mosquitto_acl
allow_anonymous false
# TLS MQTT — devices authenticate with client certificates (port 8883)
# require_certificate true forces client cert; cert CN becomes the MQTT username.
# ACL restricts each device to devices/<serial_number>/#
# listener 8883 0.0.0.0
# cafile /mosquitto/certs/ca.crt
# certfile /mosquitto/certs/server.crt
# keyfile /mosquitto/certs/server.key
# require_certificate true
# use_identity_as_username true
# allow_anonymous false
# acl_file /mosquitto/config/devices.acl
# Certs are issued by the backend's certificates module (see backend/src/certificates/)
# into ./mosquitto/certs — requires a broker restart after CA init/provisioning
# since there's no config/cert hot-reload.
listener 8883 0.0.0.0
cafile /mosquitto/certs/ca.crt
certfile /mosquitto/certs/public-fullchain.pem
keyfile /mosquitto/certs/public-privkey.pem
require_certificate true
use_identity_as_username true
allow_anonymous false
acl_file /run/secrets/mosquitto_acl
# TLS MQTT — admin access via username/password, no client cert required (port 8884)
# Connect with CA cert for server verification, then username/password.
# listener 8884 0.0.0.0
# cafile /mosquitto/certs/ca.crt
# certfile /mosquitto/certs/server.crt
# keyfile /mosquitto/certs/server.key
# require_certificate false
# password_file /mosquitto/config/passwd
# allow_anonymous false
# acl_file /mosquitto/config/devices.acl
# TLS MQTT — app/admin username+password access (port 8884). App usernames are orgIds;
# devices.acl confines them to ul/{orgId}/app/... . No anonymous listener is exposed.
listener 8884 0.0.0.0
certfile /mosquitto/certs/public-fullchain.pem
keyfile /mosquitto/certs/public-privkey.pem
require_certificate false
password_file /run/secrets/mosquitto_passwd
allow_anonymous false
acl_file /run/secrets/mosquitto_acl

View File

@@ -1,5 +1,4 @@
brent:$7$101$PPXeFRMHZwxYKE8F$/+qnOjXhFdxQYuiPxFrtVKdDME2ddCogKcOyV/Q/BYdw8UB8LJcfX23ghcVpr8cifK0z6/DZcgo0TORpungMOA==
admin:$7$101$3g0kY+V60o3BKzNi$fphdbnq1TwE7nFTuZHPu86K6619owoIUNZK/w+6iE77utXhgCj/zTRmWoCzCbbvNZRnGLac2PZqdcMGGDXHAWQ==
# Test MQTT user
testuser:$7$101$bKvyrR98MaR7giOp$RcwsxHrBENyVBWSCkzQo0hGA8nLh3CMaTi1GtTpvUzky7D2cReWc68dujesEf+PMh6EWIufl0D4YnPmiAwDSWw==
backend:$7$1000$oUW5cxuZZxX50zTybgkJFsr4dwTAGgoPNwmmlw1Q6zTTtqFfnVX9akCJZcFGb3b/anFpeBcO4AfrnVJEiXClyg==$/JsO5qrshpNsHqfrKy1B4a8NJOaaCrWqHYPfCZd95l4mq1zntjCiM/lZo4TE3YklOaouQidMjxmgXKchQPrNbg==
testuser:$7$1000$D6b1NWJ2AqoYF1zCBfJetGS13J1SksxPouVO4CPCCwijMkpS0bZDU+GIA13kvvGoikcfxAI9h2JE/BQDQR9EFw==$MCLvYJh6QHlrcE75cyaYwkHkFuEW4Z0gROrLKjeFiAhnTGR2S2cLQsk22URZDxG3LnnL6E8rZ1IcWRv1mwhIRg==

View File

@@ -6,18 +6,6 @@ server {
root /var/www/certbot;
}
location /mqtt {
# proxy MQTT over WebSocket (strip the /mqtt prefix so Mosquitto sees /)
proxy_pass http://127.0.0.1:9001/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_read_timeout 300s;
}
location / {
return 301 https://$host$request_uri;
}
@@ -33,6 +21,8 @@ server {
client_max_body_size 20m;
location /mqtt {
# Authenticated, ACL-confined MQTT-over-WebSocket. Mosquitto is bound to loopback;
# nginx terminates publicly trusted TLS so phones can use standard port 443.
proxy_pass http://127.0.0.1:9001/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;

View File

@@ -1,7 +1,22 @@
"""Publish sample locate points to the UlHub backend via MQTT.
The backend subscribes to devices/# and expects points at
devices/{mqttUsername}/points with a JSON body:
{"serial": "...", "ticket": "TKT-...",
"points": [{"lat": .., "lng": .., "ts": "ISO8601", ...}]}
The MQTT username identifies the publisher (app/gateway) and its org; "serial"
names the locator receiver the readings came from (auto-registered on first
sight). Points for an unknown ticket auto-create a stub job (source=DEVICE).
"""
import random
import json
import os
import time
from datetime import datetime
from datetime import datetime, timezone
import paho.mqtt.client as mqtt
@@ -9,27 +24,57 @@ BROKER_HOST = os.getenv("MQTT_HOST", "127.0.0.1")
BROKER_PORT = int(os.getenv("MQTT_PORT", "1883"))
USERNAME = os.getenv("MQTT_USERNAME", "testuser")
PASSWORD = os.getenv("MQTT_PASSWORD", "testpass")
TOPIC = os.getenv("MQTT_TOPIC", "devices/demo")
SERIAL = os.getenv("SERIAL", "DEMO-0001")
TICKET = os.getenv("TICKET", "TKT-2026-0001")
INTERVAL = float(os.getenv("MQTT_INTERVAL", "2"))
client = mqtt.Client()
if USERNAME:
client.username_pw_set(USERNAME, PASSWORD)
# Walk northeast from this location, one point per interval
START_LAT = float(os.getenv("START_LAT", "33.15012345"))
START_LNG = float(os.getenv("START_LNG", "-96.83512345"))
TOPIC = f"devices/{USERNAME}/points"
client = mqtt.Client()
client.username_pw_set(USERNAME, PASSWORD)
client.connect(BROKER_HOST, BROKER_PORT, 60)
client.loop_start()
try:
counter = 0
seq = 0
while True:
payload = {
"message": f"hello from host {counter}",
"timestamp": datetime.utcnow().isoformat() + "Z",
"serial": SERIAL,
"ticket": TICKET,
"points": [
{
"lat": START_LAT + seq * 0.0000135,
"lng": START_LNG + seq * 0.0000042,
"alt": 187.4 + seq * 0.02,
"fix": "FIXED_RTK",
"hAcc": 0.014,
"vAcc": 0.021,
"sats": random.randint(18, 26),
"hdop": 0.7,
"depth": round(1.1 + random.uniform(0, 0.3), 2),
"freqHz": 33000,
"currentMa": round(50 - seq * 0.4 + random.uniform(-1, 1), 1),
"signalDb": round(62 - seq * 0.2 + random.uniform(-0.5, 0.5), 1),
"gainDb": 40,
"mode": "PEAK",
"compassDeg": round(17.5 + random.uniform(-3, 3), 1),
"distortionPct": round(random.uniform(2, 8), 1),
"utility": "GAS",
"seq": seq + 1,
"ts": datetime.now(timezone.utc).isoformat(),
}
client.publish(TOPIC, json.dumps(payload), qos=0)
print(f"published to {TOPIC}: {payload}")
counter += 1
],
}
client.publish(TOPIC, json.dumps(payload), qos=1)
print(f"published point {seq + 1} to {TOPIC} (ticket {TICKET})")
seq += 1
time.sleep(INTERVAL)
except KeyboardInterrupt:
print("stopped")
finally:
client.loop_stop()
client.disconnect()

Some files were not shown because too many files have changed in this diff Show More