Fail2banMqttActionBanishment/docs/deployment.en.md

Production deployment

This page details deploying a master + nodes set in production, beyond the single isolated node covered in Installation.

Overview

Three ways to use this project, from simplest to most complete — the same base install (sudo make install) is the foundation for all three:

• Solo: sudo make install, alone. This server detects and bans locally, without centralizing anything elsewhere. Enough to protect a single VPS. The relay service to a master (fail2ban-emitter-master-client) still runs, but silently keeps retrying in a loop as long as no master is configured (no harm done, Restart=on-failure).
• Solo + master: sudo make install then sudo make install-master, on the same server. This VPS protects its own services (like solo mode) and also serves as a master, ready to take on other nodes later — useful for starting out alone today without closing the door on a second server tomorrow.
• Master + nodes (several VPSes): one server in solo+master mode, and one or more other servers, each installed solo and then attached to that master via token-based enrollment (below). The master itself stays a node like any other (it detects and bans locally in addition to centralizing the decision) — going from "solo+master on its own" to "master + nodes" therefore requires no reinstall of the master, just adding nodes to it over time.

Recommended order for moving to several nodes:

1. Deploy the master (once — solo+master if this same server should also protect itself, which is the common case).
2. On the master, issue a token for each new node.
3. On each new node, enroll using that token.

Deploying the master

sudo make install-master

Requires a first sudo make install already done on this same machine. Provisions:

• a dedicated private CA (scripts/master-ca-init.sh), distinct from the Let's Encrypt certificate that serves the broker's TLS identity — it only signs nodes' client certificates (mTLS authentication);
• the public Mosquitto listener (port 8883, TLS, MQTT_MASTER_DOMAIN in .env) with per-node ACLs;
• master-side ingestion (fail2ban-emitter-master-listen, runs with its own mTLS identity master-internal, generated automatically).

MQTT_MASTER_DOMAIN must point to a real public domain (DNS already set up) — the broker obtains its Let's Encrypt certificate for that name.

Token-based node enrollment (recommended)

The problem

Adding a node by hand would require generating a key + certificate on the master, then copying those files to the new node (scp), renaming them correctly, and editing its .env — several manual, cross-machine steps, and the private key would travel over the network (never a good practice).

The model

The master issues a single-use token, short-lived (1h by default). The new node uses that token to enroll itself:

• The node generates its own private key locally and never sends it — only a CSR (signing request) goes to the master.
• The token is never stored in plaintext on the master: only its SHA-256 hash is persisted (same logic as a password hash).
• The token is atomically marked used as soon as it's validated, and automatically given back if a later step fails (inconsistent CSR, signing failure) — a transient hiccup doesn't force issuing an entirely new token.
• The CN of the received CSR must match the node name declared when the token was issued — a stolen token can't be used to claim a different name.
• Any error returned to the client is deliberately generic ("invalid or expired token"), regardless of the actual cause — the detail (unknown, expired, or already-used token, inconsistent CN, signing failure) only goes into the master's logs, so as to give no hint to a third party probing the endpoint.
• The transport is the master's HTTPS dashboard (already publicly exposed, nginx + Let's Encrypt): no new port or service to open, and it solves the chicken-and-egg problem (a node can't use mTLS MQTT to obtain... its first mTLS certificate).

Step by step

On the master:

make join-token NODE=node-name

Prints the plaintext token only once, along with the full command to run on the new node (lost token: simply issue a new one, the previous one stays valid independently until it expires).

On the new node (having already gone through sudo make install):

sudo make join MASTER=https://<master-domain> NODE=node-name TOKEN=<token>
sudo make install

The second make install applies the received configuration (MQTT_MASTER_NODE_NAME in .env, certificate in place) and restarts the services.

What happens behind the scenes

[new node]                                   [master]
  generates key + CSR (local, never sent)
  POST /api/join/ {node_name, token, csr} ──►
                                              validates the token (hash, expiry, single use)
                                              checks the CSR's CN == node_name
                                              signs the CSR with the private CA
                                              adds the ACL entitlement for this node
                                              restarts mosquitto
                        {cert, ca_cert} ◄────
  writes cert/ca_cert, updates .env

Troubleshooting

• "Invalid or expired token" — the message is deliberately generic. Possible causes: token already used (run make join-token again to issue a new one), token expired (1h default, adjustable via make join-token NODE=... TTL=<minutes>), or a typo in the node name (must be identical between token issuance and make join).
• The new node must be able to reach the master over HTTPS (443) — a firewall blocking outbound traffic would prevent enrollment.

Databases

SQLite is the default, nothing to configure. Two alternatives:

sudo make install-mariadb   # provisions a local MariaDB server + dedicated database/user
sudo make install           # re-run afterward to apply migrations on the new database

PostgreSQL is supported at the configuration level (DB_ENGINE=postgresql in .env, driver emitter/requirements-postgresql.txt) but without an automated provisioning subcommand for now — server and database must be created manually before switching DB_ENGINE.

Migrating data from one backend to another (engine-agnostic dump via manage.py dumpdata/loaddata):

make dump-db   # writes emitter/db-dump-<date>.json (git-ignored, contains real data)
# ... switch DB_ENGINE, migrate ...
emitter/.venv/bin/python emitter/manage.py loaddata <file>.json

nginx reverse proxy + public TLS

sudo make install-nginx   # reads DASHBOARD_DOMAIN from .env

Obtains a Let's Encrypt certificate and deploys an nginx vhost in front of the dashboard (WebSocket included) — needed for public access; without it, the dashboard is only reachable locally/over VPN.

Alternative to systemd: supervisor

sudo make install-supervisor     # switches the emitter's 4 services to supervisor
sudo make uninstall-supervisor   # rollback (re-enables systemd)

Updating

git pull
sudo make install

Reapplies dependencies/migrations and restarts the affected services, on any node (including the master).