Fail2banMqttActionBanishment/README-EN.md

# Fail2banActionBanishment

Distributed malicious-host banishing system built on **fail2ban** and
**iptables**, with event propagation over **MQTT**.

Each node detects and bans locally (fail2ban + iptables), publishes the event
to a local Mosquitto broker, then relays it to a private **master** broker.
The master centralizes the decision (multi-node correlation, global recidive
detection, shared reputation) and republishes the actions to be executed to
all subscribed nodes.

Target machine: **Debian 13 VPS**.

Full documentation (installation, detailed production deployment,
token-based node enrollment, MQTT topics and commands reference):
`make docs-serve` then <http://127.0.0.1:8001/>, or `make docs` to
generate the static site into `site/`. See also [README.md](README.md)
for the French version.

## Architecture (overview)

```
[node] fail2ban/iptables → local mosquitto (1883) → master mosquitto (8883, TLS)
                                                              │
                                                    decision / correlation
                                                              │
[node] ◄──────────────── MQTT action (BAN, SYNC_BAN, UNBAN, ...) ◄──────────
```

## Use cases

The master/nodes model doesn't assume any ownership relationship between
the protected servers — only that they share a common decision authority.
That makes the architecture relevant beyond a single operator running
their own VPSes:

- **A set of sites** (several VPSes run by the same operator, each hosting
  one or more services): an IP attacking one site gets banned everywhere
  via `SYNC_BAN`, without manual per-site monitoring.
- **A community** (several independent administrators, each responsible
  for their own server, who agree to pool their detections): every member
  keeps their own node and their own local fail2ban — only ban
  propagation is shared through the common master. No member gets SSH
  access to another member's server.
- **An IT services company** managing several clients' sites: a
  centralized master gives an aggregated view (jail/country counters,
  per-node history, per-client aliases) without mixing client data —
  each node only sees its own events, the master sees everything.

In all three cases, a new server joins the set without ever handing out
SSH access to the others: a single-use token-based enrollment is enough
(`make join-token` / `make join`, detailed below).

## Repository layout

- `generic/fail2ban/` — jails, filters and the MQTT action (`f2b-mqtt-action-banisher`)
- `generic/firewall/` — iptables launcher script (`firewall-launcher.sh`), per-service rules
- `generic/mosquitto/` — local MQTT broker (`127.0.0.1:1883`, no persistence) and master broker (`mosquitto-master.conf`, port 8883, mTLS)
- `generic/emitter/` — systemd service templates (web/ASGI, MQTT ingestion, Celery worker, master relay)
- `generic/supervisor/` — `.example.conf` templates (optional alternative to systemd, `make install-supervisor`)
- `generic/nginx/` — `.example.conf` TLS reverse-proxy template for the dashboard (not deployed automatically)
- `emitter/` — Django emitter (`banevents` app): MQTT ingestion (`manage.py mqtt_listen`), master relay (`manage.py master_client`), `BanEvent` model, dashboard + map + history, geolocation (Celery)
- `.env.example` — configuration template (copied to `.env` by `install.sh`, never committed)
- `install.sh` — idempotent deployment (packages, config files, systemd services, including the emitter); wrapped by the `make` targets below
- `scripts/test-fail2ban-filters.sh` — custom filter checks via `fail2ban-regex`
- `scripts/master-ca-init.sh`, `scripts/generate-node-cert.sh` — private CA and mTLS client certs for the master broker (manual flow, see token-based enrollment below for the recommended flow)
- `Makefile` — every entry point (`make help` for the full list)
- `docs/` — mkdocs documentation (installation, production deployment, reference)

## Installation (Debian 13 VPS)

Deployment model: **no copy into `/opt`**. The repo cloned into a dedicated
user's `$HOME` (e.g. `banisher`) *is* the deployment — venv, SQLite
database, `staticfiles/` and `.env` all live inside that checkout, owned by
that user. `install.sh` detects this user from the repo's actual owner (no
hardcoded name):

```sh
useradd -m -s /bin/bash banisher   # once, if the account doesn't exist yet
su - banisher -c 'git clone <repo-url> ~/Fail2banMqttActionBanisher'
cd ~banisher/Fail2banMqttActionBanisher
sudo make install
sudo make status
```

`make install` (= `sudo ./install.sh install`):
- installs/checks a `NOPASSWD:ALL` sudoers entry for that user
  (`/etc/sudoers.d/<user>-full-access`, created once if missing) — this
  account should therefore be **dedicated to this project**, not shared with
  other uses;
- copies `.env.example` to `.env` if missing (never regenerated afterward,
  `DJANGO_SECRET_KEY` generated randomly) — edit `.env` for real credentials
  (MQTT password for the `emitter` account, master domain, ...);
- creates the emitter's Python venv (`emitter/.venv`), applies migrations
  and `collectstatic`, all run as the deployment user (never root);
- enables four systemd services: `fail2ban-emitter-web` (ASGI/Daphne
  dashboard, `127.0.0.1:8050`), `fail2ban-emitter-mqtt`
  (`manage.py mqtt_listen`), `fail2ban-emitter-worker` (Celery worker,
  geolocation) and `fail2ban-emitter-master-client` (`manage.py
  master_client`, relay/execution of the master's commands — won't
  actually connect until a node certificate has been obtained, see
  below).

Also edit `/etc/fail2ban/mqtt.conf` (real MQTT credentials, copied from
`generic/fail2ban/mqtt.conf.example` if it doesn't exist yet).

To update the emitter after a `git pull`, just rerun `sudo make install`:
dependencies/migrations are reapplied and all four services restarted.

Additional deployment options (detailed in the [deployment
documentation](docs/deployment.en.md), `make docs-serve` for the
browsable version):
- **master** (`sudo make install-master`) — public mTLS MQTT broker,
  private CA, multi-node ingestion;
- **token-based node enrollment** (`make join-token` on the master, then
  `sudo make join` on the new node) — recommended way to attach a node to
  a master, without ever copying a certificate by hand;
- **nginx** (`sudo make install-nginx`) — TLS reverse proxy (Let's
  Encrypt) in front of the dashboard, for public access;
- **supervisor** (`sudo make install-supervisor` / `sudo make
  uninstall-supervisor`) — optional alternative to systemd;
- **MariaDB** (`sudo make install-mariadb`) — switches the database from
  SQLite (default) to a dedicated local MariaDB server.

## Django emitter (development)

Requires a reachable Redis (Django Channels channel layer, e.g. `docker run
-p 6379:6379 redis`).

```sh
make run       # dashboard + map at http://127.0.0.1:8000/ (real-time WebSocket included)
make mqtt      # in a second terminal: MQTT ingestion
make worker    # in a third terminal: asynchronous geolocation
```

Environment variables (`MQTT_BROKER_HOST`, `MQTT_BROKER_USERNAME`,
`MQTT_BROKER_PASSWORD`, `REDIS_HOST`, `REDIS_PORT`, ...): copy
`.env.example` to `.env` at the repo root and edit it — loaded
automatically by Django (`python-dotenv`), nothing to export by hand.
Geolocation uses the free ip-api.com fallback API until a local GeoLite2
(MaxMind) database is configured.

## Translations (fr/en)

French is the source language of the template code; English is a
translation maintained in `emitter/locale/en/LC_MESSAGES/django.po`. A
selector in the dashboard's sidebar switches the active language
(`django_language` cookie).

After adding/changing a `{% trans %}`/`{% blocktrans %}` in a template:

```sh
cd emitter
.venv/bin/python manage.py makemessages -l en --no-location --ignore ".venv"
# edit locale/en/LC_MESSAGES/django.po (fill in the missing msgstr entries)
.venv/bin/python manage.py compilemessages -l en --ignore ".venv"
```

`--ignore ".venv"`: without it, `compilemessages` also recompiles (slowly,
with no useful effect) every one of Django's own bundled translations found
inside the venv, since it lives under `emitter/` — harmless but pointless.

## Master broker and node enrollment

See the [deployment documentation](docs/deployment.en.md) for the full
detail: master provisioning, the enrollment security model (single-use
token, private key that never leaves the node, token automatically given
back on a transient failure), troubleshooting. In short:

```sh
# on the master
make join-token NODE=my-node

# on the new node
sudo make join MASTER=https://master-dashboard NODE=my-node TOKEN=<token>
sudo make install
```

Master-side ingestion (`manage.py master_listen`) systematically
republishes a `SYNC_BAN` for every ingested ban, and correlates
independent bans across nodes to trigger an automatic escalation — see
the [reference](docs/reference.en.md) for the full list of master
commands and MQTT topics.

## Development constraints

- Source code in English, comments in French
- Indentation: 4 spaces, never tabs, see `.editorconfig`
  (exception: Makefile recipes, which must be tab-indented — a constraint of
  Make itself)
- Python type-checked with `pyright` (`pyrightconfig.json`), shell scripts
  with `shellcheck` — `make lint`, also run in CI
- JavaScript: ES6 classes

## License

AGPL-3.0 — see [LICENSE](LICENSE).