Denis Defolie
29 KiB
Local voice assistant — ESP32 + Raspberry Pi
A fully local voice assistant, with no cloud service: an ESP32 captures speech, streams raw PCM over MQTT to a Python server that transcribes, interprets and answers with speech synthesis; the ESP32 plays the answer back and executes the command.
Version française : README.md.
1. Architecture
INMP441 -> ESP32 -> MQTT -> Python server (Raspberry Pi / PC)
|
+---------+---------+
| |
VAD Diarization /
| identification
+---------+---------+
|
faster-whisper
|
Assistant
|
Piper
|
MQTT audio/out
|
ESP32 -> MAX98357A -> speaker
Four often-confused stages, never to be mixed up:
| Stage | Question it answers |
|---|---|
| VAD | Is someone speaking? |
| Diarization | Who speaks when? (SPEAKER_00, SPEAKER_01…) |
| Identification | Who is this person? (Denis, Marie, Unknown) |
| STT | What is being said? |
Voice identification is probabilistic. It is never an authentication or security mechanism.
2. Hardware
| Item | Detail |
|---|---|
| ESP32-WROOM-32 | two I2S peripherals: I2S0 as receiver, I2S1 as transmitter |
| INMP441 | I2S microphone, 3.3 V, L/R pin to GND (left channel) |
| MAX98357A | mono class-D amplifier, 5 V, SD tied to VIN, GAIN left floating (9 dB) |
| Button | push-to-talk between GPIO 4 and GND (internal pull-up) |
| LED | with a 330 Ω resistor on GPIO 2 |
| Server | Raspberry Pi 4/5 64-bit or Debian PC |
Wiring
Full diagram: docs/wiring.svg — detailed pinout, power budget and multimeter checks (in French): docs/BROCHAGE.md.
ESP32 INMP441 (microphone)
3V3 ----------------------- VDD
GND ----------------------- GND, L/R
GPIO 25 (SCK) ------------- SCK
GPIO 26 (WS) ------------- WS
GPIO 33 (SD) ------------- SD
ESP32 MAX98357A (amplifier, step 2)
5V (VIN) ------------------- VIN
GND ----------------------- GND
GPIO 27 (BCLK) ------------- BCLK
GPIO 14 (LRC) ------------- LRC
GPIO 22 (DIN) ------------- DIN
ESP32 User interface
GPIO 4 ------ button ------ GND
GPIO 2 --[330 Ω]-- LED ---- GND
No strapping pin (GPIO 0, 2, 5, 12, 15) carries an audio signal: their level at reset selects the boot mode. GPIO 2 drives the LED as an output only, which does not affect booting.
3. Audio format, identical everywhere
| Parameter | Value |
|---|---|
| Sample rate | 16 000 Hz |
| Resolution | signed 16-bit, little-endian (pcm_s16le) |
| Channels | 1 (mono) |
| Bit rate | 32 000 bytes/s |
| Chunk | 50 ms = 800 samples = 1 600 bytes |
No Base64: MQTT payloads carry raw PCM bytes.
4. Installation
4.1 ESP32 firmware
Prerequisites: a USB cable with data lines (not charge-only), port identified
(ls /dev/ttyUSB* or /dev/ttyACM* after plugging in).
make pio-venv # PlatformIO in its own venv — once only
make fw-build # compile without flashing
make fw-upload SERIAL_PORT=/dev/ttyUSB0
First boot — WiFi configuration portal
If no credentials have been configured yet, the board automatically opens an access point:
- The LED blinks rapidly (100 ms on / 100 ms off).
- Connect a phone or PC to the VoiceAssist-Config Wi-Fi network.
- Open
http://192.168.4.1in a browser. - Enter the Wi-Fi network (2.4 GHz required) and the MQTT broker IP address.
- Click Save and restart — credentials are saved to persistent memory (NVS) and the board reboots.
Reconfiguration: hold the PTT button for 3 s at power-on to reopen the portal, even if credentials are already stored.
Alternative — secrets.h (for repeated development builds)
cp firmware/include/secrets.h.example firmware/include/secrets.h
# Edit: WIFI_SSID, WIFI_PASSWORD, MQTT_HOST
make fw-upload SERIAL_PORT=/dev/ttyUSB0
NVS takes priority over secrets.h. To start from scratch, force the portal
with the PTT button (or erase the NVS with esptool.py erase_flash).
Raspberry Pi 5 Wi-Fi access point: WPA2 required
If the board connects to the Pi's access point (NetworkManager, here RPI5-AP),
it must use WPA2. By default the ESP32 refuses any network below WPA2, and
an access point created by NetworkManager may be WPA1. Recommended fix, on the Pi:
nmcli -t -f NAME,TYPE con show --active # name of the access point connection
sudo nmcli con modify "RPI5-AP" wifi-sec.key-mgmt wpa-psk wifi-sec.proto rsn \
wifi-sec.pairwise ccmp wifi-sec.group ccmp \
802-11-wireless.band bg 802-11-wireless.channel 6
sudo nmcli con down "RPI5-AP" && sudo nmcli con up "RPI5-AP"
proto rsn enforces WPA2; channel 6 avoids channels 12 and 13, which the ESP32
handles poorly. The Wi-Fi password does not change. On this network the Pi is
10.42.0.1: that is the broker address to enter in the portal.
Diagnosis on the serial monitor (make fw-monitor, or minicom at 921600 baud):
| Board message | Cause | Fix |
|---|---|---|
Reason: 211 |
access point uses WPA1 | switch it to WPA2 (above) |
Reason: 15 - 4WAY_HANDSHAKE_TIMEOUT |
wrong Wi-Fi password stored in the board | re-enter it in the portal; the Pi confirms it: sudo journalctl -u wpa_supplicant | grep PSK-MISMATCH |
# mqtt : echec (code 5) |
MQTT account missing on the broker, or different password | make mqtt-user NAME=esp32-01 on the Pi |
| unreadable bytes when the button is pressed | MQTT not connected: the session goes over the serial cable | check the # wifi and # mqtt lines at boot |
In the portal, both password fields (Wi-Fi and MQTT) are always shown empty: retype them on every save, a field left empty erases the stored password.
4.2 Python server
Prerequisites: Python 3.10+, Docker (for the broker) or native Mosquitto.
# 1. System packages (mosquitto, ffmpeg, venv) + serial port access (dialout group)
make install-system
# → log out and back in to activate the "dialout" group
# 2. MQTT broker (choose one)
make mosquitto-docker # Docker container (recommended)
make mosquitto-config # native Debian Mosquitto service (installed via make install-system)
# 3. Python environment + dev tooling (pytest, pyright, PlatformIO)
make install-dev
# PlatformIO gets its own venv (.pio-venv): its dependencies clash with the
# voice server's.
# 4. Speech recognition + synthesis (~550 MB)
make install-speech
make models
# 5. Diarization + speaker identification (~1.3 GB) — optional, heavy on RPi 4
# Prerequisite: Hugging Face "Read" token in server/.env (HF_TOKEN=hf_…)
# Accept the conditions at https://huggingface.co/pyannote/speaker-diarization-community-1
cp server/.env.example server/.env # then HF_TOKEN=hf_…
make install-diarization
make models
# 6. Wake word openWakeWord (~5 MB) — optional
make install-wakeword
make models
# 7. Run the assistant
make run
Files to configure:
| File | Contents |
|---|---|
server/.env |
MQTT_USERNAME, MQTT_PASSWORD, HF_TOKEN |
server/config.yaml |
rooms, voices, access rights, machine list |
For help on all targets: make help.
4.3 Linux agent (machines controlled by voice)
Each Linux machine to control (desktop PC, NAS…) runs a lightweight service that executes only the actions listed in its own configuration file — no shell exposed.
On the target machine:
# 1. Copy the agent/ directory from the Raspberry Pi
scp -r /home/rpi5/voice-assistant/agent/ user@desktop-pc:~/voice-agent-install/
# 2. Install the service (creates /opt/voice-agent/, /etc/voice-assistant/, systemd unit)
cd ~/voice-agent-install
sudo make install
# 3. Create the MQTT account on the server (if the broker is secured, step 15)
# (run on the Raspberry Pi, not on the target machine)
make mqtt-user NAME=pc-bureau # note the password
make mosquitto-docker # or mosquitto-config — reloads the broker
# 4. Configure the agent on the target machine
sudo nano /etc/voice-assistant/agent.yaml
# id: pc-bureau ← name used in MQTT topics
# mqtt.host: 192.168.1.200 ← broker IP address
# mqtt.username: pc-bureau ← same value as id (empty if anonymous broker)
# mqtt.password: <password> ← generated in step 3 (empty if anonymous broker)
# dry_run: false ← true to test without executing anything
# 4. Enable and start
sudo systemctl enable --now voice-agent
sudo systemctl status voice-agent
On the server, add the machine to server/config.yaml:
machines:
list:
- id: pc-bureau
name: "PC du bureau" # spoken as: "le PC du bureau"
mac: "AA:BB:CC:DD:EE:FF" # MAC address for Wake-on-LAN
ip: 192.168.1.50
Try it without shutting anything down (dry_run: true in agent.yaml):
make run # server (terminal 1)
make agent-run # agent in test mode on this machine (terminal 2)
# Say: "Hey Mycroft, éteins le PC du bureau"
# → the agent prints: essai : systemctl poweroff (non exécuté)
Default available actions: shutdown, reboot, lock; wake
(power on over the network) is handled by the server via Wake-on-LAN.
5. Usage
make fw-build # build the firmware
make fw-upload SERIAL_PORT=/dev/ttyUSB0
make record OUT=recordings/test.wav # record one session (step 1)
make play OUT=recordings/test.wav # play it back (picks a working player)
make inspect OUT=recordings/test.wav # analyse it without listening
make play-esp OUT=recordings/test.wav # play it on the ESP32 speaker (step 2)
make tone-esp TONE=440 # pure tone on the ESP32 speaker (step 2)
make run # assistant: answers every question (step 6)
make speech-check TEXT="Quelle heure est-il ?" # recognition and synthesis, no ESP32
make say WAKE="Hey Mycroft" TEXT="Ouvre la porte du salon." VOICE=fr_FR-tom-medium # WAV + MP3 for a phone
make latency # server latency before/after step 16, on real recordings
make transcribe OUT=server/recordings/<file>.wav
make vad OUT=server/recordings/<file>.wav # speech found by the VAD (step 7)
make vad-all # VAD over every server recording
make diarize OUT=server/recordings/<file>.wav # who speaks when, who, who said what (steps 8-10)
make diarize-demo # two voices: yours and Piper's
make enroll NAME=Denis # voice profile from the last 5 sessions (step 9)
make profiles # list profiles; make forget NAME=Denis deletes one
make mqtt-watch # text messages voice/… and home/# (no audio)
make mqtt-user NAME=esp32-01 # MQTT account for a board, or the server (step 15)
make agent-run # Linux agent of this PC, test mode (step 15b)
make mqtt-agents # agent states, commands and reports
make fw-monitor # serial monitor
make test # Python tests
make lint # pyright type check
6. Progress
The exact state of the project, step by step, lives in CONTEXT.md.
Current step: 16 — optimisation. Steps 1 to 15b are validated.
Step 16 — optimisation
Measure first, optimise next: make latency replays the latest recordings through the
old chain, then the new one. On 54 real commands (PC, 20 logical cores): 1.6 s → 1.1 s
between the end of the sentence and the recognised text, same text 54 times out of 54.
| Optimisation | Measured gain | Setting |
|---|---|---|
| Short speech: direct voice embedding, no diarization | 0.40 s (0.42 → 0.02 s) | diarization.direct_below_s: 4.0 |
| Short speech: Whisper without word timestamps | 0.10 s | automatic |
| Answers already spoken kept in memory (synthesis cache) | 0.03 to 0.14 s | automatic, 64 sentences |
| End of command checked every 100 ms (250 before) | 0.075 s on average | — |
Immediate network sends (TCP_NODELAY: server, broker) |
to be measured: premier son a … ms |
set_tcp_nodelay true in voice.conf |
Measured, then rejected: diarization and Whisper in parallel (0.03 to 0.10 s only: both
computations saturate the processor, even in two processes), beam_size: 1 (0.05 s, and one
different text out of 16), 16 threads for Whisper (slower than 4).
- On the Raspberry Pi (broker):
git pull, thenmake mosquitto-config. make run, a few commands: the log shows1 locuteur (parole courte, empreinte directe 20 ms)andreponse prete en ~1.1 s(1.5 s before).make latency: before, after, gain, and the identifications that differ.
Speech shorter than 4 s is assumed to come from a single person. Two voices in a short sentence are no longer separated (they were at step 10); beyond 4 s nothing changes.
Step 15b — Linux machines on the network
"Hey Mycroft, éteins le PC du bureau": each machine runs an agent
(agent/voice_agent.py), a small MQTT client that only runs
the actions in its own list (/etc/voice-assistant/agent.yaml): fixed commands,
no shell. The server only sends an action name. "Allume" wakes a machine that is
off through Wake-on-LAN (a network packet received by the sleeping network card).
| Action | Words | Rights (config.yaml) |
|---|---|---|
wake (server) |
allume, démarre, réveille | standard |
lock |
verrouille | standard |
shutdown |
éteins, arrête | full + confirmation |
reboot |
redémarre, relance | full + confirmation |
- Try it on this PC without shutting anything down:
make agent-run(createsagent/agent.yamlin test mode). make run, then "Hey Mycroft, éteins le PC du bureau" → confirmation → "oui": the agent printsessai : systemctl poweroff (non execute).- Real machine: copy
agent/to it,make -C agent install, edit/etc/voice-assistant/agent.yaml, and name it inserver/config.yaml(machines.list).
Anonymous broker: any device on the network can then publish on agent/<machine>/command.
In production, use the accounts and rights of step 15 (mosquitto/acl has the agent rules).
Step 15 — reliability
| What | How |
|---|---|
| MQTT accounts and rights | no anonymous access; a board reads and writes its own topics only (docs/MOSQUITTO.md §2) |
| Server presence | voice/server/status: retained online, offline last will; server gone = board microphones off |
| Watchdog | board frozen for more than 15 s → reboot, reason watchdog in the boot event |
| Network | TCP connection to the broker within 1 s; Wi-Fi restarted after 30 s without it |
| Log | copied to server/logs/voice-server.log (rotating files, 4 MB at most) |
Setup, in this order: a broker that is still open already accepts clients that have an account.
make mqtt-user NAME=voice-server, thenmake mqtt-user NAME=esp32-01(passwords:openssl rand -hex 16).server/.env:MQTT_USERNAME=voice-server,MQTT_PASSWORD=...;secrets.h:MQTT_PASSWORD="...".make fw-uploadandmake run: everything works as before, the server announces its presence.make mosquitto-docker: the broker now requires the accounts.
Step 14 — per-profile rights
| Level | Rights | Given to |
|---|---|---|
| complet (full) | everything; sensitive actions (open the door) after a "oui" confirmation | profiles listed in access.users, if the voice is clearly recognised (score ≥ 0.6) |
| standard | lights, shutters, closing the door | enrolled profile without a level (access.default) |
| limite (limited) | conversation only | unknown voice (access.unknown) |
Rules per device and action in config.yaml (section access); an action missing from the
rules requires the full level. Names in access.users are those shown by make profiles.
make run(server only, firmware unchanged).- "Hey Mycroft, allume la lumière" → executed (Denis, full).
- "Hey Mycroft, ouvre la porte" → "Confirmez-vous : ouvrir la porte du salon ?" → "Hey Mycroft, oui" → "J'ouvre la porte du salon."; "non" → "D'accord, j'annule."
- A phone voice (standard profile): "ouvre la porte" → refused; a non-enrolled voice:
"allume la lumière" → "Je ne vous ai pas reconnu…". Ready-made sentences:
recordings/voix-test/tom-porte.mp3,pierre-lumiere.mp3; more withmake say. The wake word is spoken by an English voice (WAKE): a synthetic voice saying it the French way is not recognised.
Reminder: a voice is never proof of identity. These rights prevent mistakes and guests' commands, not an attack (recording, impersonation).
Step 13 — wake word (hands-free)
The server asks the board to stream its microphone continuously (voice/<board>/audio/stream)
and listens for the wake word with openWakeWord: "Alexa", "Hey Mycroft" or "Hey
Jarvis", pronounced the English way. Once heard, it keeps the command until the final
silence, then handles it like a push-to-talk session. The button still works at any time.
LED: short flash every 2 s = continuous listening; on = wake word heard, command in progress; slow blink = ready, no continuous listening.
make install-wakewordthenmake models(5 ONNX models, 5 MB).make fw-upload SERIAL_PORT=/dev/ttyUSB0(step 13 firmware), thenmake run.- "Alexa, allume la lumière" → LED on during the command, then answer and command.
- A sentence without the wake word → nothing happens, nothing is recorded.
- "Alexa" alone, then silence → back to standby after 4 s.
Privacy: before the wake word, only the last 500 ms are kept in memory, nothing is
written to disk (checked by a test). But audio flows continuously over the network: as long
as the broker accepts anonymous clients (step 15), anyone on the LAN can listen to it.
Turn listening off: wake.enabled: false, or
make mqtt-control MSG='{"cmd":"listen","enabled":false}'.
Step 12 — anti-jitter output buffer
The board plays nothing until it holds 300 ms of audio in reserve (priming); the
Wi-Fi may then deliver chunks late or in bursts without it being heard. If the reserve
runs dry, playback stops cleanly and resumes once the reserve is rebuilt (re-priming)
instead of stuttering. The played report gives the delay to the first sample, the
minimum margin (network delay still absorbable) and the number of re-primings.
make fw-upload SERIAL_PORT=/dev/ttyUSB0(step 12 firmware).make tone-mqtt: expectedreamorcages 0, minimum margin ≈ 250 ms.make tone-mqtt JITTER=200: a 200 ms stall every 500 ms; sound still continuous,reamorcages 0, minimum margin ≈ 100 ms.make tone-mqtt JITTER=400: beyond the reserve; clean pauses (re-primings > 0), never any crackling.make run: assistant answers, withreamorcages 0at the usual signal level.
Step 11 — home automation
A command is three words: an action (allumer, éteindre, ouvrir, fermer), a device
(lumière, volets, porte) and a room (listed in config.yaml, section home). Without a
room, the board's own room is used. The server publishes home/<room>/<device>/set for any
home automation system; when the room is the board's, the board also executes it: demo LED
on GPIO 21 for the light, simulated shutters and door (serial log).
- Wire the LED (optional): GPIO 21 → 330 Ω → LED → GND (docs/BROCHAGE.md §10).
make fw-upload SERIAL_PORT=/dev/ttyUSB0(step 11 firmware), thenmake run.- In another terminal:
make mqtt-watch(voice/… as text and home/#). - "Allume la lumière" → "J'allume la lumière du salon.", LED on,
home/salon/light/setand the confirmation{"event":"device",...,"ok":true}. - "Ferme les volets de la cuisine" →
home/cuisine/shutter/setonly (another room). - "Allume les volets" → "Je ne sais pas allumer les volets."; "Ouvre" → "Que dois-je ouvrir ?".
Security: voice identification is never proof. Do not connect a door strike or an alarm to these commands; per-profile rights come in step 14.
Step 10 — who said what
Whisper gives the time of each word, the diarization the time of each speaker turn, on
the same audio: each word goes to whoever was speaking at that moment. The server
publishes the result on voice/<board>/transcript and the assistant calls the person by name.
{"session": "453cce", "utterances": [
{"speaker": "Denis", "text": "Ouvrir la porte.", "start": 0.0, "end": 1.26, "score": 0.6},
{"speaker": "Siwis", "text": "Je suis la voix de synthèse...", "start": 1.5, "end": 4.46, "score": 0.6},
{"speaker": "Denis", "text": "Fermez la porte.", "start": 4.92, "end": 5.86, "score": 0.6}]}
make diarize OUT=server/recordings/<file>.wav: who said what, and the MQTT message, no board needed.make mqtt-watchin one terminal,make runin another.- "Bonjour": the board answers "Bonjour Denis !";
voice/esp32-01/transcriptshows up. - A two-voice session (you, a phone voice, you again): three utterances.
Step 9 — speaker identification
Each voice found by the diarization is summarised by an embedding (256 numbers, WeSpeaker model from the pyannote pipeline), compared by cosine similarity with the enrolled profiles. Above the threshold (0.45, measured on this bench), the name; otherwise "Inconnu". It is an estimate, never a security feature: a recording or an impersonation can fool it.
make run, then 5 varied sentences of 1 to 4 s, alone, at a normal distance.make enroll NAME=Denis: profile built from those 5 sessions; sessions that are too short, contain several voices or do not match the others are dropped.- New questions (the server picks up the profile without restarting): the log shows
SPEAKER_00 = Denis (0.74); another person or a loudspeaker voice givesInconnu (0.05, proche de Denis). make diarize-demo: your voice recognised, Piper's voice "Inconnu".
Profiles are biometric data: server/profiles/, never versioned, one readable JSON
file per person; make forget NAME=... deletes it.
Step 8 — diarization: who speaks when
The speech found by the VAD is split into speaker turns: SPEAKER_00 from 0.0 to 1.3 s,
SPEAKER_01 from 1.4 to 2.9 s... Labels only hold within one session: putting a name on
a voice is step 9. Model pyannote/speaker-diarization-community-1 (33 MB), run on CPU,
pyannote telemetry disabled (enabled by default in version 4).
- Create an account on https://huggingface.co, then accept the conditions at https://huggingface.co/pyannote/speaker-diarization-community-1.
- Create a "Read" token at https://huggingface.co/settings/tokens and put it in
server/.env:HF_TOKEN=hf_...(cp server/.env.example server/.envif missing). make install-diarizationthenmake models. No Internet access afterwards.make diarize-demo: your voice, Piper's voice, your voice → two speakers expected.make run: one session where two people take turns; the log shows2 locuteur(s) : SPEAKER_00 0.00-1.30, SPEAKER_01 1.40-2.90.
Too heavy for a Raspberry Pi 4: set diarization.enabled: false in config.yaml.
Step 7 — voice activity detection (VAD)
The server now transcribes speech only: leading and trailing silence is removed, sessions without speech (silent press, click) are dropped without calling Whisper. Silero ships with faster-whisper: nothing more to install.
make vad-all: one character per 32 ms window (#speech,_silence) for each recording, plus the number of sessions without speech.make vad OUT=server/recordings/<file>.wavthenmake play OUT=recordings/parole.wav: speech only, silences removed.make run(restart it: the code changed), ask a few questions and press once without speaking. The log showsparole 1.36 s sur 2.00 s (segments 0.49-1.85 s), andsans parole (VAD)for the silent press, answered immediately.
Settings in config.yaml, section vad: threshold, minimum pause, minimum speech, padding.
Step 6 — MVP: the assistant answers
make install-speechthenmake models(faster-whispersmall, Piper voicefr_FR-siwis-medium, about 550 MB inserver/models/, once).make speech-check: Piper says "Quelle heure est-il ?", Whisper reads it back, the assistant answers; no hardware needed.make transcribe OUT=server/recordings/<file>.wav: transcribes a sentence recorded by the ESP32 in step 4.make run: press the button, ask "Quelle heure est-il ?", release; the board answers. Recognised questions: time, date, hello, thanks, your name; anything else is repeated back as understood.
The log shows the time spent in each stage: transcription 1.0 s, synthese 0.1 s, reponse prete en 1.1 s. make run-echo keeps the step 5 echo mode,
without models.
Step 5 — the server makes the ESP32 speak, over MQTT
make fw-upload SERIAL_PORT=/dev/ttyUSB0(step 5 firmware).make tone-mqtt: 3 s of pure tone sent over MQTT; the board report is printed (chunks 60/60, underruns 0).make play-mqtt OUT=server/recordings/<file>.wav: any WAV, converted to 16 kHz mono, peak at −3 dBFS.make run-echo: the server plays each sentence back on the board that said it.
Step 4 — voice to the server, over MQTT
make install(server dependencies: paho-mqtt, PyYAML, numpy...).make run: the server connects to the broker and waits for sessions.make fw-upload SERIAL_PORT=/dev/ttyUSB0, then press the button, speak, release.- The server writes
server/recordings/esp32-01_<date>_<session>.wavand logs a report:complete, received / announced chunks, peak and RMS. make play OUT=server/recordings/<file>.wavormake inspect OUT=....
If the broker is unreachable when the button is pressed, the session goes over
the serial link: make record remains available for troubleshooting.
Step 3 — Wi-Fi and MQTT
- Broker: see docs/MOSQUITTO.md (Debian service or container).
cp firmware/include/secrets.h.example firmware/include/secrets.h, then set the Wi-Fi (2.4 GHz) and the broker IP address.make mqtt-watch MQTT_HOST=<broker ip>in a first terminal.make fw-upload SERIAL_PORT=/dev/ttyUSB0in a second one.
Expected in the first terminal: voice/esp32-01/state {"status":"online",...}
and voice/esp32-01/event {"event":"boot",...}. After cutting the ESP32 power:
voice/esp32-01/state {"status":"offline"} within 15 s.
Step 2 — test the audio output
- Wire the MAX98357A amplifier and the speaker (§2), including C1 and C2.
make fw-upload SERIAL_PORT=/dev/ttyUSB0- At boot, a 440 Hz beep plays: it tests the output on its own.
make tone-esp: 3 s of pure tone streamed from the PC in real time. Any dropout is heard as a click.make play-esp OUT=recordings/test.wav: the step 1 recording played back on the ESP32 speaker.
Expected: clean tone, no clicks or crackle; the ESP32 report shows 0 refuses
(rejected) and 0 sous-alimentations (underruns).
Step 1 — test the audio input
- Wire microphone, button and LED as in §2.
make fw-upload SERIAL_PORT=/dev/ttyUSB0make record OUT=recordings/test.wav- Press the button, say a sentence, release it.
- Listen:
make play OUT=recordings/test.wav - With no audio output:
make inspect OUT=recordings/test.wavdraws the signal envelope in ASCII and reports peak, RMS and DC offset.
Expected: the sentence is intelligible and not clipped; the script reports a peak
between 10 % and 90 % of full scale. Above 99 %, lower MIC_GAIN in
firmware/include/config.h; below 2 %, raise it.
7. Repository layout
voice-assistant/
├── firmware/ ESP32 firmware (PlatformIO, Arduino)
│ ├── include/ config.h, secrets.h.example
│ └── src/ main.cpp + one .h/.cpp pair per class
├── server/ Python server (one module per class)
├── tools/ standalone test tools
├── mosquitto/ voice.conf
├── Makefile every system operation
├── CONTEXT.md progress log and frozen technical decisions
└── README.md / README-EN.md
8. License
MIT — see LICENSE.