Skip to content
Devendra Jangiddevendra.pro
Voice AI & edge hardwareLocal networkPrototype to working system

Personal engineering build

Voice control across several rooms at once

Several small devices, each with a microphone and speaker, all streaming audio to one server. The server works out who said what, matches it against known commands even when the words are not exact, and triggers the right response on the right device.

Multi-device
Several endpoints on one server
Real time
Response begins as speech ends
Tolerant
Fuzzy matching handles imperfect speech
Low cost
Runs on single-board computers
The situation

What was broken

  • Streaming live audio from several devices at once without the server falling behind
  • Deciding when someone has actually started and stopped speaking
  • People never say a command exactly the same way twice
  • Responses have to play on the device that was spoken to, not all of them
Faster-WhisperWebRTC VADFastAPIWebSocketsRaspberry PiPython
The work

What was actually done

  1. 01

    Stream, don't upload

    Devices hold an open websocket to the server and stream audio continuously, so the response starts as soon as someone stops speaking rather than after a file finishes uploading.

  2. 02

    Detect speech on the device

    Voice activity detection runs on the device itself, so silence is never sent over the network. This alone removed most of the load from the server.

  3. 03

    Match commands approximately

    Recognised text is matched against known commands with fuzzy string matching rather than exact comparison, so near-misses and accents still work instead of failing silently.

  4. 04

    Route responses back per device

    Each device has an identity and its own mapping of what to play, so the server can answer on exactly the one that was spoken to.

CallWhatsAppEnquire