Rebuilding my local voice assistant pipeline (blog.platypush.tech)

64 points by todsacerdoti 6 hours ago

12 comments:

by mmmooo 3 hours ago

older amazon echo dots and shows really make great hardware for this, combined with homeassistant, and can be had for very little cost. e.g. shameless plug: https://github.com/ygelfand/echolocal

by nl 2 hours ago

I just commented in another thread that I built this: https://github.com/nlothian/earwing

Beam forming microphone array, custom UI, custom case. Maybe costs more than a second hand echo dot though (hardware is probably around $30).

by neumann 2 hours ago

I got an echo show rooted and ROMd that I am loving - put it in the kids room as a music player in kiosk mode, but have been now wanting one for myself with wakewords. This looks like the ticket.

by mmmooo an hour ago

the gen 1 or 2 show 8s make great devices for this kind of stuff, 5's are a little small for me. the lenovo android things (google homes) are also really nice if you want a slightly bigger screen, but not as cheap second hand.

by mewse-hn 3 hours ago

Interesting write up, I've been vibe coding an android voice assistant using a similar pipeline without a wakeword - VAD > ASR > LLM > TTS

I've found parakeet to be much, much better than whisper and I made a 4 bit quant of parakeet 110M for my crappier android test devices, it would probably fit a raspberry pi:

https://huggingface.co/mewse/parakeet-tdt_ctc-110m-onnx-int4

by GaggiX 2 hours ago

There is also parakeet-redux, a 1.58bits quant of parakeet-tdt-0.6b-v3 that runs as well as the original model: https://huggingface.co/moondream/parakeet-redux

The model is actually a distillation of a different checkpoint called parakeet-ultra, same size and architecture as parakeet-tdt-0.6b-v3 but better, that's why the 1.58bit quant works so well.

by mewse-hn 2 hours ago

Oh crazy I hadn't seen this yet, I might be able to unify my 0.6B vs 110M selector, thanks!

by fragilerock 4 hours ago

Interesting read! I have my own local voice assistant and I implemented exactly the same question mark trick just the other day.

The overall pipeline is quite similar to my previous setup, but I found the STT -> LLM -> TTS pipeline just infuriatingly slow. TTS and STT with whisper/piper worked fine and reasonably fast for me locally, but the real bottleneck was always the fact that they're not running concurrently with the LLM call. I ended up forking an open source openai realtime API ESP32 plugin and patching it to my liking. It's a bit more expensive in terms of API cost but I only use it sparingly and it's so much nicer to use now.

by bityard 3 hours ago

Is this complimentary to, or an alternative to Home Assistant?

by malfist 3 hours ago

Home assistant doesn't really provide much out of the box, its mostly a modular/pluggable ecosystem and one of the things you can install are various voice assistants

by dofm 3 hours ago

Just what I want to read β€” favourited to come back to!

by stavros 5 hours ago

This is really useful, thanks! The other day I bought a small screen and didn't know what to do with it so I made an assistant that talks to my bot:

https://m.youtube.com/shorts/3FseVyAG5uM

I used microwakeword too, and training my own wakeword was pretty simple (basically just recording a few tens of samples). Highly recommended.

I also wanted to be able to issue some voice commands for lights and other things in my home with low latency, so I added a short-circuit plugin that runs right after recognition. If the recognized command matches one of the regexes, it's immediately executed, which has a latency of under a second. Otherwise, it gets sent to my assistant, which usually will tweak the regex (or add a new one) to improve the whole system as it fails.

Data from: Hacker News, provided by Hacker News (unofficial) API