Problem
Huddle voice is hardwired to two bundled on-device models:
- STT: sherpa-onnx Parakeet TDT-CTC 110M (
desktop/src-tauri/src/huddle/stt.rs) — English-only
- TTS: pocket-tts (
desktop/src-tauri/src/huddle/pocket.rs)
For English-speaking teams that's a great zero-config default. But for non-English and code-switching teams the STT output is garbled (#2478 reports this for Korean), which poisons the whole agent-in-huddle experience: the agent "hears" garbage, replies to garbage, and the reply is spoken by a small TTS voice. Users experience all of this as "voice quality is bad" when the real issue is the fixed models.
We run a self-hosted relay for a Malaysian team (English/Malay/Mandarin code-switching in a single sentence is normal here). We already operate a local ASR service on our LAN (Qwen3-ASR behind a simple HTTP endpoint) that handles code-switching well — but there is no way to point huddles at it: no config, no env var, no endpoint override. The models are compiled into the pipeline.
Proposal
Make the huddle STT (and ideally TTS) backend pluggable, keeping the bundled models as the zero-config default:
- A per-community or per-app setting like
huddle.stt.endpoint accepting an OpenAI-compatible POST /v1/audio/transcriptions URL (the de-facto standard most self-hosted ASR servers — whisper.cpp servers, speaches, vLLM audio, custom — already expose).
- The existing worker already produces 16 kHz mono PCM at utterance boundaries (VAD-segmented), so the integration point is narrow: ship the utterance to the endpoint instead of sherpa-onnx when configured, with the local model as fallback on error.
- Same shape for TTS if feasible:
huddle.tts.endpoint accepting POST /v1/audio/speech.
Why this fits Buzz
Self-hosting your relay but not your voice models is an odd asymmetry — the community that cares enough to run its own infrastructure is exactly the community whose language needs won't be covered by a single bundled English model. A pluggable endpoint keeps the default experience untouched, adds no new bundled dependencies, and turns #2478 from a model-quality bug into a solved configuration.
Happy to test against our self-hosted relay + LAN ASR and report back if this lands behind a flag.
Problem
Huddle voice is hardwired to two bundled on-device models:
desktop/src-tauri/src/huddle/stt.rs) — English-onlydesktop/src-tauri/src/huddle/pocket.rs)For English-speaking teams that's a great zero-config default. But for non-English and code-switching teams the STT output is garbled (#2478 reports this for Korean), which poisons the whole agent-in-huddle experience: the agent "hears" garbage, replies to garbage, and the reply is spoken by a small TTS voice. Users experience all of this as "voice quality is bad" when the real issue is the fixed models.
We run a self-hosted relay for a Malaysian team (English/Malay/Mandarin code-switching in a single sentence is normal here). We already operate a local ASR service on our LAN (Qwen3-ASR behind a simple HTTP endpoint) that handles code-switching well — but there is no way to point huddles at it: no config, no env var, no endpoint override. The models are compiled into the pipeline.
Proposal
Make the huddle STT (and ideally TTS) backend pluggable, keeping the bundled models as the zero-config default:
huddle.stt.endpointaccepting an OpenAI-compatiblePOST /v1/audio/transcriptionsURL (the de-facto standard most self-hosted ASR servers — whisper.cpp servers, speaches, vLLM audio, custom — already expose).huddle.tts.endpointacceptingPOST /v1/audio/speech.Why this fits Buzz
Self-hosting your relay but not your voice models is an odd asymmetry — the community that cares enough to run its own infrastructure is exactly the community whose language needs won't be covered by a single bundled English model. A pluggable endpoint keeps the default experience untouched, adds no new bundled dependencies, and turns #2478 from a model-quality bug into a solved configuration.
Happy to test against our self-hosted relay + LAN ASR and report back if this lands behind a flag.