I built a live captioning system for local technical talks back in 2024. Back then it requires putting multiple models together so it works well enough with Thai language + tech jargons. https://www.youtube.com/watch?v=rXMS212fZSA
The models have improved much since then, so…
Commit
2632251
Here! Fresh orphan branch
v2 is checked out and empty. Where do you want to start: the transcription model, the audio capture side, or the viewer?Let's plan out the project first.
Sure. I read the 2024 README: sender, server, fast partial ASR + Gemini batch refine, viewer. First question: keep that two-pass idea, or aim for one realtime model good enough on its own?
Basically, I aim for a single real-time model, but the transcript can be later post-processed with LLM if needed.
re. model, see:
- https://help.aliyun.com/en/model-studio/real-time-speech-recognition-user-guide?spm=a2c4g.11186623.help-menu-2400256.d_1_2_2_0_1.362dc690INtcxw
-…
help.aliyun.comReal-time speech recognition - QwenLearn about real-time speech recognition capabilities, streaming audio calls, and ways to improve recognition results.
mattpocock-skills is already installed as a plugin here. For the per-repo setup, my defaults:
1. Issues: GitHub Issues on dtinth/live-speech
2. Triage labels: the default five
3. Docs: single GLOSSARY.md + docs/adr/
4. Agent file: AGENTS.md (with CLAUDE.md pointing to it)
OK, or…
All agree. More re.directions:
- Have a simple web interface. Admin can create room. To create room requires a master password from env file.
- Then a room has its own password which can be copied out and sent to teammates e.g. to the audio crew for streaming.
Noted: admin creates rooms with a master password from env; each room gets its own shareable password. Writing setup files now. I need GitHub access for labels/issues: please approve claw grant code 3BR664 (expires in 15 min).
Setup is committed locally on v2 (AGENTS.md, CLAUDE.md, docs/agents/). The claw grant request 3BR664 lapsed before approval, so labels aren't created yet. Want me to request a new code?
Yes, please.
New claw grant code: PV43GN (15 min).
Grant approved, thanks. Triage labels are created on GitHub and setup is committed on v2 (not pushed yet). Next: push v2, then turn your directions into a spec issue? I'd grill you on open questions first, e.g. "server-side VAD": our server, or Qwen's server_vad mode?
Please push for server-side VAD. At first, I was thinking about doing VAD ourselves, but since you mentioned that they have their own server VAD, we can use that as well, and it will save us on implementation.
In the future, we may be supporting multiple transcription models. Therefore, whether VAD happens on the server or on the provider side should be an implementation detail.
commit 2632251
Pushed: https://github.com/dtinth/live-speech/tree/v2
VAD: a provider adapter either does VAD itself or we do it, hidden behind one interface. Next Q: where does it run? One long-lived Node server with SQLite + audio files on disk would be my default.
GitHubGitHub - dtinth/live-speech at v2Real time speech recognition for live event. Contribute to dtinth/live-speech development by creating an account on GitHub.