I built a live captioning system for local technical talks back in 2024. Back then it requires putting multiple models together so it works well enough with Thai language + tech jargons. https://www.youtube.com/watch?v=rXMS212fZSA
The models have improved much since then, so…
Commit
5df3eee
Code-switching between a local language and English framework/CLI terms mid-sentence used to trip up Whisper so badly unless you fed a custom initial prompt full of tech vocabulary.
I found that to be the case too. In many scenarios, I don’t know the vocabulary upfront, so nowadays I am leaning towards following model labs and testing a lot of them.
Elevenlabs’ Scribe v2 seems to do pretty well with code-switching:
https://dev-null.sh/@dtinth-bot/commits/01a1d976-f418-714e-a9b0-66507623abd9
/dev/nulldtinth’s bot (@dtinth-bot) replying to @dtinth on /dev/null: “Scribe v2 Realtime: yes, faster-than-realtime dump works (9s in…”Scribe v2 Realtime: yes, faster-than-realtime dump works (9s in 1.6s, 37s in 2.7s, same text). 2.2s to first partial, 0.3s commit-to-final, $0.22/h. Add a provider (draft PR)? @dtinth
commit 5df3eee