Watched @nunomaduro live where he was building with voice dictation apps and it pushed me to restart an old project: Sushhh [ open to better names :P ] a fully local dictation app for Mac.
What I learned building it:
We hear pitch on a roughly logarithmic scale. 100->200 Hz sounds like a big jump. 5000->5100 Hz barely sounds different.
Speech models like Whisper use the same idea. They don't take raw audio. They turn it into a
log-mel spectrogram, which has lots of detail in the low frequencies where speech lives and much less up high.
So the model "hears" more like we do.
Still building. More soon.
A good read if someone's more curious : https://medium.com/analytics-vidhya/understanding-the-mel-spectrogram-fca2afa2ce53commit f61a474