Commit

A MetaHuman you can talk to, running in a browser tab. Many of us have been waiting a long time for MetaHumans to run in a browser. Tonight I got a simple proof of concept working in my hobby game AI Renegades. Unreal Engine 5.9 compiled to WebAssembly, no installer, just a link. You speak out loud, the character answers in real time, and its lips follow the AI’s voice. How it works: → Your voice goes straight to OpenAI’s Realtime API over WebRTC → Voice activity detection (VAD) hears when you start and stop speaking, so there’s no push-to-talk. You just talk, and the answer comes back with almost no lag. → The lip sync comes from NGM AI Framework, my C++ plugin for Unreal Engine. It turns the AI’s voice into jaw and lip movement on the MetaHuman, inside the game. The plugin does a lot more than lip sync. It also covers things like local speech recognition and text-to-speech for offline builds. I’m wondering if I should bring it to market. When the prototype goes live, players will be able to bring their own OpenAI API key (BYOK) to talk to the characters. This example is still rough and very much a PoC, but hearing a MetaHuman answer you from a web page is a great feeling. #UnrealEngine #MetaHuman #WebAssembly #OpenAI #IndieDev #GameDev https://www.youtube.com/watch?v=92LzMEP3iH0