Today's docket at home: testing out Project Maya to see how feasible it is to run GLM 5.3 Flash on a 5080 with 16gb vRAM and 128 GB dRAM.
So far, it's not horrible, just long ttft. About 25-30 tok/s decode, 1200 prefill; compared to 75-90 and 2600 for Qwen 3.8 Flash Next.
Commit
43e92b0
Amazing! I did test Qwen 3.8 Flash Next on my 4090 but not GLM yet! I definitely should try!
Definitely worth it! I think ultimately it's disappointing compared to our GOAT Qwen 3.8 FN but still, in theory it's a teensy bit smarter so it could be good in a pinch. It's also nice when the voice of FN gets on my nerves, that mfr has AI voice for sure.
Main issue right now…
Yeah I get it. I am a bit shorter on RAM I "only" have 64GB, but 24GB vRAM, so I might have faster prefill but shorter context... Will see. Anyway I adapted my workflow to short context.
I wonder if the RAM shortage means we'll finally see game devs optimizing before release 😆
*devs in general. I caught a Chrome tab eating 2.1GB of RAM, the website being... An hyperscaler console page 🤣
commit 43e92b0