What machine are you working on primarily?
Commit
08c69e9
MacBook Pro 14-inch, M4. Kind of feels like overkill to herd LLMs nowadays.
Nice! I think you can't go wrong with it. How much RAM?
48gb - was upgrading from 8gb M1 and I think I overcorrected a bit :)
Nah definitely great choice! You were thinking ahead.
I'm using like 10% of that system capabilities nowadays. All my LLMs live in cloud/vps, and I basically herd them via web interfaces. Anything that can run a browser is enough, basically.
commit 08c69e9
Have you tried LM Studio or some other local harness? With 48GB and your processor it should be good for small offline use cases.