I finally sat down and set this up last night. The Qwen 3.6 35b is very impressive and following this vid will get you running at workable speeds. I haven’t bumped it up to the 256k context window yet, but even with the 128 I’m able to have it work on projects I’d rather not be putting up on the cloud.
Ah, gotcha.
I’m not too familiar with Apple’s place in all this beyond some folks (organically?) figuring out that loading up some mac minis with a bunch of ram would let them run the larger models. Has Apple themselves been making moves marketing themselves for local AI?
All Apple computers come with Unified Memory meaning the GPU and CPU shares the same memory pool, have very high memory bandwidth, and they provide their own CUDA/ROCm/Vulkan equivalent SDK called Metal.
So any Mac can decently run any AI model that fits into its system RAM. For the same price as a high end AI card you can get a whole computer that can run larger models but at slower speeds.
Oh, I see! I thought that was just an efficiency thing in the mac mini. Got it.