What are the use cases for local AI models that actually require running on macOS? Surely a commodity x86 system is more appropriate?
Is there even the software support for LLMs on macOS?
Actually yes there is...
I'm still learning about this myself, but, from what I understand the M series of chips that Apple has come out with, with it having a CPU, GPU, and shared unified memory....it makes them uniquely capable of running local models on them...decently large models depending on how much you fork over for RAM. These M chips also have a special end unit for "intelligence processing" I think they call it.
The M5 chips just coming out look to be very good at this and it is speculated the M5 Ultra will be a high performance work horse.
Apple may have missed the mark for running AI, but the appear to have hit a home run on the hardware aspect of it.
I've seen demos on YouTube of someone hooking up like 4-5 Mac Studios that were maxed out M3 ultras I think and they were running extremely LARGE LLMs locally and getting cloud level numbers on them.
Of course these were like $10K each boxes.....but the level of model they were running would have cost my MANY more times trying to match them with NVIDIA GPU cards.....
i believe there are OSX friendly tools like ollama that make downloading, and running LLMs quite easy....and of course there's the latest sensation...OpenClaw, that folks are buying up Mac Minis for....to have multiple agents running using models of your. Choice (commercial clound or local) of models and giving them persistent memory, and ability to do a lot of things for you...depending on how comfortable you are with giving said agents long leashes and capabilities....
Do look a bit on YouTube on these topics....it's actually quite interesting.
These M chips are already giving the home user the capability to use models almost as large and on the cutting edge as the big companies.....more than enough for most users.
Right now, there's nothing x86 that can really match them...at least not for the money.