Apple MacBook Pro M3 Max

On the other side, as someone doing a lot of work in the GenAI space, I'm simultaneously amazed that I can run Flux [dev] on my laptop and use local LLMs for a variety of tasks, while also wishing that I had more RAM and more processing power, despite having a top of the line M3 max MBP. But it is wild that two years ago running any sort of useful genAI stuff on a MBP was more-or-less a theoretical curiosity, and already today you can easily run models that would have exceeded SotA 2 years ago. Somewhat ironically, I got into the "AI" space a complete skeptic, but thinking it would be fun to play with nonetheless. After 2 years of daily work with this models I'm starting to be increasingly convinced they are going to become increasingly disruptive. No AGI, but it will certainly reduce a lot of labor and enable things that we're really feasible before. Best of all, it's clear a lot of this work will be doable from a laptop!

64 m3 max. 64 is probably too much for about 90% of what I do, but will likely become 'just right' in the next year or so. I was coming from 16g m1 mini, and i got a refurb m3 max mbp for $3k even, which was at the time a decent deal. 1.5 years in and it'll see me through for a while longer, barring some physical destruction.

I’ve been running LLMs on my laptop (M3 Max 64GB) for a year now and I think they are ready, especially with how good mid sized models are getting. I’m pretty sure unified memory and energy efficient GPUs will be more than just a thing on Apple laptops in the next few years.

Macbook (base) yes, but my MBP (M3 Max) is driving 4 screens for me without issue.

Not the GP, but I run this model as daily driver too. It runs great on a Macbook Pro 64GB (M3 Max). Token generation speed can be about 100 tokens/sec with multi-token prediction, although it depends on the context. Worst case speed is around 50 tokens/sec. The weaker point is prompt prefill, which starts at 1,400 tokens/sec but decreases significantly at high contexts. That said, for agentic scenarios, if you're using a harness that doesn't needlessly bust the cache, it doesn't feel slow. I really hope they release a Qwen 3.8 35B, although the lack of a mention seems ominous.

If you're using a MoE model, then why do you care about the larger RAM offered by these devices? That's the main problem with low bandwidth devices: they limit the effective ram you can make use. I do (and have historically done) quite a work with both local LLMs and local diffusion models. I have an M3 Max MBP at 400 GB/s and also a desktop with a RTX 4090 with 1,008 GB/s While the M3 Max MBP can serve up MoE reasonably fast (~60 token/sec)the RTX 4090 is an entirely different experience (~170 token/sec). I also do a fair bit of experimentation and am currently running a custom decoder that requires expensive look-ahead, but I'm still able to get a usable 25 token/s on the RTX. The raison d'etre for the DGX spark is not practical home inference, but rather offering the same fundamental architecture as data center cards for a affordable CUDA prototyping. If you want to build software to run on H100s, you probably can't justify buying (and running) a single card. The DGX spark solves this by having the same fundamental setup as what those cards have. That makes these non-NVIDIA DGX-like devices confusing to me. The entire benefit of the DGX series is the NVIDIA architecture itself. Anyone interested in home LLMs should decide whether a Mac or a dedicated GPU is the more sensible path based on their budget and other computer use. Each has their own benefits.

I have been running qwen 3.6 35b a3b with opencode on my macbook pro 16" with m3 max and 64gb ram, and it's been great for local planning and coding. To be honest I have been on and off wishing I had future proofed with the 128gb after seeing how powerful 64gb is. On the other hand, I also haven't run up against a wall with a model that is just slightly larger than qwen.

yeah i totally understand and I am thoroughly impressed it works. And the electricity cost isn't that bad since it was on a ARM laptop (MacBook M3 Max) and not a beefy workstation with a GPU. I just let the agent do its work while watching the World Cup.

Depends on the workload. The M3 Ultra is no slouch. It sits at the top with best processors on both single-core and multi-core x64 processors. Meanwhile my mini PC with a Ryzen 7 8845HS processor, which is nowhere near an M3 Ultra, feels much snappier using it as a day to day desktop in both Linux and Windows. I think this speaks more to the sluggishness of the macOS experience rather than hardware performance. But, then I start doing something data/gpu/local LLM intensive and my M3 Max shines.

This is pure speculation on my part, but the way I think this will play out is something like this: - current CapEx will make the production side increase capacity - advances in TPUs, NPUs, open weight and quantization will keep going at a rapid pace - when the spending slows/stops, hardware prices will drop, hard - most AI workloads will move to the edge (except frontier models) because the hardware is cheaper than a subscription (and at some point there could be a crash like 2008) For example, most of my AI use lately has been running Qwen3.6-35B-A3B-UD-Q8_K_XL on a 64GB MacBook Pro with an M3 Max. It runs at ~57 tokens/s and it's mostly fine. I do use the frontier models a bit, but only when the task is too complex for the local model. Basic crap, like analyzing an existing codebase and bouncing ideas, making small changes, the local model is enough.

The problem with this model is that DeepSeek v4 Flash runs quite well quantized to 2 bit (see https://github.com/antirez/llama.cpp-deepseek-v4-flash ), at 30 t/s generation and 400 t/s prefill in a M3 Ultra (and not too much slower on a 128GB MacBook Pro M3 Max). It works as a good coding agent with opencode/pi, tool calling is very reliable, and so forth. All this at a speed that a 120B dense model can never achieve. So it has to compete not just with models that fit 4-bit quantized the same size, but with an 86GB GGUF file of DeepSeek v4 Flash, and it is not very easy to win in practical terms for local inference. Note: I have more uncommitted speed improvements in my tree that I'll push soon, the current tree could be a little bit slower but not much, still super usable. I don't understand one thing about Mistral, which I'm a fan being in Europe: they opened the open weights MoE show with Mixtral. Why are they now releasing dense models of significant sizes? In this way you don't compete in any credible space, nor local inference, nor remote inference since the model is far from SOTA and not cheap to serve. So why they are training such dense big models? Dense models have a place in the few tens of billion parameters, as Qwen 3.6 27B shows, but if you go 5 times that, it is no longer a fit, unless you are crushing with capabilities anything requiring the same VRAM, which is not the case.

I bought a 3k M3 Max mbp just a couple months away from the Tahoe and liquid glass announcement which I am a little miffed about, but it's still an awesome machine I enjoy using while it's on Sequoia. I am really hoping macOS 27 will be this decade's Snow Leopard