Apple Mac Studio M3 Ultra

Energy efficiency is in another league with the Mac Studio for local workloads. I can run agents using deepseek v4 flash or Qwen 3.8 on my m3 ultra and it will be lukewarm and the fan will eventually start blowing softly.

Two Mac Studio M3 Ultra 512GB and 1 USB cable can run all those models - maybe about $30,000 in hardware - and based on my benchmarks, those Mac Studios were twice as fast as the A100s on Deepseek v4 Flash, which has a quantization but not really a lossy one.

Back in January, I was debating whether to buy an M3 Ultra Mac Studio or waiting for the M5 version, which many believed would be announced two months later. The 192GB M3 Ultra was on sale at the local Microcenter for $200 below what Apple's own site advertised. Since I knew the RAM shortage would significantly increase the price of the M5 Studio when (or if) it finally did come out, I decided to buy the M3. Time has shown that was the right decision.

The problem with this model is that DeepSeek v4 Flash runs quite well quantized to 2 bit (see https://github.com/antirez/llama.cpp-deepseek-v4-flash ), at 30 t/s generation and 400 t/s prefill in a M3 Ultra (and not too much slower on a 128GB MacBook Pro M3 Max). It works as a good coding agent with opencode/pi, tool calling is very reliable, and so forth. All this at a speed that a 120B dense model can never achieve. So it has to compete not just with models that fit 4-bit quantized the same size, but with an 86GB GGUF file of DeepSeek v4 Flash, and it is not very easy to win in practical terms for local inference. Note: I have more uncommitted speed improvements in my tree that I'll push soon, the current tree could be a little bit slower but not much, still super usable. I don't understand one thing about Mistral, which I'm a fan being in Europe: they opened the open weights MoE show with Mixtral. Why are they now releasing dense models of significant sizes? In this way you don't compete in any credible space, nor local inference, nor remote inference since the model is far from SOTA and not cheap to serve. So why they are training such dense big models? Dense models have a place in the few tens of billion parameters, as Qwen 3.6 27B shows, but if you go 5 times that, it is no longer a fit, unless you are crushing with capabilities anything requiring the same VRAM, which is not the case.

I have a Mac Studio m3 ultra on my desk, and a user account on a HPC full of NVIDIA GH200. I use both and the Mac has its purpose. It can notably run some of the best open weight models with little power and without triggering its fan.

I am! I moved from a shoebox Linux workstation with 32MB of RAM and a 12GB RTX 3060 to a 256GB M3 Ultra, mainly for unified memory. I've only had it a couple of months, but so far it's proving its worth in the quality of LLM output, even quantized. I generally run Qwen3-vl at 235b, at a Q4_K_M quantization level so that it fits, and it leaves me plenty of RAM for workstation tasks while delivering tokens at around 30tok/s The smaller Qwen3 models (like qwen3-coder) I use in tandem, of course they run much faster and I tend to run them at higher quants up to Q8 for quality purposes. The gigantic RAM's biggest boon, I've found, is letting me run the models with full context allocated, which lets me hand them larger and more complicated things than I could before. This alone makes the money I spent worth it, IMO. I did manage to get glm-4.7 (a 358b model) running at a Q3 quantization level; it's delivery is adequate quality-wise, although it delivers at 15tok/s, though I did have to cut down to only 128k context to leave me enough room for the desktop. If you get something this big, it's a powerhouse, but not nearly as much of a powerhouse as a dedicated nVidia GPU rig. The point is to be able to run them _adequately_, not at production speeds, to get your work done. I found price/performance/energy usage to be compelling at this level and I am very satisfied.

I have a maxed out M3 Ultra. It runs quantized large open Chinese models pretty well. It's slow-ish, but since I don't use them very frequently, most of the time is waiting to the model to load from disk to RAM. There are benchmarks on token generation speed out there for some of the large models. You can probably guess the speed for models you're interested in by comparing the sizes (mostly look at the active params). Currently the main issue for M1-M4 is the prompt "preprocessing" speed. In practical terms, if you have a very long prompt, it's going to take a much long time to process it. IIRC it's due to lack of efficient matrix multiplication operations in the hardware, which I hear is rectified in the M5 architecture. So if you need to process long prompts, don't count on the Mac Studio, at least not with large models. So in short, if your prompts are relatively short (eg. a couple thousand tokens at most), you need/want a large model, you don't need too much scale/speed, and you need to run inference locally, then Macs are a reasonable option. For me personally, I got my M3 Ultra somewhat due to geopolitical issues. I'm barred from accessing some of the SOTA models from the US due to where I live, and sometimes the Chinese models are not conveniently accessible either. With the hardware, they can pry DeepSeek R1, Kimi-K2, etc. from my cold dead hands lol.

Tahoe running fine on my M3 ultra. The only issue on the studio and the laptop was the initial sluggishness that I ended up attributing to the first spotlight indexing. Nothing new then.

I had been fairly anti apple my entire life. Believing they were massively over priced, and lacked support for many of the things I did personally. In my younger years I was pretty big on graphic design, still big on video editing. People used to always say "Mac is better for design". I would ask, where did you get this from? Adobe runs on both platforms and updates often come out for Windows before they are ported to Apple (Adobe updates). And my Windows machines had powerful graphics cards, unlike macs. However. A couple years ago I built myself an insanely expensive computer build with the best parts I've ever put into a machine. It died 2 years later. I decided I just couldn't go through that anymore. I'm done building computers. I'm done with the little problems they always get. I'm done fixing my machine more than using it. Since the $500 Mac mini, Apple has started looking interesting. Finally I bought the Mac Studio M3 Ultra. So far so good. I wish I could play more games, but I could be looking into things like Crossover. There are a few old bespoke programs for old versions of Windows I also wish I could run once in a while but nothing too major. And best of all, Apple Care +. I'm not dealing with a broken machine again. I'll be buying warranties for it until I'm done with it. Bought 2 years for I think about $200, and my understanding is I should be able to renew that service/warranty at the end of the two years. After that, we'll see.

It’s functional if your goal is to run models that won’t fit into RAM on a single machine. Functional. the slow interconnects (yes, even at 40Gbps thunderbolt) severely limit both TtFT and tokens/second. I tried it extensively for a few days, and ended up getting a single M3 Ultra Mac Studio, and am loving life.

Yep, I love Apple, follow them closely, own a Mac Studio with an M3 Ultra and a MacBook Pro with an M4 Max, and it's still confusing. :) I mean, surely a Mac Studio with an M4 Max must be the best, right? It's an entire CPU generation ahead and it's maximum! Of course, it's not... the M3 Ultra is the best. Naming things is hard.

$10,000 divided by five divided by 12 is only $166.66 a month without interest over 5 years a Mac M2 Ultra or Mac M3 Ultra are easily usable computers over five years, so are most of the medium to high-end laptops that Apple sells for one person or locally in a small company. It’s not necessarily out of reach or unusable over the course of time, the thing I keep hearing over and over is how usable many people find Mac’s very useable over the course of time, obviously software support plays a big part of that. On top of that, AI models are getting more useful despite their smaller size. The future is a personal computer future, not a mainframe computer one. Yes, larger computers are useful, but most people will not be using that larger computers, which will be relegated to universities and larger companies.

It depends on your use case. There's a lot of hype around machines like the DGX spark (I'm assuming this is the type of device you're referring to) because they look awesome, and are priced reasonably well. However all of these have notoriously low memory bandwidth despite the high ram. These devices, especially the DGX line, are fantastic if you are interested in low-level CUDA programming. The DGX spark can be used to prototype CUDA code/libraries for GPUs that most of us couldn't think about affording. If you want to learn how to program for datacenter level GPUs then these are the best way to get that at home. Sure your code will run very slow compared to the real thing, but you can take that code and, theoretically, run it on the real thing. For anything else though, I feel there are better options. If you're interested in pure inference I'm pretty partial to Apple devices. The M4 Max gets you 546 GB/s, the M5 MAX 614 GB/s, and the M3 ultra (you'd have to buy used at this point) 819 GB/s. Plus you have a very useful computer even if you realize you don't want a full time home inference server. Additionally these devices require very low power (if you're running high end consumer GPUs you do have to think about what your energy costs are per hour and how warm you like your room). If you're interested inference and training , or already have a pretty beefy desktop PC, or simply demand the most token/s you can get, then GPUs are the way to go. The downside is they're…

> M3 Ultra Mac Studio: $5,299 (up from $3,999) I knew i should have bought a maxed one when i had the chance...

I'd buy that Mac studio m3

A M3 ultra mac Studio can run models that do not fit in similarly priced computers with multiple Nvidia GPUs. And it will use a lot less electricity while still having good enough performance. Except the pre-filing perfs that are quite poor on the M3.