Apple Mac M2

How do these compare? I'm very familiar with Augment and particularly enjoy its fast completions. I mostly don't use its agent, but rather Claude Code in the terminal, because the agent seems superior. But Augment is not the most stable. I've had lots of serious problems with it. The newest problem that's pushing me over the edge is that it's recently have been causing the IDE to shoot up to use all cores (it's rare to see an app use 1,000% CPU in the macOS Activity Monitor, but it did it!) when it needs to recompute indexes, which is the only thing that has ever made my M2 Mac run its fan. It's not very reliable generally (e.g. autocompletions don't always appear), so I'd be interested in trying alternatives.

I think it's highly unlikely that the M4 mac is actually limited to 60Hz for all external displays. I'm running a 1440 ultrawide at 100Hz on an M2 right now. Instead this is likely the maximum px*Hz configuration supported, and smaller or fewer monitors are supported at higher Hz.

Primarily concerned about the memory bandwidth for training. Though I think I've been able to max out my M2 when using the MacBook's integrated memory with MLX, so maybe that won't be an issue.

I like my M2 Mac, it's great hardware-wise. I thought hard about Linux laptops and decided for the Mac. In a lot of ways it feels like linux on a slick piece of hardware with good commercial support. Coming from linux I miss the freedom though. That maybe sounds kinda stilted or melodramatic or something but I don't mean anything ideological. I just mean it annoyed me, for example, to find out that a stock market app was installed and I couldn't just get rid of it easily. I looked into it but it involved a whole series of things that I decided just wasn't worth it. But it's still there. Is it a big deal? No, but then again why should I not be able to remove a stocks app? I also miss KDE which has been my favorite GUI of everything I've used. Basically I just miss being able to take things off I don't want there, and put on things I do. Seems like something simple to me. I don't regret buying my Mac and would do it again. I sort of feel like a good combination is Mac on laptop and linux on a desktop/workstation/server. But if for some reason Asahi went away I'm not sure I'd buy another Mac. I haven't used Asahi linux, but it's like an insurance or canary or test case or something. It's this thing in the back of my mind that doesn't really affect me day to day but is always there. When I was thinking about linux laptops, I was leaning toward an HP developer laptop or Framework, or Starlabs I think? I mention it only because I'm not sure if your XPS experiences would transfer al…

Windows are not bad if you have control over them. But in corp env, you don't and the force updates, AV, FW, extra AV scans and other monitoring shit so even with 32 GB of RAM, fast SSD and latest i7 PC would start to act like it's Win98. That's why I switch to Mac M2, zero corporate shit and total control so it works great.

No, running it locally means running it on my laptop. My Mac M2 is quite capable of running stable diffusion XL models and 30M parameter. LLMs under llama.cpp. What I don’t like is the trend towards the way to do that being to open up network listeners with no authentication on them.

It used to be that for locally running GenAI, VRAM per dollar was king, so used NVidia RTX 3090 cards were the undisputed darlings of DYI LLM with 24GB for 600€-800€ or so. Sticking two of these in one PC isn't too difficult despite them using 350W each. Then Apple introduced Macs with 128 GB and more unified memory at 800GB/s and the ability to load models as large as 70GB (70b FP8) or even larger ones. The M1 Ultra was unable to take full advantage of the excellent RAM speed, but with the M2 and the M3, performance is improving. Just be prepared to spend 5000€ or more for a M3 Ultra. Another alternative would be a EPYC 9005 system with 12x DDR5-6000 RAM for 576GB/s of memory bandwidth with the LLM (preferably MoE) running on the CPU instead of a GPU. However today, with the latest, surprisingly good reasoning models like QwQ-32B using up thousands or tens of thousands of tokens in their replies, performance is getting more important than previously and these systems (Macs and even RTX 3090s) might fall out of favor, because waiting for a finished reply will take several minutes or even tens of minutes. Nvidia Ampere and Apple silicon (AFAIK) are also missing FP4 support in hardware, which doesn't help. For the same reason AMD Halo Strix with a mere 273GB/s of RAM bandwidth and perhaps also NVidia Project Digits (also speculated to offer similar RAM bandwidth) might just be too slow for reasoning models with more than 50GB or so of active parameters. On the other hand, if th…