AMD Radeon 9070

I have a server in the closet running AMD Ryzen 5 3600, 9070 (non-xt) and 64GB of DDR4-2400. Running Qwen3.8-flash-next on Strata and pushing ~60 t/s decode and reading ~1.5k t/s prefill. Works well for the tasks I'm doing with it. For external access I'm exposing the OpenAI compatible endpoint via a CloudFlare tunnel and using it on a hosted Viberglass instance. The cost was around 850 for the GPU and memory a few months ago, other hardware has been in the closet running my server for around 5 or so years.

4k works great when combined with FSR. On a 9070 GPU the frametime cost of FSR 4 is around 2 msec so you can just render the game at 1080P, upscale to 4k. I usually play like this, it gives a very sharp image at 120-180W with a barely audible fan.

> Note that the performance of the RTX 5050 is completely irrelevant if it doesn't work. I don’t really understand the point of this comment. Shouldn’t we operate under the assumption that it will work? Is there something particular problematic about the 5050? Also, AMD GPU’s are still very affordable and totally viable. I have a 9070 and I love it.

Nvidia makes a fine GPU. The problem with Nvidia on linux is the drivers. You're beholden to Team Green for driver updates, and when they decide not to support a GPU anymore, that's it. Now, linux does have the nouveau driver, but that doesn't support all the hardware or much 3d at all, and never will. I particularly got fed up with Nvidia on linux playing War Thunder - I had a regular crash that Gaijin and Nvidia each blamed on each other, and I never did get it fixed. Nvidia driver updates can also leave you stuck with no desktop environment on occasion and while fixable, it's a pain in the rear. However, when the drivers are right, Nvidia performance is second to none. AMD has drivers built right into the kernel, and as long as you have whichever nonfree firmware repos your distro supports (I use Devuan, a Debian derivative), AMD cards 'just work'. If using xorg, install xserver-xorg-video-amdgpu for modern cards, and xserver-xorg-video-radeon for older cards. I'm currently playing on a Radeon 9070 (non-XT) on a 1440p monitor with plenty of performance. I know that it also works on wayland, but I have no experience there.

Yeah to be clear I’d never say it’s “easy” and ready for mass adoption. But I also had 0 issues getting bazzite going on my PC I built with an AMD 9800x3d/9070 working out the gate. I played expedition 33 the day I finished building! Kind of remarkable given the GPU was only a month or two old. What’s striking was that I never had to open a terminal window or install a single driver. Some of the distros are near-turnkey at this point.

I am not saying that this was the reason I bought it, but I recently purchased a Radeon 9070 and I was surprised how little power this card uses in idle. I was seeing figures between 4W~10W on Windows (sadly slightly more on Linux). In general this generation of Radeon GPUs seems highly efficient. Radeon 9070 is a beast of a GPU.

So, you have DDR4? Pick any B550 with a VRM heatsink for $90, a 5600 for $190, and you are back in business. Get a 5070 or 9070 before GDDR supply is completely gone if you want an actual upgrade. As for GPU price, that Vega 64 was $500 in 2018. Even a midrange GPU hasn't been $300 since the Radeon RX 480 launched a decade ago.

Not quite $500, but at $650, the 9070 is an absolute monster that outperforms Nvidia's equivalent cards in everything but ray tracing (which you can only turn on with full DLSS framegen and get a blobby mess anyways) AMD is truly making excellent cards, and with a bit of luck UDNA is even better. But they're in the same situation as Nvidia: they could sell 200 GPUs, ship drivers, maintain them, deal with returns and make $100k... Or just sell a single MI300X to a trusted partner that won't make any waves and still make $100k. Wafer availability unfortunately rules all, and as it stands, we're lucky neither of them have abandoned their gaming segments for massively profitable AI things.

It used to be that for locally running GenAI, VRAM per dollar was king, so used NVidia RTX 3090 cards were the undisputed darlings of DYI LLM with 24GB for 600€-800€ or so. Sticking two of these in one PC isn't too difficult despite them using 350W each. Then Apple introduced Macs with 128 GB and more unified memory at 800GB/s and the ability to load models as large as 70GB (70b FP8) or even larger ones. The M1 Ultra was unable to take full advantage of the excellent RAM speed, but with the M2 and the M3, performance is improving. Just be prepared to spend 5000€ or more for a M3 Ultra. Another alternative would be a EPYC 9005 system with 12x DDR5-6000 RAM for 576GB/s of memory bandwidth with the LLM (preferably MoE) running on the CPU instead of a GPU. However today, with the latest, surprisingly good reasoning models like QwQ-32B using up thousands or tens of thousands of tokens in their replies, performance is getting more important than previously and these systems (Macs and even RTX 3090s) might fall out of favor, because waiting for a finished reply will take several minutes or even tens of minutes. Nvidia Ampere and Apple silicon (AFAIK) are also missing FP4 support in hardware, which doesn't help. For the same reason AMD Halo Strix with a mere 273GB/s of RAM bandwidth and perhaps also NVidia Project Digits (also speculated to offer similar RAM bandwidth) might just be too slow for reasoning models with more than 50GB or so of active parameters. On the other hand, if th…