NVIDIA GeForce RTX 5080

This is good timing. My setup at home involves a two servers with 5080s each and an Rtx spark laptop with 128gb. I plan I have qwen flash next on the latter and the 27b variant on the former. I would’ve preferred to have purchased a dgx spark but everything is just ridiculously expensive now. My own setup would be 3x the cost to buy again. I’m mainly trying to replicate Gemini flash and muse agent for my family of 4 without breaking the bank (already had a 5080 for gaming and the other for image gen). It is unclear if this memdoor works for agentic tasks, e.g.: “check daily if dune 3 tickets are for sale and if they are buy tickets for 4 where all 4 seats are next to each other in the same row and as center as possible.” There is a huge cliff once you go past 128gb. Basically you need either Rtx 6000 pros (20k+) or a cluster of dgx sparks (15k+)

I confirm this as my experience as well. Parakeet did not perform well for me (English with heavy accent). Whisper large is much better. But once we get to model of this size look at qwen3-asr (1.7b parameters). I'm quite happy with how it performs (both transcription quality and speed on rtx5080). By the way the model is also available on openrouter and very/very cheap. Performance is reasonable ( 10s)

I got a RTX5080 as primary and a RTX3050 (8gb) as secondary. I always max out on context size so that means i need to split the model between both gpus + offload some into system ram when using LM studio. I'm guessing that is what accounts for most of the slowdown when i use that. With this recipe i don't have to worry about it. It's pretty fast, and since its all running in the 5080 with max context it's stable (though i got to look out not to load anything big on the same gpu)

Isn't the same amongst same architectures? For instance, when I created script to train my model it worked fine on RTX 5080, but when I rented H100 to hopefully wait less for completion of training, the training would collapse just in a few epochs, suggesting they compute things differently (RTX 5080 would run thousands of epochs without collapsing. The same script and the same data).

Thanks! I'm seeing a 10/90 split between CPU/GPU with gemma4:26b, so I guess there's at least something to win there by adding the other GPU. And perhaps something to win by connecting the monitor to the iGPU instead to free up VRAM, from what I gather. Just in case someone should be interested in how a consumer PC setup like this performs, still using only 1x RTX 5080 + 64GB system RAM and Intel Ultra 270K-Plus; I tested Qwen3.6:35b-a3b now (using ollama and default settings) and I'm getting around ~86 t/s. The lowest I've seen so far is 70 t/s. The CPU/GPU split with 35b is 39/61% (with 4K 165 fps monitor connected to 5080, so there's probably some room for optimization here by moving it to the iGPU). Best thing is that this setup is basically dead silent (it could, hypothetically speaking, be running in my bedroom just fine, and I'm a light sleeper).

I was seriously irritated with Windows 11 for the past few months. If my PC went to sleep, it would wake up and freeze at the logon screen. The constant updates that would reset some preferences - and worse, reinstall CoPilot and OneDrive - pushed me over. I went and installed Bazzite on the PC I built early this year with a Nvidia RTX 5080, somewhat skeptical that it would work well despite my extremely positive experience with the Steam Deck. The result is outstanding.

Modern GPUs like RTX 5080 are much faster for the applications that are limited by computational capabilities, mainly because they have more execution units, whose clock frequencies have also increased. I suppose that most games are limited by computation, so they are indeed much faster on modern GPUs. However, there are applications that are limited by memory throughput, not by computation, including AI inference and many scientific/technical computing applications. For such applications, old GPUs with higher memory throughput are still faster. This is why I am still using an old Radeon VII and a couple of other ancient AMD GPUs with high memory throughput. Last year I have bought an Intel GPU, which is still slower than my old GPUs, but it at least had very good performance per dollar, competitive with that of the old GPUs, because it was very cheap, while the current AMD and especially NVIDIA GPUs have poor performance per dollar.