The stuff Hacker News actually recommends.

GPUs and CPUs · NVIDIA RTX Pro

NVIDIA RTX Pro 6000

14owners with an opinion
80%of verdicts positive
8/2positive/negative since 2024
0happy after 3+ years

Praised for performance and value; criticized for reliability.

What owners wrote

Neutral
The 'by the pallet' buys are for small local projects, not data center size projects (of which we have more than a couple as well), as you probably already knew. This isn't some silly dick measuring contest. The point was: they're buying them from the retail channels in relatively high volume because there is a massive shortage of GPUs in general, but that does not seem to stop parties with smaller projects who do not qualify for buying directly from NV (assuming NV even has stock to sell). 5090's now sell for the same amount (or more) that you could buy an RTX 6000 for a few months ago, and RTX 6000's are ~19K and up. I found a stash somewhere, bought two, then two more and when I was ready to buy the remainder they were all gone. Serves me right for being too careful.
Likes it
At GTC2025, NVIDIA more or less pressed a Founders Edition DGX Spark on me (and many other conference attendees hoping for a future of local models). Full disclosure: I'm the guy letting out a screm when Jensen announced the Spark at the conference. But like everyone else, I was disappointed when we found out what the memory throughput was. I didn't use it for a year, but then Qwen3.8-Flash-Next came out, which I think is the first model where the Spark's architecture actually mattered, and the paperweight became useful. I even bought a second one in preparation for Qwen 4-Flash, and so far, I'm happy with it. The RTX PRO 6000 was a present to myself a year ago. Same story: the drivers didn't work at first, so for a while I used it to heat my flat. Now they work, and it trains everything here without a cloud bill. So yes, renting is the saner choice, and nothing here needs local hardware. The final training runs were about 24 GPU-hours on one RTX PRO 6000: 2 hours for the 0.8B base, 3.5 for the 2B, and 18 for all nine adapters. Verda lists that card at $2/hour on demand and $1 on spot, so roughly $50, or $25 if you're patient. The real cost was everything around it: days of synthetic data generation on the Sparks, the runs that didn't make the cut, and a 27B benchmark that kept a Mac busy and warm. If you just want an adapter for your own task, one rented GPU for an afternoon is plenty. The adapter kit is in the repo.
Neutral
Yes that was the second part of my comment. I've been benchmarking many things AI or not on those boards and while one can sometimes get to 600W I haven't seen more than 10% gain for those last 250W. If someone has a workload that gets more from those watts I'd be interested. For now on anything I run (full-capacity, continuous, batched...) capping at 350W seems better in bang-for-bucks when including energy costs (from the GPU + added cooling).
Likes it
I have a RTX PRO 6000 96GB when the pricing was way better than now i also have a RTX 5090 too. What I noticed is that (1) the great local models are optimized run inference (diffusion & LLMs) well on 32GB VRAM <= GPU's because that that's what the target has ... (2) The quality of local models (esp. in diffusion) is increasing faster than the need for more VRAM - additional reason for the value of these FAST GPUs to increase! (3) RTX PRO 6000 96GB is really great for fine tunes (ai-toolkit) :) but doesn't outperform my RTX 5090 with inference by anything significant on the good local models. I have never run an AI job on a Mac, i also have doubts about performance and compatibilities - since the reviews almost never compare directly.
Likes it
My card (RTX Pro 6000) is always doing something all the time from my queue; like some synthetic dataset generation up next. I still actively use runpods and openrouter for scaled stuff, I was spending a bit and then did the maths, and invested in it. The maths to me was basically equivalent to prepaying for 242 days of runpod pricing for the same GPU; and I reckon I'd be able to get 6+ years of use out of this card with 96GB. Plus there's the resell value -- it's actually appreciated by ~50% since I bought it. Plus I do really enjoy that it's 100% local. I wouldn't feel comfortable giving my agents this much information if inference wasn't 100% local. I wouldn't get another one, I wouldn't have as much value, but one is definitely paying off for me on the financial side.
Doesn’t like it
Finally, gonna be a lot of fun to finally play around with, lets see how it compares to Python for ML ultimately :) The build process is quite heavy, pinning all my cores at 100% on a Threadripper 9970X for ~15 minutes which isn't so common. Curious to know what setups they've tested Mojo on so far, as I understood it to be mainly for ML/AI stuff, so commonly pretty beefy workstation, yet I'm getting warnings that it's not recognizing my GPU (RTX Pro 6000). Seems easy enough to work around by editing the gpu_mapping in MODULE.bazel so I guess isn't a huge issue. Just strange to see a really CPU-heavy build, then not recognizing a somewhat common "prosumer" GPU (common in the ecosystem and expected user audience at least).
Likes it
DiffusionGemma goes one step further even, and does this denoising over multiple "canvases" which lets it do reasoning and separate out a "final reply" canvas, looks something like this: https://gist.github.com/embedding-shapes/f4cb46bad704b6d0168... Diffusion text models for me is the more interesting type of LLMs for local usage, as it really makes good use of single GPUs for single responses, rather than auto-regressive ones, and is a lot faster! Probably the fastest model I've been able to run so far, ending up doing ~670 tok/s (depending on the type of text) on a Pro 6000
Likes it
The biggest gain you'll get is faster memory, provided you have enough capacity to load all the weight into vram. The DGX sparks and Apple silicon memory bandwidth (and also memory access latency) drag down the decode speed quite a bit. I have two GPU rigs both with 2x RTX Pro 6000, can get ~250 tk/s decode with deepseek-v4-flash in native mixed precision. For context, in antirez's dwarfstar project he only gets ~20-40 tk/s on the same model @ 2bpw. The latter is for sure usable if it's your only option, but it's really hard for me to personally go back to speeds like that when I've experienced the former. (Also worth noting dwarfstar only has experimental support for dspark spec dec, when that lands it will definitely give a big boost at higher acceptance rates)
Recommends it
A pair of RTX6000 cards will give you a good performance boost due to tensor parallelism, though. I haven't tried the newest predictive quants but I see about 35 tps when running the 8-bit Qwen 3.6 27B model on one board and about 50 tps on two. Probably could come close to 100 tps on an optimized setup with the latest GGUFs. Also, the 4-bit quants of MiniMax 2.7 will run at 100 tps or so with two cards, which is pretty decent. It doesn't go any faster at all with 4 GPUs from what I've seen, so if you don't actively need 384 GB of VRAM, 2x RTX6000 is a good place to be.
Doesn’t like it
RTX 6000 is some-what obviously my fastest card but my biggest problem with the RT 6000 is the immense heat. The GPU itself is almost 200F and the exhaust from the fans itself is over 150F. I'm worried that my hard drives are going to fail. I was told that the GDDR7 is even hotter than the GPU which is surprising to me. After my last run, I'm going to wait for the new case I ordered to come in and cannibalize my kid's PC that we built beginning of this year to form an entirely separate computer. And then figure out better ways to deal with the heat, especially with summer coming up. I'll have to play around with undervolting and running vents directly outside my house to see if that helps.
Likes it
If I could find a RTX Pro 6000 for $5K I'd definitively grab it, I'm running RedHatAI/Qwen3.6-35B-A3B-NVFP4 on one (I had to pay closer to $10K for it though) with 260K context and it's a blast! ds4 by antirez also works well, even IQ2XXS seems to work relatively well but Qwen3.6-35B-A3B-NVFP4 is both faster and higher quality responses (at least for coding and translations which I use them mostly for).
Likes it
I use Qwen 3.5 122B on an RTX PRO 6000 with open code, and very pleased. I don't feel a need for using a closed model any more. The result after answering questions in Plan mode is almost always what I want, with very few occasional bugs. It does a lot of effort to see how the code I am working on is written now while extending it in the same style. If they release a Qwen 3.6 that also makes good use of the card, may move to it.
Doesn’t like it
> You are forgetting about how with the multiple high quality open weights models, we are gonna quickly/(already have?) reached the point where using completely local models will make sense. I'm not, since I'm a heavy user of local models myself, and even with the beast of a card I work with locally daily (RTX Pro 6000), the LLMs you can run locally are basically toy models compared to the hosted ones. I think, if you haven't already, you need to give it a try yourself to see the difference. I didn't mention or address it, as it's basically irrelevant because of the context. And besides that, how affordable how GPUs today in the developing world? Electricity costs? How to deal with thermals? Frequent black outs? And so on, many variables you seemingly haven't considered yet. Best way of making the difference between hosted models and local modals is to run your own private benchmarks against both of them and compare. Been doing this for years, and still local models are nowhere near the hosted ones, sadly. I'm eager for the day to come, but it will still take a while.
Doesn’t like it
I am working on a platform for GPU rental and have recently encountered an extremely annoying issue. On all machines with RTX 5090 and RTX PRO 6000 GPUs, the cards occasionally become completely unresponsive — usually after a few days of VM usage or at seemingly random times during startup/shutdown. Once it happens, the GPU can’t be reassigned. GPU is in a limbo state and doesn't respond to FLR. The only way out is a complete node reboot, which is undesirable, as it will stop VMs that are already running on the node. H100s, B200s, and older RTX 4090s are solid, but these newer RTX cards are a menace. I understand that RTX cards are not designed for virtualization, and NVIDIA likely doesn't care; however, those cards are very well-suited for a variety of applications, and it would be nice to make virtualization work. Is there a way to recover the GPU from this state without a complete node reboot? We've put a $ 1,000 bounty on it if anyone is interested in helping. We also have an open position for an engineer who can help with problems like this.
Likes itHasn’t owned it
This is a pretty good article but it ignores one very important factor: the amount of VRAM you get per slot is often just as important as the amount of compute that accompanies it. An RTX 5090 for instance, in the largest configuration, is 32GB. An RTX 6000 PRO is 96 GB. That means that in the same number of slots you'll be able to put 3x as much VRAM and that's a massive advantage. 4 RTX 4090's are way easier to administer than 12 5090s, the power consumption will be lower and due to less bus activity you will probably get (much) better overall performance. One of these days someone will figure out a reliable way to transplant a 5090 GPU to a 64 or 96 GB remanufactured card and then NVidia will have to find an answer if the volumes are high enough. I don't know the ratio of 5090's sold into the gaming channel versus RTX 6000 PRO and other datacenter products but I would not be surprised if the 5090's still held the edge.
Likes itHasn’t owned it
These GPUs are extremely inflated in price because Nvidia effectively has a monopoly on hardware that is used to train models. Apple Silicon tends to have good inference software available but as soon as you want to train even a YOLO model bits and pieces fall back to software implementations. Try to train an LLM and it'll get even worse. The M3 Ultra's GPU performance is around a 4070 Ti. The M5 Ultra more like a 5080. They're both amazing deals compared to Nvidia for local inference because of their massive pool of high bandwidth memory. But a single RTX PRO 6000 should be 2 or 3x the compute of an M5 Ultra.
Doesn’t like itHasn’t owned it
There are - see the RTX Pro 6000, which has 96 GB. There are a few problems though, primarily, a GPU with lots of VRAM and very high bandwidth is inherently very expensive (on top of which there is also the CUDA premium); AI use cases are better served by SoCs with lower (but still high) bandwidth and more RAM.
Recommends itHasn’t owned it
> This is a professional tool for AI R&D. Not really, not it isn't, because it's deliberately gimped and doesn't support the same feature-set as the datacenter GPUs[1]. So as a professional development box to e.g. write CUDA kernels before you burn valuable B200 time it's completely useless. You're much better off getting an RTX 6000 or two, which is also gimped, but at least is much faster. [1] -- https://github.com/NVIDIA/dgx-spark-playbooks/issues/22
Recommends itHasn’t owned it
> $10k gets you a Mac Studio with 512GB of RAM, which definitely can run GLM-4.7 with normal, production-grade levels of quantization (in contrast to the extreme quantization that some people talk about). Please do give that a try and report back the prefill and decode speed. Unfortunately, I think again that what I wrote earlier will apply: > In practice, it'll be incredible slow and you'll quickly regret spending that much money on it I'd rather place that 10K on a RTX Pro 6000 if I was choosing between them.