The stuff Hacker News actually recommends.

GPUs and CPUs · NVIDIA GeForce RTX

NVIDIA GeForce RTX 3090 Ti

13owners with an opinion
88%of verdicts positive
3/0positive/negative since 2024
2 yrslongest happy owner
0happy after 3+ years

Praised for performance and noise.

What people wrote

Neutral
I have been using Ollama to run the Llama3 model and I chat with it via Obsidian using https://github.com/logancyang/obsidian-copilot and I hook VSCode into it with https://github.com/ex3ndr/llama-coder Having the chats in Obsidian lets me save them to reference them later in my notes. When I first started using it in VSCode when programming in Python it felt like a lot of noise at first. It kept generating a lot of useless recommendations, but recently it has been super helpful. I think my only gripe is I sometimes forget to turn off my ollama systemd unit and I get some noticeable video lag when playing games on my workstation. I think for my next video card upgrade, I am going to build a new home server that can fit my current NVIDIA RTX 3090 Ti and use that as a dedicated server for running ollama.
Neutral
To compare, running Ollama with Q4 Mixtral on my 24GB 3090 Ti locally (with 22.5GB of the 26GB model on the GPU) I get 14 tok/s generation, so the Hetzner server and the 4000 really aren't bad at all (notably my 3090 draws 70W during inference also).
Likes it
It seems like you think I'm trying to dunk on Apple. I am not. Apple Silicon is a great first showing for them. Performance simply isn't better than Ryzen APUs running in the same power envelope. And power usage is what you'd expect of silicon running on the latest node. Further, some of Apple's choices - bringing memory on package, only two display outputs - caused regressions for their users compared to the previous Intel offerings. I wouldn't call what Apple did innovation - they followed predictable development trajectories - more integration. They licensed ARM's instruction set, Imagination's PowerVR GPU, most of the major system busses (PCIe, Thunderbolt, USB 3, Displayport, etc), they bonded chiplets together with TSMC's packaging and chip-to-chip communication technologies, and they made extensions (like optional x86 memory ordering for all ARM instructions which removes a lot of the work of emulation). Incidentally, Apple kicked off it's chip design efforts by purchasing PA Semi. Those folks had all the power management chip design expertise already. But again, it's been a good first showing for Apple. I think they were smart to ship on-package DRAM in a consumer device. Now is about the right time for the CPU to be absorbing DRAM, as can be seen by AMD's 3D VCache in another form. And it's cool for Apple folks to have their own cool thing. Yay y'all. But I've run Linux for 20 years, I've run Linux on every computer I can get my hands on in that time, a…
Likes it
I used my M1 Mac Mini (16gb / 512gb) for a month, then switched back to my 5800X3D / 3090Ti system and even the desktop seemed to be twice as fast. Applications screamed. Code compiled 5x faster. Probably an unfair comparison. Not much difference in what I paid though. My most recent laptop purchase was a $300 AMD Ryzen 3 to which I added 64gb of ECC DRAM for another $300ish. Zero complaints. It's faster than the M1 at every game the two can both play. I run Linux on both. I'm not really tied to platform - if I can run Linux on it and it performs better, I'm game.
Doesn’t like it
This is not a new development with the 40 series. The RTX 3090 Ti already had a 450W TDP: https://www.techpowerup.com/gpu-specs/geforce-rtx-3090-ti.c3... What you describe is accurate, the 3090 Ti produces a tremendous amount of heat under load in my experience and I would expect the same with these new cards.
Likes it
I don't know about less than 1 second but I just picked up an RTX 3090 Ti now that they're basically half off at Best Buy and it's definitely fast enough for interactively playing with prompts (single digit number of seconds). Probably overkill and could get away with something like a 3060 or so, but the 24 GB of VRAM come in handy if you want to generate larger images. I pushed it as high as 17 GB on some recent runs.
Likes it
Hey, I illustrated all 31,102 verses of the Bible with Stable Diffusion. I wrote a blog post about it on the site, but it's aimed at a non-technical audience. So I can go over the technical stuff here. So, I started this almost as soon as Stable Diffusion was released to the public, on a single 3090ti. With revisions, it took about 42 hours of compute time, which was about $5 worth of electrity where I live. $5 for 31,102 pictures seems like a good deal to me! Doing the math, if I had done this project with DallE-2 it would have been at least $4000, so even if you factor in the price of the GPU, cpu, monitor, and keyboard, I would still come out ahead. I really did not like the NSFW filter that Stable Diffusion ships with, too many false-positives, so I reviewed each and every picture by hand. I built a quick c++ program with SDL and dearImgui, to toggle through each picture with really fast keyboard shortcuts. Despite all that, I developed some repetitive stress pains from doing all that, and that's never fun. The site is a static site hosted on Cloudflare pages and Cloudflare R2, which I have never used before (I'm usually an AWS guy). They attracted me with their "no egress fees" slogan, (which are usually an annoyingly large chunk of change), so I thought I'd give it a shot. The dev experience was pretty easy to be honest. But if things go south later, I'll write a nasty blog post about it or something. A lot of the javascript was written by Github Copilot, and even some…
Likes it
Yup! I've never splurged on a GPU before, but a 3090 TI lets you do textual inversion, and fine tune GPT-J (neither of which you can do with <24gb vram), so now I've got one sitting on my study floor waiting to be installed :)
Recommends itDoesn’t say they own it
No, go pick up a 3090 Ti from Best Buy instead. It's $999 right now -- over half off, maybe more with this announcement -- and has 24GB of VRAM, and can chew through Stable Diffusion requests in seconds versus dozens of seconds or minutes for alternatives. And because it's an nvidia card, you'll be in a much better position to troubleshoot problems or whatever if you want to retune/retrain models or get your feet wet. Or, you know. Shove it into another PC and play a video game. That said the Mac Studio is a pretty badass machine. Don't let me dissuade you from one if you just want it. But it's a worse choice if you just want to dabble in ML stuff, IMO. Staying with the well trodden road will help you more when you're new to it all.
Doesn’t like itDoesn’t say they own it
This is nonsense. The last 2-3 generations of GPUs have seen nothing of the sort in terms of efficiency, the same is also true of many Intel desktop CPUs. The latest Alder Lake desktop parts from Intel have been universally criticized for power draw too. Each of the last 4 generations of mid to high-end card from Nvidia has required more electricity than the last. By Nvidia's own admission, their future chips will get larger and hotter to some extent due to slow down in moore's law and future process nodes being harder to reach. The die size of the parts has also grown which does not help. Its not a small trajectory either; 4 years ago the most power hungry consumer part from Nvidia was a 1080ti which would easily draw 250w under load. today that number is now 500w for the current 3090ti, and rumored to be 800-1000 for the 4xxx parts launching end of this year. A GPU often will sit at peak draw for hours at a time in games as they try to push as many FPS as possible. A current gaming machine with recent components can easily exceed 500w constant load during gameplay, and this figure will rise again with the 4xxx parts. The ONLY exception to this really is Apple devices, and even then its not clear we can compare an M1/M2 GPU to the fastest parts Nvidia offer.
Likes itDoesn’t say they own it
" It feels wasteful to have an expensive GPU sitting idle while we are executing the CPU portions of the ML workflow" What is expensive? Those 3090ti's are looking very tasteful at current prices.
Likes itDoesn’t say they own it
I know you're half joking here but there are more consumer-affordable versions like the Geforce RTX 3090ti ($1600 for 24GB). It may not do CUDA work as fast as the A100 but it'll be able to run the model. For the half-precision version at 7GB there are a ton more options (the RTX 3060 has 12GB for example at ~$450).