Nvidia’s new AI “Superchip” combines CPU and GPU to train monster AI systems
- Thread starter JournalBot
- Start date
If they didn't announce this board during a thunderstorm, lightning arcing overhead, the board rising in a cloud of steam on a slab from deep within the stage, the hunchbacked product manager of Hybrid Generative Processing pulling the switch upon command of the VP in charge to reveal it to the world, ah-hahahahahaha! then they missed a golden opportunity there.
Upvote
80
(82
/
-2)
Our institution will be getting an HPC system based on these. The $$$ question is whether one can do useful physics simulations taking advantage of all the GPU power, rather than whether AI can write publishable research papers for us.
Upvote
72
(72
/
0)
Upvote
25
(31
/
-6)
It seems like just yesterday that the computing world was excited about putting a "teraflop in a box". Now a bunch of these can make an "exaflop in a box" (a large box, but still). Pretty amazing.
Really curious to see how the Grace CPU performs compared to other platforms like Amazon's Graviton3 and Apple's M2. I know those serve vastly different purposes, but still an interesting comparison if individual cores and efficiency can be compared.
Really curious to see how the Grace CPU performs compared to other platforms like Amazon's Graviton3 and Apple's M2. I know those serve vastly different purposes, but still an interesting comparison if individual cores and efficiency can be compared.
Upvote
54
(54
/
0)
Can they just sell off their gaming GPU division to someone who actually wants to make GPUs?
The 4060 Ti is the kind of product you put out when you no longer give a shit.
The 4060 Ti is the kind of product you put out when you no longer give a shit.
Upvote
-15
(21
/
-36)
As a shareholder this makes me happy. As a gamer this makes me sad, because it almost certainly means less focus on producing GPUs that are good for gamers (which means both good performance AND good price).
Upvote
-17
(14
/
-31)
Cerebras has done something similar: their Wafer-Scale Engine 2 chip (more like a personal pizza) has 850k cores. Pittsburgh Supercomputing Center has two installed.
ETA I should've read to the end where Cerebras is mentioned.
ETA I should've read to the end where Cerebras is mentioned.
Last edited:
Upvote
10
(11
/
-1)
Pity they couldn't make a Leading-Edge Generative processor to pair with the Advanced RISC Machine chip. Then they could have said building it cost them an....
Upvote
79
(81
/
-2)
Question - These monster GPUs are only needed while training, yes? Once the model is built, you use a vector database yes? Explain it like I'm 5 please.
Upvote
20
(23
/
-3)
Hasn't PC gaming been dying for like 30+ years? Call me when it's dead.
Upvote
71
(74
/
-3)
normally butters
Ars Praefectus
AMD was in a slump at the wrong time, allowing CUDA to become the x86 of high-performance parallel computing, but now there's no DoD second-source policy requiring vendors like NVIDIA to license their instruction set architecture to other vendors, like Intel was forced to do with x86 back in the day.
So not only are they playing catch-up with ROCm, and neglecting the "grassroots" market by excluding ROCm support from most consumer-grade devices, but they're also pushing uphill on cross-platform projects like OpenCL with messy roadmaps and underwhelming adoption.
AMD is in a much better place today than they were in mid-2010s, but the widespread adoption of CUDA makes it really difficult for AMD to dig themselves out of this hole in GPU computing.
Upvote
47
(47
/
0)
I agree. Software is Nvidia's real mote when it comes to AI. AMD and other companies could potentially play catch up and make hardware that was just as fast or faster, but then they ALSO need to come up with comparable software and Nvidia has a huge lead there. It's not insurmountable but there's a good reason Nvidia saw such a huge jump in their stock price based on AI. They really are that far ahead as a total solution.
Upvote
13
(14
/
-1)
And now I'm wondering when the robot from the future arrives to protect the woman who saved humanity from the AI wars until a liquid metal robot arrived to kill her and her kid and prevent them from saving the world.
If an Nvidia lab blows up in the near future, we'll know what happened, and that Skynet probably won't be built after all...
If an Nvidia lab blows up in the near future, we'll know what happened, and that Skynet probably won't be built after all...
Upvote
2
(7
/
-5)
There is training (which is what it sounds like) and then there's 'inference,' which is the process of using the resulting model to turn inputs into outputs. Both are accelerated by hardware that's dedicated to matrix multiplication (tensor cores).
Doing inference once requires a trivial amount of compute power when compared to training a new model.
But if you're going to be doing inference millions of times per day (e.g., you're serving ChatGPT), then it quickly ends up needing more compute power than training the model in the first place. So any improvements to efficiency and/or performance are welcome.
Upvote
57
(57
/
0)
Correction: It takes a lot of power to pretend to be human.
For comparison, a human brain uses up only about 20W.
For comparison, a human brain uses up only about 20W.
Upvote
19
(20
/
-1)
How is Nvidia s**tting on anybody?
Given the same amount of money, can't you buy a better graphics card now than you could a couple years ago? Even if you factor out the supply problems, scalping, etc., that were happening a couple years ago?
Offering better products for similar/less money seems like the opposite of s**tting on the customer base.
Upvote
6
(18
/
-12)
Who cares what the cards are named or what their halo card costs? Seriously, what practical difference does that make to anything?
Unless you want the social status of owning a card that's in a particular spot in the product lineup?
I'm not sure that's been true since ~2000. What card could you buy for $200, and when, that would give you better-than-console performance?
Really? Name one card that they're offering now that is slower and more expensive than a card they were offering a few years ago.
Last edited:
Upvote
20
(25
/
-5)
Those are two separate issues. To the first, when considering MSRP, no you can't, actually. To the second, sure, but that's largely because supply chain constraints are mostly in the past and because crypto has crashed.
While I don't agree with any claims that PC gaming is outright dying, it is somewhat stagnant in the GPU space right now. The 4XXX series of cards from Nvidia are not steps up in generational performance when looking at the 4070 and 4060 series of cards, especially if you take out Nvidia's DLSS upscaling (which can be nearly useless on those tiers of cards, depending on the game). Rasterization performance of the 4060 Ti is nearly identical to the performance of the 3060 Ti, same for the 4060, etc. and the low amount of VRAM is hamstringing the cards at anything higher than 1080p. At a time when resolutions are jumping and widescreens are common, Nvidia seems content to deliver consumer mid-range GPUs that are incapable of 1440p gaming in many cases.
There are multiple sources citing that the interest in these GPUs is nearly nonexistent. Here's one:
Literally no one wants to buy Nvidia’s RTX 4060 Ti | Digital Trends
A damning new report says there is zero consumer interest in Nvidia’s RTX 4060 Ti GPU, with only one person bothering to show up to its launch in Japan.
www.digitaltrends.com
Upvote
4
(10
/
-6)
Sure, but replacement batteries are unbelievably hard to come by.
Upvote
43
(44
/
-1)
OrangeCream
Ars Legatus Legionis
It kind of tells you where their bread is buttered though.
GPUs have to compete with AI compute hardware for chips. 4 additional A100s bumps up the price $69k, and 8 A100s cost $101k more than 8 A6000 and $71k more than 8 6000(Ada) chips.
Upvote
8
(8
/
0)
Uhh, really?
So I'm looking at the card you mentioned, the 4060 Ti, which retails for $299.
It seems to be somewhat faster than a 3060 Ti (and, interestingly, uses MUCH less power), and the 3060 Ti had an MSRP of $399.
So it seems to me like Nvidia is offering a faster and better product for less money than it did a couple years ago. How are you getting slower and more expensive?
Upvote
-5
(7
/
-12)
The 4060 (no Ti) retails for $299. The Ti model retails for $399. And I can buy a 3060Ti from Amazon right now for $325. I should have said "street pricing" instead of MSRP, though. My bad.
The "somewhat faster" is the issue there. If you dig deeper, it's more like "only faster in certain cases". Sure, it uses less power, but I don't think most people with a gaming desktop care.
Upvote
-7
(5
/
-12)
From what I'm reading it seems like interfacing can be massively parallelized.
Upvote
0
(0
/
0)
So you're comparing the MSRP of a product that was just released to the street price of an outgoing product that, if it hasn't already been discontinued, will be soon. That hardly seems like a fair comparison, does it?
Dunno. I'm looking at a couple reviews of the 4060 Ti and I'm not seeing any instances where it's slower than a 3060 Ti. Can you point me to what you're looking at?
Certainly, in some cases, it's only marginally faster. A few frames per second. But it's still faster. Faster isn't slower.
Upvote
-2
(6
/
-8)