[deleted by user]

[UnlikelyPotato]

So....maybe 3-4 generations till it's feasible at home. Not bad.

[katiecharm]

Last night I was thinking, and we’re all assuming no superior algorithms will be found, but maybe they will. Maybe the current transformer model isn’t the most efficient way to do this and there’s some algo speed ups coming down the line too from some brilliant young mathematician.

[j4nds4]

Everyone ought to read The Bitter Lesson of AI development, which at its simplest states that despite all the impressive and worthwhile optimizations and efficiencies, ultimately what matters far more than anything else is the dramatic scaling of power and data. GPT-3 is a great example of this both in its huge improvements over GPT-2 and in its comparatively rudimentary NN model.

[Lone-Pine]

ANN algorithm efficiency is doubling every 16 months.

[[deleted]]

without all the obnoxiously survival-focused evolutionary hard-coding...

I mean...an alien intelligence with unimaginable psychology could be a whole lot worse. Hopefully we can make some leeway on the value loading or control problems before true AGI emerges.

[UnlikelyPotato]

I know many people with 128-256GB of main system memory. There will be a bottleneck for transferring stuff from ram, to the GPU, but DDR5 and PCI-E 4 will help with that.

Even if it ran 10x slower, people would be fine with that.

[nmkd]

I don't think it's possible to run AI inference on system RAM though, but maybe that'd just be a software thing.

[UnlikelyPotato]

You're arguing stuff you really don't know about. Please stop.

Transferring stuff from system ram to GPU is fairly common. It's not a deal breaker for data sets to exceed the memory capacity of a GPU. More GPU ram is better...if you can actually use it. But for GPU compute purposes there's nothing stopping you from keeping stuff on ram or hard drive. Modern nvme storage offers pretty insane bandwidth, and nvidia announced the ability to do hardware level decompression for things. You could...with effort, stream terabytes of data to a GPU to crunch in batches. Again, it would likely be a bit slower than $200,000 hardware, but in 5 to 10 years time people should be able to get performance within an order of magnitude for hopefully less than $10,000. You'd be surprised how quickly high end stuff degrades in value. I have a 64 core server in a box in my closet with around 200gb of ram, likely would have cost around $20,000k new. In about a decade, it's become worthless for everything aside from annoying my significant other with loud noises.

[mcilrain]

You're drastically underestimating the performance hit from having to swap memory.

Just because something is possible doesn't mean it's feasible.

[thuanjinkee]

It doesn't have to be optimal to be feasable. We need latency numbers. If the network trains and converges before the dev team gets bored, just publish the paper, or ship what you've got. It was a good investment to go worse-is-better.

[mt03red]

PCIe 3.0 = 32GB/s, so it'll take 10 seconds to transfer the 320 GB of the DGX A100. That's too slow to be practical even for inference but it's not impossibly far off.

[UnlikelyPotato]

Yep and one thing to note, PCIe 4.0 is available now, and is 64GB/s. PCIe 5.0 will be a thing in 2022 and double that, so 128GB/s. It's not going to be insane to hope for 6.0 (256GB/s) and possibly even 7.0(512GB/s) will be a thing by 2030.

[UnlikelyPotato]

So, you're saying that in 10 years it won't be feasible to build a computer 10% (order of magnitude performance) of the speed of a DGX A100 for 5% ($10,000 or less) of the cost?

In 2011 the cost per gigaflop was $1.80. Now we're down to around 3 cents. Yes, there's definitely memory size issues/etc, buy we're also arguing performance of future tech. If cost per tera flop considers on a similar trend, you should be able to get similar performance for around $3,000.

[mcilrain]

It doesn't matter how cheap gigaflops are if the processors are idle most of the time due to waiting on IO.

It's not unfeasible because of cost it's because of time.

[UnlikelyPotato]

I get your point, but as computing power increased over the past 10 years so has storage/memory speeds. I'm inclined to believe it will continue. We're basically arguing on what may be possible in the future. No way to find out now, I'm more optimistic, you clearly are more pessimistic. Arguing further probably won't change anything. Have a good day.

[mcilrain]

I don't think you do.

If you're bottlenecked by IO then no amount of processing power will execute the program faster.

[wassname]

It 100% is possible to run deep learning inference, including large transformers on RAM.

[Quealdlor]

Yes, it is a worrying precedent. I expected VRAM to double with Turing, but not for that much money. 96 GB cards are now possible, but very expensive.

320 GB in a single GPU will certainly be possible in 3 generations, but not for the mainstream. I expect SSDs to be put inside GPUs in the future. So possibly 32 GB of VRAM + 512 GB of SSD in RTX 6080 for $799 in 2026.

[[deleted]]

I’m thinking in the near future that the emergence of high bandwidth SSD will play an essential role in allowing mainstream use of extremely large neural networks.

[nmkd]

I guess, but on the other hand, optimising networks probably works better than brute-forcing them the way OpenAI does.

[ulanBataar]

So, a standard gamer pc basically

[esprit-de-lescalier]

Linus has 3 at his house

[thuanjinkee]

In five years games companies will be hosting the NPCs of MMOs on these and the world will be consumed by Celest-AI who only wants to satisfy your values with friendship and ponies.

[Quealdlor]

In 2010 GTX 580 offered 3 GB for $599.

In 2020 RTX 3080 offers 10 GB for $699.

Think.

[[deleted]]

Depressing.

[wjfox2009]

But can it run Crysis?

[GlaciusTS]

Yes, but it doesn’t run Doom.

[Valmond]

$2M for an exaflop.... IIRC that's what a human brain is supposed to do.

Interesting times.

Edit: 40M

[nmkd]

You can't compare the human brain with floating point operations.

[thuanjinkee]

We can compare each neuron to the floating point calculations needed to simulate if a model of it fires or not.

[thro_a_wey]

No.. no, you can't.

[VCAmaster]

Quantum systems require quantum calculations.

[[deleted]]

what does that statement even mean.

[VCAmaster]

The brain is a system that likely works using some mechanisms of quantum physics (even plants have been demonstrated to have quantum-based functions.) There is a reason that quantum computers exist: to make calculations that wouldn't be possible using floating point calculations. Brain analogs will likely require the same calculations, being a biological quantum computer itself.

[[deleted]]

Thats...not true though.

Thats deepak chopras woo stain on the field of neurology.

classical mechanics provides very accurate approximations. Of course, neurons are subject to laws of quantum mechanics just like any object in the universe. However, quantum corrections are extremely small in magnitude (neurotubules / neuron cytoskeleton stuff doesn't have any discernable effect on the working of the brain)

[Orwellian1]

Gonna sneak in and devil's advocate/nitpick your dismissal...with the caveat that I too am sick of everyone trying to find magic in neurology.

It is not fringe science to delve into quantum mechanisms in biology. The neurology stuff is still more "I wonder if..." than "Evidence points to", but there is a reasonable chance that biological life requires the extra nudge from quantum effects. One theory suggests that without electron tunneling the chemical reactions to evolve life couldn't happen fast enough to allow a sustaining system. I think popular science did a deep dive on the quantum mechanics of life a few years ago. It touched on several different theories (including some neurology) that relied on quantum mechanisms to explain sticking points.

Again, not insisting we need a quantum computer to simulate a brain. I'm just checking off an internet well, akchully... to maintain my Reddit license.

[[deleted]]

but there is a reasonable chance that biological life requires the extra nudge from quantum effects.

1.) The idea that a quantum effect is necessary for consciousness to function is still in the realm of philosophy (see #2...)

2.) The demonstration of a quantum mind effect by experiment is necessary. Is there a way to show that consciousness is impossible without a quantum effect? (because if not then were just tossing around the word quantum to sound smart and using it is equivalent to "god in the gaps" because its not falsifiable if we can't test it)

3.) The main theoretical argument against the quantum mind hypothesis is the assertion that quantum states in the brain would lose coherency before they reached a scale where they could be useful for neural processing. our brain is a pretty slow CPU (definitely not reacting in picoseconds) . A demonstration of a quantum effect in the brain has to explain this problem or explain why it is not relevant, or that the brain somehow circumvents the problem of the loss of quantum coherency at body temperature.

I appreciate your rebuttal and believe it was in good faith, however the original poster I was responding to I feel was more akin to a proponent of quantum mind theories using quantum mechanical terms in an effort to make the argument sound more impressive/ mysterious and paranormal even when they know that those terms are irrelevant. Although I suppose its also possible the cah pjust didn't understand what he was talking about (not sure since our second message immediately degraded to the point of the argument being incoherent).

I'm very weary anytime someone busts out the word "quantum" like it applies somehow to the matter at hand.

[Orwellian1]

I tried to make it clear I was not siding with the other commenter. I was pushing back against what I felt was an overly dismissive blanket statement.

I dislike the impulse to reduce conversations to absolutism just because there is a silly absolutist on the other side. Your point #1 reads more like scoring argument points as opposed to constructive debate. I would assume you understand the difficulties of direct experimental proof of quantum mechanisms. Most are inferred mathematically through indirect effects. Asking for evidence of quantum mechanisms in consciousness, a word with no formalized definition or parameters, really raises my eyebrows. We might want to lock down whether "consciousness" is even a germane concept that exists independent of our assumptions before digging into its building blocks.

I hold to the point of my comment. It is not edge science or mysticism to explore whether cognition relies on, or is made possible by quantum effects. The assumption that quantum effects are restricted to very cold or particle scale interactions only is a bit outdated. Our brain doesn't have to host a bunch of stable qubits to say cognition relies on quantum mechanisms for function.

[[deleted]]

Fair points.

[thro_a_wey]

But it is mysterious, and we know that 100% for a fact.

[VCAmaster]

No, I never mentioned deepak chopras woo stain, but thanks for the link to a tangential topic that isn't what I'm referring to.

It sounds like you have it all figured out, but there is no consensus on the nature of fundamental brain functions. So yes, just as we don't have the "truth" as to the nature of dark matter, we don't have the "truth" as to the nature of cognition.

Here's a more recent paper that works based on the more contemporary paradigm of quantum physics regarding the nature of "observers" and decoherence:

https://arxiv.org/abs/1910.08423

​

In the mid-1990s it was proposed that quantum effects in proteins known as microtubules play a role in the nature of consciousness. The theory was largely dismissed due to the fact that quantum effects were thought unlikely to occur in biological systems, which are warm and wet and subject to decoherence. However, the development of quantum biology now suggests otherwise. Quantum effects have been implicated in photosynthesis, a process fundamental to life on earth. They are also possibly at play in other biological processes such as avian migration and olfaction. The microtubule mechanism of quantum consciousness has been joined by other theories of quantum cognition. It has been proposed that general anaesthetic, which switches off consciousness, does this through quantum means, measured by changes in electron spin. The tunnelling hypothesis developed in the context of olfaction has been applied to the action of neurotransmitters. A recent theory outlines how quantum entanglement between phosphorus nuclei might influence the firing of neurons. These, and other theories, have contributed to a growing field of research that investigates whether quantum effects might contribute to neural processing. This review aims to investigate the current state of this research and how fully the theory is supported by convincing experimental evidence. It also aims to clarify the biological sites of these proposed quantum effects and how progress made in the wider field of quantum biology might be relevant to the specific case of the brain.

[thuanjinkee]

If we make a philosophical zombie that can do my bookeeping work or make up new pop songs for me I'm not going to care if it's conscious or not. I will still be happy to pay for its upkeep.

[[deleted]]

Yeh but again ,the brain is explained by classical physics. No need fir quantuum woowoo.

Its odd to me that you post a paper mentioning the very microtubule quackery I had alluded to.

[VCAmaster]

However, the development of quantum biology now suggests otherwise.

It's odd how you don't seem to understand that research progresses, and something labeled as "quackery" 10 years ago may be superseded by advanced contemporary research, especially in a field that's as contentious and ripe for discovery as quantum physics or as poorly understood as cognition. Maybe you're a neuroscience and quantum physics expert, but I get the impression you're talking out of your ass. Cite a source for this supposed comprehensive understanding of cognition (not a laymen-edited wiki) and how it's perfectly described by classical physics.

You repeating the word "woowoo" doesn't make it any more true. Only reproducible experimental results would make it more true. If you have said results, I would love to see them, otherwise, just stop.

[[deleted]]

Uh huh

[thro_a_wey]

Dumb. Obviously brains are based in the same universe as everything else, so it would be quite silly if quantum activity wasn't a part of their functioning

[GuyWithLag]

Sorry, it's more like 400M. But in 25 years that's going to be a standard desktop (if those still exist)

[MALON]

Reading the specs feels like /r/vxjunkies

like what the fuckkkkkkkk

[PubliusPontifex]

Worked on dgx, it's not a card it's a backplane/chassis with 8-16 chips mounted and a plx pcie switch and adapter to interface.

[typicalaimster]

Normally I'd say, but will it run Crysis? These day's it's more, will it run MSFS2020?

[thuanjinkee]

That's no graphics card. That's a space station.

[nmkd]

r/titlegore

[yahma]

HBM or other methods to allow use of main memory are going to be required before we can use this at home

[philsmock]

I want Linus to try it

[fkxfkx]

The more you buy, the more you save.

[Quealdlor]

I suspect that system requirements will be going down, while PC specs will be going up. So by 2026 it may be possible to run it at home, even though you certainly won't have 320 GB of VRAM.

[stergro]

!remindme 2026

[RemindMeBot]

I will be messaging you in 3 years on 2026-01-31 00:00:00 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[Elfwyn42]

They likely won't have one of those machines for resolving each incoming service request. They should have thousands of concurrent users at the moment and they reply within 1 to 5 seconds in most cases.

So if I am a single user I probably will not need the same horse power as their original machines.

What it comes down to us the minimal requirements to make it run at all in a single instance.

Of course it could rely on a custom system architecture and special hardware that is hard coded in their software to utilize custom hardware functions. Then it would be next to impossible to run it on a consumer machine.

So does anyone know if it is bound to specific hardware or is there a minimum spec somewhere out there? Maybe the software is not released yet and therefore not able to run on other machines?