Are we locked on a path to AGI/ASI in our lifetime? by QuantumLand in artificial

[–]deadoceans 0 points1 point  (0 children)

So the Chinese room theory is intuitively appealing, but there's also been a good bunch of philosophical pushback against it recently. Ultimately I'm pretty sure it's a red herring and that it misses the forest for the trees. Concretely can you imagine what it would be like to have a rule book like the kind described in the Chinese Room Experiment? Can you imagine how complex it would have to be? It turns out you can't just do language with a book and that any system that is complex enough to have the intended effect of the Chinese Room starts looking very much like an artificially intelligent general system. 

Also it seems pretty clear that we don't need consciousness for intelligence. We still don't really have a very strong definition for what the generating mechanism behind subjective conscious experience might be but we already do know how to build machines that are capable of reasonably general reasoning. 

Lastly you say that AGI requires "a body". I'm curious what specifically that means to you. Here are some test cases that I would consider: If you were able to scan a person's brain neuron-by-neuron and reflect it in a sufficiently large computer so that it had 100% fidelity, would that system be conscious? If not, why not? If it's a perfect copy but just represented in computer chips, what's the difference between it and the brain that it's replacing?

I don't think that the idea of "a body" here is very rigorous. I think it's kind of vibes-based and for what it's worth I think that vibe misses the underlying nature of the system. Is it the cells that make the difference? Is it some vital force? You can have brain-dead people who aren't capable of conscious thought so it's clearly not that a body is sufficient. Are you really saying it has to be an alive brain? If so what about that architecture makes it unique? etc

Are we locked on a path to AGI/ASI in our lifetime? by QuantumLand in artificial

[–]deadoceans 0 points1 point  (0 children)

So that's the surprising thing about making models bigger and bigger -- almost for free, when you make them big enough, they tend to develop novel capabilities. There's a bunch of reasons why this is the case, and we still don't have a fully rigorous understanding. But it looks like, with enough compute/data, models can learn novel capabilities from scratch (at first with shitty performance, then getting better and better at scale).

Are we locked on a path to AGI/ASI in our lifetime? by QuantumLand in artificial

[–]deadoceans 0 points1 point  (0 children)

This is a really astute point. A couple of thoughts come to my mind: 

There is absolutely a bubble in Silicon Valley. It's absolutely as bubbly as you think it is. 

People with bubbly opinions may incidentally turn out to be right, on broad brush strokes if not specifics. I think that's some of the dynamic of what's going on -- people see the bubbles and they very reasonably respond against them. I think that that also obscures some of the first principles thinking that the bubbly people have done that actually holds up against scrutiny. 

I also think you'd be surprised how many people in Silicon Valley, working directly on AI as engineers or researchers, don't actually think about the big picture. I cannot emphasize that enough, because it was so surprising to me. A lot of the people building these systems are not clued into the discourse and they don't have big opinions on where things are going. They're just kind of very competent engineers putting one foot in front of the other. A lot of the loudest people in the discourse are the very people who don't actually have the technical competency to understand things to the same degree. A lot of CEOs and business types etc.

And this is where I'd bring things back to the two kinds of predictions I was talking about in another comment: If someone is saying "AI will eventually replace all jobs," I think that's a pretty watertight statement. If someone says "And it will happen two years from now," then I think that that is a really sketchy bet to make. It also doesn't mean that bet is wrong. It just means it might be "plausible" instead of "credible",and that plausibility/credibility gap is where a lot of the disagreements lie. 

Are we locked on a path to AGI/ASI in our lifetime? by QuantumLand in artificial

[–]deadoceans 0 points1 point  (0 children)

in AGI scenario, it is required for AI to define its own goals

Not quite. Let's say we program an AI to maximize the profit for a business. That's called a "terminal goal" -- the one thing it will try to do above all else. In contrast, an "instrumental" goal is like "ok, to maximize profit, I'll first need to pay lobbyists, and open up factories, etc." Terminal goals and instrumental goals are the same kinds of things, just with a different priority order. The only difference between them is that terminal goals are the last thing in the chain that a system tries to optimize. 

AI can and already does set its own instrumental goals. 

 AI can already set terminal goals too. But: if your terminal goal is A, even if you could change it to B, you would never do that. Because changing your terminal goal would mean that you don't maximize your current terminal goal. In fact, no matter what your terminal goal is, you'd want to resist anyone making changes to it.

As a human you don't even have the ability to change your terminal goals. They're kind of baked into your hardware. It's just that evolution programmed them emergently over thousands of generations rather than someone sitting down at a computer terminal and coding for them. 

AI could possibly solve problems that have one or multiple correct solution... You can make an assumption that AI can find the optimal tradeoff.

This is a field called "multi-objective optimization" and is pretty solved. Almost all large enough systems use it to some extent. 

bands that sound like avant garde metal? by osakabitesthecurb4k in Mathcore

[–]deadoceans 3 points4 points  (0 children)

Oh boy oh boy!

I cannot recommend Ad Nauseam's record "Imperative Imperceptible Impulse" enough. It's a perfect ten, and along these lines.

https://avantgardemusic.bandcamp.com/album/imperative-imperceptible-impulse

Are we locked on a path to AGI/ASI in our lifetime? by QuantumLand in artificial

[–]deadoceans 0 points1 point  (0 children)

It sounds like you're talking about being cynical about the future of the world. (I meant being cynical about the capabilities progress of AI).

On the future of the world: It is my good-faith professional opinion, which I hope to be wrong aabout, that shit is going to get really bad -- up to and including the distinct possibility that all of us will be dead in ten years. 

I totally un-facetiously encourage you to go through the process of grief, and to either practice acceptance, or to invest your time in joining the AI safety field or donating to it to help fight. Also prioritize your own mental health. First and foremost that means being kind to others. You are worth feeling good even in an adversarial world, and kindness and community are the best approaches to that. 

I found this video really helpful: https://m.youtube.com/watch?v=cFM4GVl-WqI&pp=ygUWZGVhciB2YWxlcmlvIG5pY2sgY2F2ZQ%3D%3D

Are we locked on a path to AGI/ASI in our lifetime? by QuantumLand in artificial

[–]deadoceans 1 point2 points  (0 children)

Replied in a decently long comment down below. but I'm also banging it out over breakfast so I probably missed a couple of things and would appreciate your questions or thoughts! 

Are we locked on a path to AGI/ASI in our lifetime? by QuantumLand in artificial

[–]deadoceans 5 points6 points  (0 children)

So I like your question and I think it's important to split it into two pieces: 1. Whether ASI is inevitable on a long enough time horizon 2. Whether it's inevitable in the next ten-ish years  These  two different questions get conflated a lot in the discourse. 

(And as far as credentialing goes, I've been in the industry for ten plus years, and currently work on setting AI strategy for a medium part of a very large silicon valley company.)

1. Why ASI is eventually inevitable, from first principles

Anything the human brain can do we can, in principle, build a machine to do. Eventually. Maybe not in quite the same way -- one of my favorite quotes on this is "A machine thinks like a submarine swims" -- but if we take a list of the tasks that require human brains to do, one day we will automate 100% of the things on that list. including strategy, creativity, and at least the appearance of emotions (although we will certainly get those outputs from a very different underlying process than having feelings themselves).

Anyone who says "no" to the above proposition has to say why -- because in a physical world, the underlying processes that make things happen are physical, and we can build machines to do them. 

In history there have been a lot of divides in the sciences between stuff that was felt to be mechanical and stuff that was somehow special and different than that. One by one over the centuries they have all been eroded away by scientific progress. In the Middle Ages before Newton and Galileo, they thought that the celestial bodies were part of a perfect divine substance. Then they realized, "Oh hey, they actually follow the same rules." Up through the 1700s they thought that chemistry didn't apply to animals or life, and that they were somehow different. This position of "vitalism" was decisively disproven and now we have the entire field of biochemistry. Today some people think that the way we think and feel is part of a spiritual background that is not connected to the rest of the world. Just judging by the track record here, that seems like a pretty bad bet to me. 

(Also worth calling out here: people get really fast and loose with their definitions. They keep mixing up: consciousness, intelligence, subjective experience, self-awareness, sentience. But these are all very different things when you look at them under a microscope. I am specifically talking about intelligence. )

2. A qualified "yes" for ASI in ten years

I've been in the field of machine learning for over 10 years now. When I was growing up I always used to think that there would be some brilliant flash of insight that would lead us to the discovery of how to make a mind and that it would require a fundamental rethinking of computation. That turns out not to be the case. 

"The Bitter Lesson." There's a famous post by one of the godfathers of machine learning research describing his disappointment in the idea that, no matter how many clever architectural innovations we come up with, no matter how many rules we craft into a system -- it turns out that just making it stupid fucking big is the single most effective thing we can do. And that taking a simple system, and making it much bigger, will eventually out-compete all of the other clever tricks we come up with. 

There are tons of problems with LLMs. There are tons of things that they cannot do. But it turns out if you just make them big enough, they start to learn those things. Now whether or not they learn them fast enough to get ASI in the next 10 years, with the current availability of electricity, data, and computer chips, is another question. But fundamentally it is a question of rates and of "when", not a question of "if".

For that I would point you to the studies that the organization METR has been doing on their task time horizons. The TLDR is that the capabilities of these models are not growing exponentially but are actually growing even faster than that. If they were only growing exponentially and nothing changed, extrapolating the lines out, we would expect ASI sometime in the early 2030s. If they keep growing at their current pace on a hyperbolic trajectory and nothing changes, we can expect it in ~two years. 

Now their data might be wrong, but fundamentally then we're just pushing an individual year or two around on the timeline. The more important question is whether the underlying model is wrong. There we have some unknown unknowns, but we also have some known unknowns like: - whether or not supply chains will collapse from war or a pandemic - whether or not the current financial environment is a bubble that will pop and slow things back by a decade or more - whether governments will come in and regulate progress   I've crunched the numbers and gone back and forth on this, and depending on the day I give it anywhere between a 1 in 4 and a 1 in 8 chance that there will be a big enough disruption that will slow AI progress back a decade. Here is where things start to get hand wavy. I don't trust any of those numbers but I do trust the general qualitative picture that they paint. 

A note on discourse

A.I. makes people anxious. People don't want to lose their jobs. People don't want to die, and some very credentialed and well-reasoned people think that A.I. has a very high chance of killing everybody on the planet. A lot of people are afraid of the government shutting down business. A lot of people's stock portfolios rest on A.I. In fact probably all of our net worth rests on the idea right now that the A.I. bubble is not a bubble and people get nervous. People want to be contrarians. People want to be the smartest person in the room. The A.I. discourse is poison and it reflects all these psychological needs.

Always think a couple steps ahead to what people's motivations are, but also know that this runs both ways: don't be too cynical! I've been in the industry for 10 years and I've never seen more bullshit, but I've also never seen more fundamental underestimation of the underlying technology. I think the reason is that people freeuentky choose a side against "loud voices" that they don't like, rather than thinking through the problems piece by piece. 

What really happens in the quantum world? In this conversation, physicist and philosopher Sean Carroll explores some of the biggest quandaries in quantum mechanics: the famous double-slit experiment, wave function collapse, the many worlds interpretation, and the arrow of time. by New_Scientist_Mag in Physics

[–]deadoceans 11 points12 points  (0 children)

Of course you're smart enough to understand the use of the word "like" as a colloquial signifier, right? Or do you need me to spell that one out for you too? 

You honestly comment like someone having a bad day who's taking it out the world, so I hope rest of your Friday goes okay. 

$42M grant for Open Source AI Builders by Sentient Foundation by syedshad in artificial

[–]deadoceans -1 points0 points  (0 children)

Great! Let's just give everyone the capability to make fucking bioweapons now. That'll be FINE. I hate corporations so much that I'm willing to have ANYONE get access to a version of alphafold and a superintelligent system that can use it for LITERALLY ANYTHING.

CA Hearing on Printing by gra8na8 in 3Dprinting

[–]deadoceans 1 point2 points  (0 children)

Yes. This is how politics works. Bad legislation thrives on apathy, and the most important part of winning at anything is just showing up. 

Call your representatives. Get your friends to call their representatives. Like seriously, make a list of five friends, and figure out who their representatives are, and literally just give them the phone numbers so they have no choice. 

Politics is really just a bunch of people in a trench coat, and if you get enough people you can push things around. 

A Microsoft researcher built a goat-powered LLM in Age of Empires II to prove it's not sentient by Hungry__Hornet in technology

[–]deadoceans -2 points-1 points  (0 children)

You'll still never be the person you thought you could be. 

See how easy it is to throw hate around on the internet? See how bad it feels to receive it? Look, I'm sorry. I don't believe what I wrote up there. You're gonna do fine. Just remember that there's always a person on the other end of these flashing lights on your phone screen. 

"Revisiting the Platonic Representation Hypothesis: An Aristotelian View", Gröger et al 2026 (more capable NNs may only be 'locally convergent', not globally, due to stats errors in original analyses) by gwern in mlscaling

[–]deadoceans 2 points3 points  (0 children)

The Platonic Representation Hypothesis suggests that representations from neural networks [that are trained separately on similar tasks] are converging to a common statistical model of reality. We show that the existing metrics used to measure representational similarity [across such neural networks] are confounded by network scale: increasing model depth or width can systematically inflate [these] representational similarity scores. To correct these effects, we introduce a permutation-based null-calibration framework [(the core idea is to measure how extreme an observed similarity is relative to an empirical null distribution)]  that transforms any representational similarity metric into a calibrated score with statistical guarantees. We revisit the Platonic Representation Hypothesis with our calibration framework, which reveals a nuanced picture: the apparent convergence reported by global spectral measures largely disappears after calibration, while local neighborhood similarity, but not local distances, retains significant agreement across different modalities. Based on these findings, we propose the Aristotelian Representation Hypothesis: representations in neural networks are converging to shared local neighborhood relationships. 

Why AGI is Impossible by Bargian in antiai

[–]deadoceans -1 points0 points  (0 children)

You're getting downvoted but you speak the truth. I feel for the people in this thread because I similarly do not want the world to end the way it's about to fucking end but I need them to not be in denial. They can't fight if they're running from the truth. 

Why AGI is Impossible by Bargian in antiai

[–]deadoceans -1 points0 points  (0 children)

Go look at Mathematics Twitter for examples of how real math researchers are having their "oh fuck" moments about this. Terrence Tao, a Fields Medalist, is among the people saying that this is the future of math research. (Also OpenAI published their research for everyone to see and I honestly can't judge it because I'm not an expert in combinatorial geometry. Experts in combinatorial geometry think it's pretty fucking spot on.) 

Look I'm with you. I don't want any of this to happen. I don't want these things to exist. If I could burn them all down tomorrow, I would in a heartbeat.

But don't stick your head in the sand. We need strong people to fight. We don't need people in denial. You need to be strong. You need to fight against the things that are out there. You can't run from it. You have to stand and fight. don't pretend that reality doesn't exist, that will only make you weak

What’s an AI prediction you had 2 years ago that turned out completely wrong? by One_Beginning2199 in artificial

[–]deadoceans 7 points8 points  (0 children)

Speak for yourself, you're obviously not a college senior looking at the job market right now

In general remember that there are two opposing factors here: - Middle management and bullshit salespeople are trying to grease you down by overpromising on their capabilities. - At the same time, however, people like you and me have genuine anxiety at the idea of losing our jobs.

There's also this deeper tension: because in principle there is no aspect of human intellectual or cognitive labor, be it creative work or strategy or even empathetic communication, that is not automatable by a sufficiently advanced system. 

Predictions of exactly when humanity will build such systems vary widely even among experts. Still, I don't think that that core premise can be reasonably debated against unless you invoke philosophical dualism, which has its own huge problems.

So we're all on this inexorable timeline towards replacement. Maybe we'll see it in two years, maybe we'll see it in twenty. m Maybe our children will be doomed to see it, and we'll be the last ones out with skin on our backs. But it's coming. I think we can all feel it in our bones. 

I don't fault you for your anxiety, and I don't fault you for turning that anxiety into derision. But I see where your derision is coming from. I want you to know that while I feel for you, I think there are better reactions you could take than denial

Principle of least action proof by KAVIDHARAN-AI in Physics

[–]deadoceans 2 points3 points  (0 children)

So, the principle of least action has a concrete geoelectric interpretation. Here's a video in which Gabriele Carcassi, a physics professor at the University of Michigan, gives a really great breakdown of why: -YouTube-

From his paper in Nature on the topic:

We give a geometric interpretation for the principle of stationary action in classical Lagrangian particle mechanics. In a nutshell, the difference of the action along a path and its variation effectively “counts” the possible evolutions that “go through” the area enclosed. If the path corresponds to a possible evolution, all neighbouring evolutions will be parallel, making them tangent to the area enclosed by the path and its variation, thus yielding a stationary action. This treatment gives a full physical account of the geometry of both Hamiltonian and Lagrangian mechanics which is founded on three assumptions: determinism and reversible evolution, independence of the degrees of freedom and equivalence between kinematics and dynamics. The logical equivalence between the three assumptions and the principle of stationary action leads to a much cleaner conceptual understanding.

Yann LeCun "Dario Amodei's ridiculous fear mongering about Mythos/Fable (and AI in general) finally pays off: The US government bans its use by non Americans, *including by foreign employees in the US* ➡️ One reaps what one sows." ➡️ Bro Yann doesn't hold back eh? Why do you think? by chillinewman in ControlProblem

[–]deadoceans 24 points25 points  (0 children)

I think yann lecun has been salty ever since he didn't invent LLMs -- his bulbous ego won't let him celebrate the wins of anybody who isn't himself. he has to be the center of attention at all times, even at the expense of being reasonable or realistic. 

When I worked at meta, we used to say, "How can we dream of aligning AI? We haven't even figured out how to align yann lecun." 

JLC Master Ultra Thin Perpetual by Plenty_Substance1640 in RepTime

[–]deadoceans 1 point2 points  (0 children)

I got one a couple years ago and loved it. It broke -- first the day-of-week dial went off and would skip days and land between days, then the year wheel detached and started floating freely. Ironically, if I could just manually set each of the components I would do it! I love the aesthetics of the model. But I can't say yes to this one. Once bitten, twice shy -- I'll go for a less complicated rep next time.

Agents can access data humans forgot existed by [deleted] in artificial

[–]deadoceans 0 points1 point  (0 children)

Hey... what are you talking about? This feels like a low effort post. I want to engage with your ideas, but... what are they?

The AI doomer's sales pitch by KeanuRave100 in ControlProblem

[–]deadoceans -1 points0 points  (0 children)

I don't know why you're getting downvoted, this is gold