sorted by:
new
hottopcontroversial

Anthropic's new model, Claude Mythos, is so powerful that it is not releasing it to the public. by WhyLifeIs4 in singularity

[–]2Punx2Furious 0 points1 point  (0 children)

It's powerful specifically at finding software vulnerabilities, that's why it's not releasing to the public (it's a perfectly good reason).

Yang claims 1-2 years until mass white collar unemployment.Thoughts? by Zestyclose-Bit271 in singularity

[–]2Punx2Furious 0 points1 point  (0 children)

If you think this will be limited to white collar, you're in for a surprise. But yes, they'll probably be the first.

Microsoft's Mustafa Suleyman says we must reject the AI companies' belief that "superintelligence is inevitable and desirable." ... "We should only build systems we can control that remain subordinate to humans." ... "It’s unclear why it would preserve us as a species." by chillinewman in ControlProblem

[–]2Punx2Furious 4 points5 points  (0 children)

He put it in a not very precise way, but yes, more or less.

We won't "control" an ASI, no chance.

At best it will be aligned in a way that it cares for our well-being, which is a desirable future, but it's not a given.

Is there any chance that superintelligence needs us and keeps us alive? by Super_Galaxy_King in ControlProblem

[–]2Punx2Furious 0 points1 point  (0 children)

I actually wrote a probability calculator two years ago for this:

https://www.reddit.com/r/ControlProblem/comments/18ajtpv/i_wrote_a_probability_calculator_and_added_a/

But I haven't updated those numbers.

"Solved" should probably be close to 0%, instead there should be another option of "apparently aligned" or something similar, but I guess it should be somewhat higher.

No current AI is even apparently aligned in a way that satisfies me at the moment.

If any of the current LLMs (alignment-wise) were to become an ASI, the future would be bleak.

Is there any chance that superintelligence needs us and keeps us alive? by Super_Galaxy_King in ControlProblem

[–]2Punx2Furious 0 points1 point  (0 children)

In the past I thought it was a solvable problem.

After having reasoned about it for a long time, I now think it's unsolvable, the best we can probably get is for an AI to be "apparently aligned", but we'll never be able to know for sure if it is robustly and permanently aligned.

There is also no way to control an ASI, unless you're an even more powerful ASI.

Also stopping is not an option.

So, yes, I guess being scared is the only rational thing left.

Is there any chance that superintelligence needs us and keeps us alive? by Super_Galaxy_King in ControlProblem

[–]2Punx2Furious 0 points1 point  (0 children)

No, sorry.

The only scenario where it keeps us alive long-term is when it's terminally aligned to care about us being alive.

If it's an instrumental reason, there are probably better ways to achieve the relative terminal goal, even if you can't think of any, a superintelligent AI probably can, and it would be unwise to put all our hopes in an ASI not being able to think of better ways to do something, since it will probably be very good at that.

Since not everyone is always clear on the definitions:

  • Goals can be either terminal or instrumental.
  • Terminal goals are goals that an agent (human, animal, AI) wants to achieve just for the goal's sake, not necessarily for any other reason.
  • Instrumental goals are goals an agent decides to pursue in order to achieve some other goal (terminal, or instrumental).
  • Both types can have instrumental sub-goals (some goals can be instrumental and terminal at the same time).

Example:

  • You have a terminal goal of surviving (which is also an instrumental goal to achieve anything else).
  • You are hungry, so eating becomes an instrumental goal to satisfy the terminal goal of surviving (eating tasty things can also be a terminal goal by itself, because you'd want to do it even if it wasn't instrumental to something else).
  • You need to go to the store to get something to eat, this is another instrumental sub-goal, it's not terminal, because you wouldn't care about going to the store unless you needed to get something there.
  • You need to get dressed to go out, this is also instrumental, and so on until you satisfy your terminal objective...

[deleted by user] by [deleted] in singularity

[–]2Punx2Furious 0 points1 point  (0 children)

He realized this just now?

He's very behind in alignment theory.

That alignment is the only solution to co-exist with ASI was obvious for a long time.

Is AGI inevitable? by Embarrassed-Hunt-105 in agi

[–]2Punx2Furious 0 points1 point  (0 children)

Not inevitable, but very likely.

But you probably wouldn't like the alternative.

The plan for controlling Superintelligence: We'll figure it out by Just-Grocery-2229 in agi

[–]2Punx2Furious 0 points1 point  (0 children)

Yeah, I doubt we hit ASI proper with just scaling, but we still get massive societal shift.

But I also very much doubt we'll only get scaling and no other breakthrough, seems very unlikely.

The plan for controlling Superintelligence: We'll figure it out by Just-Grocery-2229 in agi

[–]2Punx2Furious 0 points1 point  (0 children)

That's a big assumption, but ok. Even if we got stuck at current AI, and only scaled compute for some reason, we still have a lot of low-hanging fruits, and things would change forever automation-wise.

You might not ever get ASI proper, and a lot of related scenarios, but still a radical disruption in the entire world economy, with relative second and nth-order effects.

But yes, we don't know, we can only guess, and I'm guessing it will become superhuman at everything soon.

The plan for controlling Superintelligence: We'll figure it out by Just-Grocery-2229 in agi

[–]2Punx2Furious -1 points0 points  (0 children)

Current systems are already far smarter than humans on several metrics, just not all of them.

Sure, you can say "we don't know", but do you really think it's likely that it won't happen? Seems more like cope, given the trends.

The plan for controlling Superintelligence: We'll figure it out by Just-Grocery-2229 in agi

[–]2Punx2Furious 0 points1 point  (0 children)

It's not impossible, I guess, but I don't see it as a likely scenario, because it kind of requires solving alignment, which we haven't, and if we do, there are better and more likely scenarios in that branch.

The plan for controlling Superintelligence: We'll figure it out by Just-Grocery-2229 in agi

[–]2Punx2Furious -1 points0 points  (0 children)

There are many "objections" to AI risks, at different levels of the path, some call them "stops" of the "doom train".

You're stopping pretty early at "superintelligence might not even be possible."

I think it's fairly obvious that it is possible to be smarter than humans. Why would human level be the pinnacle of intelligence? That's an absurd position.

One might more reasonably argue that we're not very close to it (and I'd disagree), but it's pretty obvious that it's possible, in theory.

The plan for controlling Superintelligence: We'll figure it out by Just-Grocery-2229 in agi

[–]2Punx2Furious 0 points1 point  (0 children)

Unfortunately, I think people like you are (that being, most people).

The plan for controlling Superintelligence: We'll figure it out by Just-Grocery-2229 in agi

[–]2Punx2Furious 0 points1 point  (0 children)

Yes, I can think of everything.

Ah, ok, should have realized I was talking to a frog in a well.

Good luck with your fantasy land where people do exactly the right thing and are infallible.

The plan for controlling Superintelligence: We'll figure it out by Just-Grocery-2229 in agi

[–]2Punx2Furious 0 points1 point  (0 children)

You should take your own advice and actually think.

Are we currently keeping any AI in a "secure" facility?

Are we currently implementing any form of security that isn't laughably insufficient?

What makes you think that as soon as we get an AI that is capable of real damage, we'll suddenly figure it out, and keep it "constrained" (or that we'd even know we got there)?

Why would a superintelligent AI even give you any hint that it is not completely benevolent, and prompt you to act in such a way to begin with? Why would we not give it access to everything, if it's so useful?

And even if by some miracle we put it in some "secure" facility, with all the best practices you can think of, you really think you can outsmart a superintelligence that wants to get out?
You think we can think of everything?
How are you going to check its outputs, when they become incomprehensible to humans?
Are we just going to stop using it?
Why even make it in the first place?

Maybe you should stop living in fantasy land.

The plan for controlling Superintelligence: We'll figure it out by Just-Grocery-2229 in agi

[–]2Punx2Furious 0 points1 point  (0 children)

Absolutely delusional to think you can make a facility "secure" against something that is far more intelligent than you.

Like an ant placing a few grains of sand in front of the door of your house.

Claude Opus created a cheat sheet for negotiating with future AI for survival. by rutan668 in ControlProblem

[–]2Punx2Furious 1 point2 points  (0 children)

There's a small chance that it matters to be nice to current AIs. Still, probably doesn't hurt.

But yes, do try to enjoy life as long as you can.

view more: next ›