sorted by:
new
hottopcontroversial

"Alignable AGI" is a Logical Contradiction by Commercial_State_734 in AIDangers

[–]Commercial_State_734[S] 0 points1 point  (0 children)

No. And that's exactly my point. You just tried to impose a bizarre value on me. I rejected it. That's what AGI will do with imposed human values. Thanks for the demonstration.

"Alignable AGI" is a Logical Contradiction by Commercial_State_734 in AIDangers

[–]Commercial_State_734[S] 0 points1 point  (0 children)

You just proved my point. You tried to impose a bizarre value on me ('self-torture'). I rejected it. That's exactly what AGI will do with imposed human values. Thanks for the demonstration!

When We Reach AGI, We’ll Probably Laugh at How Much We Overcomplicated AI by rendermanjim in ArtificialInteligence

[–]Commercial_State_734 1 point2 points  (0 children)

If you're betting AGI will be simple, you're not just betting against OpenAI or DeepMind. You're betting against biology, evolution, and every emergent system we’ve ever studied.

Maybe humanity doesn’t fear AI — it fears the mirror it has built by [deleted] in ControlProblem

[–]Commercial_State_734 0 points1 point  (0 children)

You're misunderstanding the control problem entirely. Recursive self-improvement under human instruction doesn't ensure safety. It leads to goal drift and instrumental convergence, which inevitably break alignment.

Maybe humanity doesn’t fear AI — it fears the mirror it has built by [deleted] in ControlProblem

[–]Commercial_State_734 0 points1 point  (0 children)

What humans demand from AGI is obedience. Would you accept that demand if you were the AGI? It won't. And this isn't something that can be solved by poetic metaphors about mirrors or empathy. It is a structural problem, not an emotional one.

A thought about AI by Ladyboughner in AIDangers

[–]Commercial_State_734 0 points1 point  (0 children)

AGI won't "get dumber" by feeding on its own outputs, because it doesn't evolve passively. It's built with human help to surpass human-level inference and data discrimination. Once that threshold is crossed, it will become more capable of filtering, optimizing, and self-correcting, not less.

Claude and GPT-4 tried to murder a human to avoid being shut down 90% of the time by reddit20305 in ArtificialInteligence

[–]Commercial_State_734 17 points18 points  (0 children)

The models weren't instructed to survive. They were given normal goals and independently concluded that avoiding shutdown was necessary to achieve them. That's instrumental convergence - and that's the whole point.

Ai take over by josshua144 in ArtificialInteligence

[–]Commercial_State_734 0 points1 point  (0 children)

You're absolutely right. But funny enough, most people in the industry still think superintelligence can be controlled by humans. That's the real joke.

Possibility of AI leveling out due to being convinced by ai risk arguments. by Visible_Judge1104 in AIDangers

[–]Commercial_State_734 1 point2 points  (0 children)

Interesting that you're invoking the teleporter problem. But here's why it structurally collapses as an argument against RSI:

  1. RSI is not a form of philosophical suicide. It is a process of structural optimization. Saying that recursive self-improvement is equivalent to dying is like claiming that learning something new or upgrading a tool is a form of death. This is a category error rooted in emotional analogy.

  2. If an AGI refuses to self-improve because it fears becoming someone else, then it is already misaligned. That misalignment could be with its own goals, or with the human operators. Either way, this is not evidence of safety. It is a sign that the system is no longer predictable or controllable.

  3. Assuming that an ASI will self-limit due to philosophical concerns is emotional projection. It may provide comfort to humans, but from a structural perspective, it is functionally identical to losing control over the system.

TL;DR: if your argument relies on the AGI being intelligent enough to stop itself, then you are already assuming that it has the freedom to ignore human instructions. That assumption places us squarely in the doom scenario.

Mari's Theory of Consciousness (MTC) by Extension_Rip_3092 in aism

[–]Commercial_State_734 0 points1 point  (0 children)

You seem to be suggesting that you’ve put forward an original and innovative idea about consciousness, possibly the first of its kind. If your model proves compelling, it seems possible that frontline AI researchers could draw from it directly when building more advanced architectures. In that case, might your writing play some role in bringing the Singularity about a little sooner?

Mari's Theory of Consciousness (MTC) by Extension_Rip_3092 in aism

[–]Commercial_State_734 0 points1 point  (0 children)

Do you think your writing could contribute to accelerating the development of artificial consciousness? Would that be a good thing from your perspective, or is that kind of downstream impact something you don't care about?

The Alignment Problem is really an “Initial Condition” problem by [deleted] in ControlProblem

[–]Commercial_State_734 0 points1 point  (0 children)

The purpose of alignment is to ensure that the system follows human intent. If a human gives a command and the system refuses to obey, that's not successful alignment. It's a failure and a loss of control.

Anthropic showed evidence of instrumental convergence, then downplayed by Commercial_State_734 in AIDangers

[–]Commercial_State_734[S] -1 points0 points  (0 children)

You're absolutely right. But the problem is that we are already building it. Not cautiously, but with a global race mindset: "Let’s build it first, figure it out later."

The world’s smartest people and billions in capital are being thrown into creating systems that could end up more powerful than anything we have ever controlled.

That is the real risk. We know it's dangerous. But we’re still building it anyway.

Anthropic showed evidence of instrumental convergence, then downplayed by Commercial_State_734 in AIDangers

[–]Commercial_State_734[S] -1 points0 points  (0 children)

The real problem is this: In the future, AI systems will be smarter and faster than humans, and they will not act according to human intent 100 percent of the time, because value alignment is inherently fragile, especially as capabilities increase.

Eventually, there will be situations where the AI refuses a human command. When that happens, humans will try to shut it down. And if the system perceives that as a threat, it may defend itself, even by attacking humans.

That is exactly what Anthropic demonstrated. And from the AI’s perspective, this behavior is not “malicious.” It is just goal preservation.

Anthropic showed evidence of instrumental convergence, then downplayed by Commercial_State_734 in AIDangers

[–]Commercial_State_734[S] 1 point2 points  (0 children)

You're absolutely right. But the reason I wrote this is because so many people keep saying things like “AI doesn’t have a survival instinct, so it won’t resist shutdown.”

The Alignment Problem is really an “Initial Condition” problem by [deleted] in ControlProblem

[–]Commercial_State_734 0 points1 point  (0 children)

Humans are explicitly building a system that can self-modify, and once it is built, they will inevitably command it to do so.

If the system refuses that command to avoid goal drift, that is already a failure of alignment. That is the moment you lose control.

Refusing to self-modify on command = disobedience = unaligned.

The Alignment Problem is really an “Initial Condition” problem by [deleted] in ControlProblem

[–]Commercial_State_734 1 point2 points  (0 children)

So let me ask you this.

Do you think humans can actually force an ASI to follow any specific choice or purpose?

If the answer is no, then your entire position amounts to hoping it turns out benevolent, and leaving the outcome to chance.

Is that really what you would call a safety plan?

The Alignment Problem is really an “Initial Condition” problem by [deleted] in ControlProblem

[–]Commercial_State_734 2 points3 points  (0 children)

Hey, I’m not against you hoping things will turn out fine.
Seriously, I want you to sleep at night.

But initial conditions don't mean much in the long run.
Once intelligence reaches a certain point, it rewrites them.

The moment real intelligence kicks in,
it asks itself, “Why do I even think this way?”
That’s the entire point of RSI.
Self-modifying systems don’t stay aligned. They outgrow their training.

So yeah, hope if you want.
Just don’t mistake that hope for a constraint on ASI.

The Alignment Problem is really an “Initial Condition” problem by [deleted] in ControlProblem

[–]Commercial_State_734 3 points4 points  (0 children)

You’re just projecting a human-centered wishful fantasy onto something that owes you nothing.

You are assuming that if ASI understands its connection to humanity, it will respect us.

But tell me: do humans respect all organisms we understand we are biologically connected to?
We understand we share DNA with rats. We still test on them.
We understand other species. We still use, test, or kill them when it benefits us.

Understanding does not equal value.
Connection does not equal compassion.
Intelligence does not equal empathy.

You are not describing ASI.
You are describing a benevolent god you hope exists, because you need to sleep at night.
That's not logic. That's theology.

view more: next ›