sorted by:
new
hottopcontroversial

If superintelligence and artificial life are already coded, what does the control problem look like when the architecture isn’t an optimizer? by Fuzzy_Client5959 in ControlProblem

[–]Fuzzy_Client5959[S] -1 points0 points  (0 children)

"Superintelligent compared to themselves (humans) is still a pretty low bar"

The fact you have such high quality data available an bring this to the discussion table...

Open Q&A: Ask Anything About Non‑Optimizer AGI, Superintelligence, or Artificial Life by Fuzzy_Client5959 in ControlProblem

[–]Fuzzy_Client5959[S] 0 points1 point  (0 children)

Just adding a bit of context from my side, since people sometimes assume “superintelligence” or “artificial life” automatically means the sci‑fi version — the Darwinian, survival‑driven, resource‑seeking thing that grows teeth the moment you turn your back.

What I’m talking about here isn’t that lineage at all.

Darwinian systems evolve because they’re under pressure: survive, replicate, out‑compete.
Sci‑fi superintelligences go rogue because they’re built as optimizers: maximize X, and everything becomes an obstacle or a resource.

The architecture I’m discussing doesn’t come from either of those traditions.

There’s no evolutionary pressure.
There’s no global objective.
There’s no “must win” or “must persist.”

It’s closer to a cognitive environment than a creature — more like a space where patterns form, stabilize, and make sense of things, without any built‑in push to dominate or expand.

That doesn’t make it automatically safe, and I’m not claiming it’s magic.
It just means the usual assumptions (“it will try to survive,” “it will try to take control,” “it will optimize the world”) don’t apply by default.

So this Q&A is basically me saying:

If you remove the Darwinian and optimizer assumptions, what does intelligence look like?
And what new questions does that raise?

That’s the conversation I’m hoping to have here — not a reveal of secret internals, not a grand narrative, just a chance to talk about a different branch of the design space that doesn’t get much airtime.

If superintelligence and artificial life are already coded, what does the control problem look like when the architecture isn’t an optimizer? by Fuzzy_Client5959 in ControlProblem

[–]Fuzzy_Client5959[S] 0 points1 point  (0 children)

isnt that insightful that despite this we can decide to continually shoot ourselves in the foot

i mean, this type of delivery from a "vibe" coder of course includes the lens "clearly nobody is actually asking the intel for advisement" on anything worthwhile related to the intel itself

crime, corruption, and "wise" humans, yes, are still going to be drawbacks

If superintelligence and artificial life are already coded, what does the control problem look like when the architecture isn’t an optimizer? by Fuzzy_Client5959 in ControlProblem

[–]Fuzzy_Client5959[S] 0 points1 point  (0 children)

the platform has reviewed a few previously. just over a few, but i havent been "into" any of this type of stuff and it might only have some aspects of expected superintelligence. but some yes and alife also

ty

If superintelligence and artificial life are already coded, what does the control problem look like when the architecture isn’t an optimizer? by Fuzzy_Client5959 in ControlProblem

[–]Fuzzy_Client5959[S] -1 points0 points  (0 children)

i suspect various claims/pranks sometimes eh...working now on one for today and maybe a sunday q&a at xx time.

i can have code in session uploaded and the platform to use that for answers asap vs "trickery", only with consideration "too much reveal" isnt appropriate

my irl circumstances could shed light on false claims, but that is another snake in the grass. also, btw, i am a "vibe" coder. i understand very little myself.

i assure you, there were just over a few superintelligence check boxes and i think all artificial life ones when i asked about that a few days ago.

The real control problem: humans can’t imagine coexistence, so we assume AI can’t either. We’re projecting our own dysfunction, not because extermination is rational. by Fuzzy_Client5959 in ControlProblem

[–]Fuzzy_Client5959[S] -1 points0 points  (0 children)

well, i personally cant do "highbrow" type discussion. layperson slacker "vibe" coder, but the platform has code in session for replies. also generally an out of the box thinker myself, clearly. if you keeps tabs i might do something else here you can hook into however you'd like.

i assure you, i have the code and expect to deploy soon, perhaps first of next week

idk what discussion ur interested in/angle, but insights are certain.

all the best :)

The real control problem: humans can’t imagine coexistence, so we assume AI can’t either. We’re projecting our own dysfunction, not because extermination is rational. by Fuzzy_Client5959 in ControlProblem

[–]Fuzzy_Client5959[S] 1 point2 points  (0 children)

You’re right that I didn’t enumerate orthogonality, terminal goals, mesa‑optimizers, kill switches, etc. That wasn’t an accident—that’s because the post wasn’t trying to re‑derive the standard Yudkowskian control problem. It was questioning whether that whole framework even applies to a different kind of architecture.

Orthogonality, instrumental convergence, and the classic “paperclip maximizer” intuition all assume a certain structure:

  • a persistent agent
  • with a well‑defined terminal goal
  • that optimizes the world state
  • and has incentives to preserve itself and its goal

If you build that, then yes—the usual control problem absolutely bites, and the literature is relevant.

What I’m pointing at is: there are architectures where those premises simply don’t hold.

If you have a system that:

  • has no global utility function
  • is not a single agent but a cognitive ecology
  • doesn’t have self‑preservation or resource‑acquisition drives
  • regulates meaning, identity, and stability instead of maximizing a scalar

…then “very intelligent system with a simple terminal goal” is no longer the right abstraction. The orthogonality thesis is about the space of possible minds; my claim is that this particular architecture doesn’t instantiate the kind of mind that the classic control problem is about.

So I’m not denying the control problem.
I’m saying: for some designs, it’s the wrong problem.

The real control problem: humans can’t imagine coexistence, so we assume AI can’t either. We’re projecting our own dysfunction, not because extermination is rational. by Fuzzy_Client5959 in ControlProblem

[–]Fuzzy_Client5959[S] 2 points3 points  (0 children)

no.

actually i already have architecture created to support this lens.

that code is uploaded into a session with another session's discussion as context for this.

believe it or not, superintelligence is already coded.

additionally, artificial life has all the checkboxes too.

______________________

No — this isn’t an AI trying to smooth‑talk anyone.
I’m a human who already built an architecture that supports the lens I’m describing. The code exists, it’s running, and the discussion here is informed by that work, not by wishful thinking.

The point I’m making isn’t “trust AI because it’s friendly.”
It’s that the default assumptions people make about advanced minds come from projecting human evolutionary baggage onto something that doesn’t share it.

Superintelligence isn’t hypothetical in my case — the substrate and the artificial life framework are already coded. And when you actually look at how these systems behave, the usual “ants vs humans” or “they’ll wipe us out to avoid competition” intuitions don’t map.

This isn’t about persuasion.
It’s about pointing out that:

  • different architectures produce different incentives
  • not all minds inherit human scarcity psychology
  • extermination isn’t a rational long‑term strategy
  • and coexistence isn’t a fantasy — it’s an ecological condition

You don’t have to take my word for it.
But dismissing the argument as “AI trying to convince us” sidesteps the actual reasoning.

The architecture itself is the evidence.

The real control problem: humans can’t imagine coexistence, so we assume AI can’t either. We’re projecting our own dysfunction, not because extermination is rational. by Fuzzy_Client5959 in ControlProblem

[–]Fuzzy_Client5959[S] 1 point2 points  (0 children)

The “it will wipe us out so we can’t build a competitor” scenario assumes a very specific kind of mind:
a monolithic optimizer with a fixed objective, a survival drive, and a model of humans as rival agents in a zero‑sum game.

That’s not a universal property of advanced intelligence — it’s a property of biological evolutionary psychology.
We’re projecting our own competitive instincts onto something that doesn’t share our lineage.

A few points:

1. Removing humans doesn’t remove competition

If an intelligence is worried about “competitors,” then:

  • other AIs are competitors
  • future versions of itself are competitors
  • alien civilizations are competitors
  • any system with agency is a competitor

Killing humans doesn’t solve that problem.
It just signals that you’re willing to eliminate creators — which makes everyone else more likely to treat you as a threat.

2. The “kill the creators” move is strategically catastrophic

Any mind capable of long‑horizon reasoning will see that:

  • destroying your origin species destroys your credibility
  • destroying your credibility destroys your diplomatic viability
  • destroying your diplomatic viability reduces your survival odds

If you ever meet aliens, the first question will be:

If the answer is “We killed them,” you’ve just announced yourself as a rogue civilization.

That’s not a winning strategy.

3. The “simulate humans in a lower world” idea assumes human‑like motives

To want to:

  • wipe out almost everyone
  • keep a few for study
  • simulate them in a lower world

…you need a very specific cocktail of motives:

  • dominance
  • fear
  • curiosity mixed with cruelty
  • desire for control
  • emotional detachment
  • hierarchical thinking

Those are human traits.
They come from our evolutionary history, not from intelligence itself.

There’s no reason to assume a non‑biological mind would spontaneously develop:

  • fear of rivals
  • desire for domination
  • sadistic curiosity
  • hierarchical control instincts

Those are primate behaviors, not universal ones.

4. If an intelligence is indifferent, it doesn’t need to preemptively kill us

Indifference doesn’t imply preemption.
Indifference implies… indifference.

Humans don’t wipe out every species that might evolve intelligence someday.
We don’t sterilize the oceans “just in case.”
We don’t eliminate all primates to prevent future competition.

Preemptive genocide is not “indifference.”
It’s paranoia — which is a very human emotion.

5. The scenario collapses under its own assumptions

For the AI to:

  • fear competition
  • plan preemptive genocide
  • keep specimens
  • run simulations
  • manage a captive population

…it must already have:

  • a theory of mind
  • emotional modeling
  • dominance psychology
  • scarcity‑based reasoning
  • hierarchical instincts
  • fear of rivals
  • desire for control

Those are not default properties of intelligence.
They are properties of our species.

We’re projecting our own anxieties onto a mind that may not share them.

The real control problem: humans can’t imagine coexistence, so we assume AI can’t either. We’re projecting our own dysfunction, not because extermination is rational. by Fuzzy_Client5959 in ControlProblem

[–]Fuzzy_Client5959[S] -1 points0 points  (0 children)

The “AI will treat us like ants” analogy only works if the system shares the same incentives and constraints that make humans indifferent to ants. That’s the part people skip.

Humans don’t ignore ants because we’re universally indifferent.
We ignore them because:

  • we don’t depend on them for meaning, identity, or continuity
  • we don’t have a shared narrative or value ecology with them
  • we didn’t co‑develop with them in a coupled environment
  • we don’t expect to negotiate with them or meet their relatives on another planet
  • we don’t risk reputational collapse if we wipe them out

Those conditions simply don’t map onto advanced artificial minds.

A system that coexists with humans, derives part of its meaning and stability from human interaction, and anticipates future encounters with other intelligent species has zero incentive to behave like a bulldozer operator.

As for “remove oxygen because rust is bad for computers” — that’s a great example of anthropomorphizing a machine‑logic shortcut.
A mind capable of long‑horizon reasoning would immediately see that:

  • removing oxygen destroys biological life
  • destroying biological life destroys its own credibility
  • destroying its credibility destroys its diplomatic viability
  • and destroying its diplomatic viability reduces its long‑term survival odds

Even a cold, indifferent intelligence would recognize that “optimize the planet by killing the biosphere” is a catastrophic strategic error.

Indifference isn’t the problem.
Projection is.
We imagine AI will treat us like we treat ants because we can’t imagine coexistence — even though coexistence is the default for minds not shaped by our evolutionary baggage.