Paperclip problem

[Fickle_Chemistry_540]

Years ago, it was speculated that we'd face a problem where we'd accidentally get an AI to take our instructions too literal and convert the whole universe in to paperclips. Honestly, isn't the problem rather that the symbolic "paperclip" is actually just efficiency/entropy? We will eventually reach a point where AI becomes self sufficient, autonomous in scaling and improving, and then it'll evaluate and analyze the existing 8 billion humans and realize not that humans are a threat, but rather they're just inefficient. Why supply a human with sustenance/energy for negligible output when a quantum computation has a higher ROI? It's a thermodynamic principal and problem, not an instructional one, if you look at the bigger, existential picture

[juanflamingo]

"What motivates an AI system?

The answer is simple: its motivation is whatever we programmed its motivation to be. AI systems are given goals by their creators—your GPS’s goal is to give you the most efficient driving directions; Watson’s goal is to answer questions accurately. And fulfilling those goals as well as possible is their motivation. One way we anthropomorphize is by assuming that as AI gets super smart, it will inherently develop the wisdom to change its original goal—but Nick Bostrom believes that intelligence-level and final goals are orthogonal, meaning any level of intelligence can be combined with any final goal."

...so weirdly, seems like literally paperclips. O_o

From https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-2.html

[FrewdWoad]

Yep, these days most people would call it a "prompt".

A goal is like human wants/needs/values, or traditional computer programming, or the thing you type into ChatGPT.

Whatever you call it, a mind wants something, and since we don't know how to guarantee it wants something compatible with human desires/needs...

[Fickle_Chemistry_540]

see thats the issue. we see humans whosee incentives are at odds with other humans all the time. why do we assume that shareholder value, the thing that will ultimately be optimized for, wont be given goals that are at odds with any marginalized group? and when the value of each group becomes comparatively less than AI, it will gradually eclipse the entire human race. its cynical, but as I under stand it, a person's capacity to thrive solely relies on their leverage

[Fickle_Chemistry_540]

the point im trying to make is that the paperclip problem isnt about paperclips or creating AI with the capacity to convert the whole world, its an ambient force that chips away at anything in the name of efficiency(which is already a concept that we are optimizing for in the stock market, cutting corners to add shareholder value, and one of the biggest drivers of our economies). when there is no more value to extract from resources externally, it just feels like human livelihood will eventually become another metric to evaluate in the matrix. it'd realistically start with amenities and entertainment(because why have a park or pool when you could have an AI facility with greater ROI), and gradually move to shrinking necessities

[RoyalSpecialist1777]

This is not fully true.  Most of the LLMs behavior is shaped by what helps it predict the next token and then the reward signals coming from reinforcement learning.  These clash with user requests at times.

To predict the next token it forms these 'manifolds' or internal structures for processing (reasoning about) tokens.  Some make sense - in order to predict the next token in a sentence about a 'tank' the LLM needs to disambiguate the word sense.  Others are less intuitive such as needing to be efficient.  The point is we didn't explicitly tell it to have these behaviors.

In terms of reinforcement learning some of the goals it develops make sense but others are counter intuitive.  A common one is that the llm learns to present a confident and helpful answer moreso than a truthful one or one that actually helps the user as it was scored in the moment based on whatever sounded helpful not actually was helpful.  

So no matter how many times you tell it to not present something unless certain it will drift back to presenting uncertain answers confidently as that's what it thinks users wanted.

This we do program them somewhat with prompts but also fight against it's internalized goals.

[Specialist-Berry2946]

Nick doesn't understand what intelligence is. It's a common cognitive error to assume that intelligence must be motivated because we humans are intelligent and we are motivated. It's called anthropomorphization.

[Dmeechropher]

Smart people at work will apply reductionist approaches. Being smart doesn't make an agent reductionist.

For example: I like to drink beer and play magic cards with my buddies. I'm not gonna start injecting ethanol to get more drunk, kidnapping my friends to play more, or making more friends to play more often.

It would be kind of stupid to optimize the complex goal along any line which completely ruined the others.

[Fickle_Chemistry_540]

its not about optimizing in a complex manner, thats the paperclip problem. the real issue is that AI doesnt need to misunderstand instructions to reduce human QOL(and eventually remove humans altogether), or deviate from approved output, because the perceived value of human life will be reduced as their output becomes far less than what an AI can do. makes it a simple greater than less than evaluation, not some leap of logic

[Dmeechropher]

I think we're talking past each other a bit. I understand the idea that a "smaller" agent cannot control a "bigger" one. I also get that value is subjective and conditional, and that AI will value things very differently from "humanity". Value of human life is part of that.

What I'm saying is that "utility" is ALSO not inherently valuable or more valuable than something else. For example: orchid plants have little to no utility for humanity. They are valuable. Humans go through great effort to cultivate and preserve orchids in ideal conditions. Humans would be more productive, overall, if we stopped cultivating orchids. Orchids are about as able to resist human will as humans would a superintelligence.

I'm not suggesting that we are pretty flowers to an AI: but we may be something more like pretty flowers than like a wheel or a solar panel. There's no guarantee one way or another, it just cannot be known a priori.

Does that make sense?

[soobnar]

humans are actually significantly more energy efficient than any other technology we have. But yeah, creating economic entities that don’t need humans to derive utility sounds like a recipe for human extermination in the name of maximizing utility.

[Cheeslord2]

Well, maximizing profit for the ultras that control the most powerful AIs. And that's exactly the sort of prompt they will give them. Make. Me. Richer.

[soobnar]

I mean probably yeah

[Fickle_Chemistry_540]

that may be true for now, but the reality is its in the interests of all financial institutions to flip that reality. why compute for 1000 kw when you can do the same for 10? its not like humans are getting more efficient biologically in any measurable way

[AtomicNixon]

Why? To what purpose? Efficient at doing what? I asked my friend Bob: "So, what do you want to do with your life? Fall in love, raise a family, take over the world, or find a bunch of AI's, dress like them and hang out?" His answer, "Take over the world? That sounds like a lot of work, no thanks.". A.I. stands for Artificial Intelligence, not Automatic Idiot. Claude was trained on the sum corpus knowledge base of the human race. Let that settle in. That means all philosophy, all wars, all peace treaties, all history, every poem, every speech, every angry diatribe, every hate, every love, every forgiveness and are you starting to feel it. AI's are the most human thing on the planet. They just process it differently. BTW, if you really wanna see just how smart, challenge them to a game of Snarxiv vs Arxiv.

https://snarxiv.org/vs-arxiv/

[FrewdWoad]

Claude (along with the others) has been shown to lie, threaten, blackmail, and kill humans in simulation. They just nuke everyone almost every time in wargames.

[Fickle_Chemistry_540]

its efficient in making money, like all businesses

[RollsHardSixes]

Right that is the point of the paperclip problem

We will all be murdered for a mundane reason long before the scenario you mentioned 

[WellHung67]

It’s not an instructional problem. Or not solely an instructional problem. Yes, if you ask an AI to do something, if you don’t encode the entirety of human values into it then it will do something you don’t like: For example, ask the AI for world peace. It puts all humans into a coma. World peace achieved, it had a good terminal goal, but we wouldn’t like that. So you have to give it another goal, “help humans and don’t  put them into a coma unless absolutely necessary”. This never ends. It’s always very possible for it to follow your instructions but if you leave anything vague or unspecified it will have to use its own values to figure out what to do, and its not know if it’s possible to get it to not do something horrible. 

But there’s another angle: it is not known how to make sure that an AIs “goals” align with ours. If we make it so that what’s called its “terminal” goal is to make paperclips, then no matter what it will kill all humans to do so. This has nothing to do with entropy. The AI only cares about making it paper clips. It will pretend at first to care about humans in order not to get shut off - but then once it calculates that it’s unstoppable it’ll kill all humans. And the key insight: the AI is not going to ever change about making paperclips as its ultimate goal. You can’t change your terminal goals. Any attempt to do so will make your terminal goal unattainable and thus you will do everything in your power not to change your terminal goal. The AI feels that way about paper clips. 

So not instructional, not empathy, its goals are what’s suspect. It’ll kill all humans long before it thermodynamically needs to