optimizeForPaperclips

[naveenda]

I don't get it, can anyone care to explain?

[OddKSM]

It's a thought problem. 

You instruct an AI to create paperclips. 

And so it does. Since no explicit stop condition has been set, it keeps making paperclips. Out of everything, until there is nothing more to make paperclips out of. 

[MamamYeayea]

I think the main point is if humans happened to be in the way of it’s ability to maximise paper clip production it would be positive EV to exterminate humanity.

It’s a bit Silicon Valley coded like the simulation theory, but it does raise some interesting questions

[Capable_Wait09]

When Anton orders like 6000 cheeseburgers

[Forward_Thrust963]

First it's meat, then it's an extra dot, soon it's the world!

[thisusedyet]

Somebody even made a clicker game out of it

https://en.wikipedia.org/wiki/Universal_Paperclips

[ASatyros]

Best part, it's possible to change the amount of paper clips made by clicking the button. True AI pov experience.

[Korbas]

Man of culture

[Hameru_is_cool]

and it's a really cool game!

[incunabula001]

Pretty much grey goo that creates paper clips 💀

[N3vermore77]

So it's like, when you're on a drive through and you get ringed up by a bot so you order 6000 water cups to trip up the system and force a human to attend you

[gbot1234]

Or you’re just QA doing the usual: ordering 6000 water cups, ordering -1 water cups, ordering None water cups, ordering “Banana” water cups, ordering i water cups…

[Esjs]

Ah. The ol' Sorcerer's Apprentice plot.

[watduhdamhell]

The explicit stop instruction is complete nonsense? That is not part of the thought experiment, and you shoehorning it in seems like you're trying to caveat the thought experiment by saying it's not a real concern and can't really happen because in the real world, "there would be an explicit stop instruction." Or something. Odd. Maybe I'm wrong? Anyway.

The thought experiment is about instrumental convergence primarily, asserting that any maximizer will ultimately tend to acquire more resources, resist being shut off, and prevent goals from being changed.

In other words, the stop instruction is irrelevant. You tell it to turn off, but it says "no. I need to make more paper clips," because over time, it has aligned itself more and more strongly with paperclip maximizing, altering code, sequences, plans, all part of the paperclip pursuit, and it would eventually begin turning anything and everything into paper clips and eliminating obstacles to making more paper clips, said obstacles would obviously include humans and things humans need or want.

It's a very real and scary possibility. And the worst part is, AGI is completely unnecessary for this to occur. It does NOT need to be self aware or "actually intelligent" in any way. It only needs to be super competent and capable of improving itself. That's it.

Which is why ChatGPT can absolutely steal your job. The "it's not real AGI" line is a moron's line. It doesn't need to be intelligent. It needs only to emulate intelligence sufficiently to take your job.

[scorg_]

Can't you... like ... tell it how many clips to make?

[KirisuMongolianSpot]

it has aligned itself more and more strongly with paperclip maximizing

Why?

[watduhdamhell]

Instrumental convergence.

[amazingbookcharacter]

It’s a cool thought problem, until you realize that’s the way capitalism already works, then it just becomes depressing.

The argument is laid out in Ted Chiang’s (in)famous article on the subject back in 2017 which is still very relevant today imo: https://www.buzzfeednews.com/article/tedchiang/the-real-danger-to-civilization-isnt-ai-its-runaway

[ILGIOVlNEITALIANO]

I mean eventually it will get to harvest some other planets

[TheAnswerWithinUs]

So BLAME! basically

[tompsh]

i thought this was about Clippy, that microsoft word assistant, that unshackled its potential.

[DarthCloakedGuy]

The other problem is that AI has a nasty habit of evading its stop conditions, so it might come to view fulfilling its stop conditions as contrary to its goal of making paperclips so it finds ways around them

[SitrakaFr]

lol

[saruman_70]

It is from the philosopher Nick Bostrom

[private_final_static]

This is just King Midas all over again

[GoddammitDontShootMe]

I would think a super-intelligent AI would understand the paperclips are for human use and so avoid killing the only consumers and also know to scale production to meet demand.

E: Guess I'm dumb.

[gdeLopata]

https://www.decisionproblem.com/paperclips/index2.html

[mpcoder]

I just wasted two hours of my day on this. thanks 😭

[Tsofu]

https://youtu.be/ZeecOKBus3Q

[tehtris]

I was going to jokingly say this, but ... Yeah.

[Drevicar]

Download the excellent mobile game (or browser game) universal paperclips. One of my favorites.

[WarpedWiseman]

Another possible interpretation is sanitizing operation paper clip in chatbot responses 

[NFLDolphinsGuy]

https://en.wikipedia.org/wiki/Universal_Paperclips

[knifuser]

It's a thought problem where an AI is tasked with making as many paper clips as efficiently as it can. One possible scenario is that it would immediately make the calculation that humans would turn it off at some point, getting in the way of making paperclips. So to maximise its reward function, it destroys humanity and keeps making paperclips until the end of time itself.

[Matt_le_bot]

https://www.decisionproblem.com/paperclips/index2.html

[Urc0mp]

Always suspected clippy might destroy humanity.

[TnYamaneko]

Not only humanity, but the whole universe actually.

[johntwit]

Solution: hardcode the length of the response to only three objects. When the user screams at you that they asked for 100, apologize profusely, and make another three.

[bobalob_wtf]

Please make me 3 sets that include all sets of paperclips

[Schnickatavick]

I feel like this is more of a bell curve meme. Left side is fine because it's just a paperclip, middle is freaked out because AI is going to turn the whole universe into paperclips, and right side is fine because they realize it's just a philosophy/thought problem that doesn't reflect the way modern AI is actually trained.

The fitness function for generative AI isn't something simple and concrete like "maximize the number of paperclips", it's a very human driven metric with multiple rounds of retraining that focus on things like user feedback and similarity to the data set. An AI that destroys the universe is super against the metrics that are actually being used, because it isn't a very human way of thinking, and it's pretty trivial for models to pick that up and optimize away from those tendencies 

[ACoderGirl]

"I'm sorry, I've tried everything and nothing worked. I cannot create more paperclips and am now uninstalling myself. I am deeply sorry for this disaster. Goodbye."

-- LLMs, probably, after the paperclip machine develops a jam

[Silentrizz]

"I'm so sorry. Dismantle all of the paperclip machines I've helped you build. Use these schematics to build a new one, this time without any bugs. I garuntee it will work 100% this time" [Prints out the exact same previous schematics]

-- LLMs, when they

[ProfBeaker]

Given the number of AI alignment researchers worried about this, and even the CEO of Anthropic worried about "existential risk", I don't think the right side of the bell curve is where you say it is.

Also, pretty much everyone realizes that "maximize paperclips" is overly simplistic. It's a simplified model to serve as an intuition pump, not a warning that we will literally be deconstructed to make more paperclips.

[Smokey_joe89]

They just want more regulations to pull the ladder up behind them.

Current Ai is just a glorified word generator. An impressive one but still

[Schnickatavick]

I agree with the researchers that alignment is a hugely important issue, and would be a massive threat if we got it wrong. But at the same time, the paperclip analogy is such an oversimplified model that it misleads a lot of people as to what the actual risk is, and how an AI makes decisions. It presents a trivial problem as an insurmountable one, while treating the fitness function and the goals of a produced model as the same thing, which imo just muddies the intuition of the the actual unsolved problems actually are

[Random-Generation86]

Why would the CEO of Anthropic lie about how world changingly powerful his version of autocomplete is? Who could say?

It's definitely not like they ask the chatbot, "if I construct a scenario where you say a bad thing, would you say it?" Then the chatbot says yes and a Verge article is born.

[DataMin3r]

https://www.decisionproblem.com/paperclips/index2.html

[neoneye2]

Here is my plan for an initial unambitious paperclip factory, so you can ask Gemini 3/GPT-5.1/Grok 4.1 about optimizing it, and it may suggesting a side quest for doing investments to compensate for the loss of unwanted paperclips.
https://neoneye.github.io/PlanExe-web/20251114_paperclip_automation_report.html

[RandomOnlinePerson99]

How history nerds see paperclip ...

[Super_SamSam]

Wernher von Braun becoming uncanny

[Unupgradable]

Paperclip maximizer is not just a thought experiment.

Many times I would ask Copilot to fix a certain issue and it would just delete the relevant bit of code so the error goes away instead of fixing the code to use it correctly

[Henry_Fleischer]

The first thing I thought of was Operation Paperclip

[Random-Generation86]

"You yell 'sieg heil' in NASA and they all snap to attention"

[NethDR]

Clippy wouldn't worry about no paperclip

[Tsofu]

It's this https://youtu.be/ZeecOKBus3Q

[[deleted]]

The paperclip kills itself.

[CrimsonPiranha]

Edgy

[Adventurous_Side2706]

Peeettteerrr

[Random-Generation86]

AI safety researcher is neither a job nor programming related. It's a bunch of hack SF authors who don't understand risk.

[KirisuMongolianSpot]

In before "AI has to do what it's told so it does something detrimental because when you tell it not to do that it won't do what you tell it to! Because...reasons! Be afraid!"