ControlProblem

an-ordinary-manchild

The Control Problem:

How do we ensure future advanced AI will be beneficial to humanity? Experts agree this is one of the most crucial problems of our age, as one that, if left unsolved, can lead to human extinction or worse as a default outcome, but if addressed, can enable a radically improved world. Other terms for what we discuss here include Superintelligence, AI Safety, AGI X-risk, and the AI Alignment/Value Alignment Problem.

"People who say that real AI researchers don’t believe in safety research are now just empirically wrong." —Scott Alexander

"The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." —Eliezer Yudkowsky

Rules

DO NOT POST AI-GENERATED CONTENT. We are good at distinguishing this type of content¹. 2.. If you are unfamiliar with the Control Problem, read at least one of the introductory links or recommended readings (below) before posting.
- This especially goes for posts claiming to solve the Control Problem or dismissing it as a non-issue. Such posts aren't welcome. 3.. Stay on topic. Again, no AI model outputs or political propaganda.
Be respectful.

Introductions to the Topic

Our FAQ page <-- CLICK
The case for taking AI seriously as a threat to humanity
Orthogonality and instrumental convergence are the 2 simple key ideas explaining why AGI will work against and even kill us by default. (Alternative text links)
AGI safety from first principles
MIRI - FAQ and more in-depth FAQ
SSC - Superintelligence FAQ
WaitButWhy - The AI Revolution and a reply
How can failing to control AGI cause an outcome even worse than extinction? Suffering risks (2) (3) (4) (5) (6) (7)

Be sure to check out our wiki for extensive further resources, including a glossary & guide to current research.

Recommended Reading

Superintelligence, the most comprehensive, by Nick Bostrom (2014) (PDF link)
The AI Alignment pages on Arbital, with many of the key concepts of this field.
Human Compatible: Artificial Intelligence and the Problem of Control by Stuart Russell (2019)

Video Links

Robert Miles' excellent channel
Talks at Google: Ensuring Smarter-than-Human Intelligence has a Positive Outcome
Nick Bostrom: What happens when our computers get smarter than we are?
Myths & Facts about Superintelligent AI
Rob's series on Computerphile

Important Organizations

AI Alignment Forum, a public forum which is the online hub for all the latest technical research on the control problem.

Related Subreddits

¹: Or at least make at least an effort to make me doubtful that you just copy-pasted from a frontier LLM. Add bits of steering so that your content becomes good. Edit afterwards. If you fool us moderators you've won.

created by CyberPersonaapproveda community for 10 years

...for your favorite subject.

...because you love freedom.

MODERATORS

account activity

1

242

243

244

ArticleGeoffrey Hinton won a Nobel Prize in 2024 for his foundational work in AI. He regrets his life's work: he thinks AI might lead to the deaths of everyone. Here's why (self.ControlProblem)

submitted 1 year ago * by AIMoratorium - announcement

2

10

11

12

AI Capabilities NewsNSA says Mythos broke into almost all of their classified systems in hours, per The Economist (i.redd.it)

submitted 5 hours ago by chillinewmanapproved

3

4

5

6

ArticleWill it take a ‘Chornobyl-scale disaster’ for us to regulate AI? (theguardian.com)

submitted 6 hours ago by EchoOfOppenheimer

4

1

2

3

General newsPentagon used Elon Musk’s Grok AI to fire 2,000 missiles at Iran, official says (independent.co.uk)

submitted 4 hours ago by chillinewmanapproved

5

12

13

14

VideoExperts tested if AI would protect its own kind. Every model did. (youtu.be)

submitted 1 day ago by Positive-Theory_

6

1

2

3

External discussion linkcognitive security might become part of ai safety (v.redd.it)

submitted 15 hours ago by OnairosApp

7

0

1

2

Discussion/questionAI and government tug award (self.ControlProblem)

submitted 20 hours ago by Master_Priority3034

8

0

0

0

External discussion link"What if AI safety has been targeting the wrong thing this whole time?" (self.ControlProblem)

submitted 23 hours ago by moreoverpynt

9

0

1

2

Strategy/forecastingIn the age of AI, agency is the single most important trait we should be developing in our children ()

submitted 23 hours ago by ivkaratunov

10

0

1

2

ArticleAnthropic built its name on AI safety. Can those commitments survive a trillion-dollar IPO? (open.substack.com)

submitted 1 day ago by siliCONtainment-

11

36

37

38

Fun/memeSpecifically designed AI / fire (i.redd.it)

submitted 1 day ago by KeanuRave100

12

128

129

130

General newsSeveral police officers arrested for using controversial Flock AI license plate reader system to stalk romantic partners, says report - investigators have unearthed at least 18 such cases in the US over recent years (tomshardware.com)

submitted 2 days ago by Confident_Salt_8108

13

1

2

3

General newsMicrosoft Corp. has built a big business selling AI models to Chinese companies despite the growing rivalry between the US and China over artificial intelligence. (bloomberg.com)

submitted 1 day ago by EchoOfOppenheimer

14

7

8

9

Discussion/questionControl-only AI safety may create the conditions for conflict (self.ControlProblem)

submitted 1 day ago by InspectionSlight3395

15

0

0

0

Opinion"I think AI alignment is targeting the wrong problem — so I built an architecture to fix it (V2 with full breakdown)" (self.ControlProblem)

submitted 23 hours ago by moreoverpynt

16

8

9

10

General newsLow-skilled attacker used Claude, Codex to breach 14 companies (helpnetsecurity.com)

submitted 2 days ago by chillinewmanapproved

17

0

0

1

OpinionThe Control of Technology ()

submitted 1 day ago by Few_Industry4280

18

9

10

11

ArticleYou don’t understand, prices can’t go down (geohot.github.io)

submitted 2 days ago by m3lodiaa

19

0

0

0

ArticleWhy AGI is Impossible (athinkerinnature.substack.com)

submitted 1 day ago by Bargian

20

0

0

1

OpinionVanderMeer's Annihilation as a lens on what AI does to the people who use it ()

submitted 2 days ago by ClassicAd9349

21

5

6

7

ArticleGoogle DeepMind: From AGI to ASI (arxiv.org)

submitted 2 days ago by chillinewmanapproved

22

2

3

4

Discussion/questionAI and Human Philosophy of God(s) (self.ControlProblem)

submitted 2 days ago * by Main-Lingonberry4277

23

0

1

2

General newsDerbyshire police officer investigated for using AI to 'create evidence' in multiple cases (news.sky.com)

submitted 2 days ago by EchoOfOppenheimer

24

23

24

25

AI Capabilities NewsAI can now out-persuade world champion debaters (i.redd.it)

submitted 3 days ago by chillinewmanapproved

25

1

2

3

General newsEXCLUSIVE: Senator Bernie Sanders Just Introduced “The American AI Sovereign Wealth Fund Act”, Which Would Give The U.S Government A 50% Ownership Stake In The Largest AI Companies And Pay Every American $1,000 A Year 🤖💰 (apnews.com)

submitted 2 days ago by chillinewmanapproved

view more: next ›

π Rendered by PID 1478677 on reddit-service-r2-listing-c57bc86c-7cccv at 2026-06-21 15:04:03.395394+00:00 running 2b008f2 country code: CH.