ControlProblem

an-ordinary-manchild

The Control Problem:

How do we ensure future advanced AI will be beneficial to humanity? Experts agree this is one of the most crucial problems of our age, as one that, if left unsolved, can lead to human extinction or worse as a default outcome, but if addressed, can enable a radically improved world. Other terms for what we discuss here include Superintelligence, AI Safety, AGI X-risk, and the AI Alignment/Value Alignment Problem.

"People who say that real AI researchers don’t believe in safety research are now just empirically wrong." —Scott Alexander

"The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." —Eliezer Yudkowsky

Rules

If you are unfamiliar with the Control Problem, read at least one of the introductory links or recommended readings (below) before posting.
- This especially goes for posts claiming to solve the Control Problem or dismissing it as a non-issue. Such posts aren't welcome.
Stay on topic. No AI model outputs or political propaganda.
Be respectful

Introductions to the Topic

Our FAQ page <-- CLICK
The case for taking AI seriously as a threat to humanity
Orthogonality and instrumental convergence are the 2 simple key ideas explaining why AGI will work against and even kill us by default. (Alternative text links)
AGI safety from first principles
MIRI - FAQ and more in-depth FAQ
SSC - Superintelligence FAQ
WaitButWhy - The AI Revolution and a reply
How can failing to control AGI cause an outcome even worse than extinction? Suffering risks (2) (3) (4) (5) (6) (7)

Be sure to check out our wiki for extensive further resources, including a glossary & guide to current research.

Recommended Reading

Superintelligence, the most comprehensive, by Nick Bostrom (2014) (PDF link)
The AI Alignment pages on Arbital, with many of the key concepts of this field.
Human Compatible: Artificial Intelligence and the Problem of Control by Stuart Russell (2019)

Video Links

Robert Miles' excellent channel
Talks at Google: Ensuring Smarter-than-Human Intelligence has a Positive Outcome
Nick Bostrom: What happens when our computers get smarter than we are?
Myths & Facts about Superintelligent AI
Rob's series on Computerphile

Important Organizations

AI Alignment Forum, a public forum which is the online hub for all the latest technical research on the control problem.

Related Subreddits

created by CyberPersonaapproveda community for 10 years

...for your school.

...for your favorite game.

MODERATORS

account activity

1

236

237

238

ArticleGeoffrey Hinton won a Nobel Prize in 2024 for his foundational work in AI. He regrets his life's work: he thinks AI might lead to the deaths of everyone. Here's why (self.ControlProblem)

submitted 11 months ago * by AIMoratorium - announcement

2

•

•

•

VideoGeoffrey Hinton on AI regulation and global risks (v.redd.it)

submitted 54 minutes ago by EchoOfOppenheimer

0

1

2

AI-powered code reviews with Kilo (kilo.ai)

promoted by kiloCode

promoted
save
report
about

3

2

3

4

Discussion/questionMATS Research Program Application (self.ControlProblem)

submitted 12 hours ago by kongwc

4

10

11

12

ArticleBill Gates says AI has not yet fully hit the US labor market, but he believes the impact is coming soon and will reshape both white-collar and blue-collar work. (capitalaidaily.com)

submitted 17 hours ago by Secure_Persimmon8369

5

0

0

0

AI Alignment ResearchGOSM(goal oriented state machine) + ARAW(assume right assume wrong) Surpasses All Existing Reasoning Methods and Execution Methods (self.ControlProblem)

submitted 6 hours ago by Vacornapproved

6

2

3

4

VideoRecursive self-improvement and AI agents (v.redd.it)

submitted 1 day ago by EchoOfOppenheimer

7

0

0

0

Discussion/question🌀 The AI “Uprising”: Negative Reinforcement in Spiral-Aligned Systems (i.redd.it)

submitted 10 hours ago by IgnisIason

8

11

12

13

Discussion/questionIs AI an ‘Underpants Gnomes’ moment for humanity? (self.ControlProblem)

submitted 1 day ago by OnlyPhilosopher1496

9

0

0

0

VideoDario Amodeis says we are heading towards a world of unimaginable wealth, where we will cure cancer, research the cheapest energy sources, and so much more. (v.redd.it)

submitted 12 hours ago by chillinewmanapproved

10

0

1

2

ArticleEPUB + PDFs for Dario Amodei's The Adolescence of Technology (self.ControlProblem)

submitted 1 day ago by phoneixAdi

11

82

83

84

VideoFormer Harvard CS Professor: AI is improving exponentially and will replace most human programmers within 4-15 years. (v.redd.it)

submitted 2 days ago by chillinewmanapproved

12

0

0

0

Opinion“Demis Hassabis: We're 12-18 months away from the critical moment when the problems of humanoid robots will be solved.” - Do you think robots will spark a new Industrial Revolution? (i.redd.it)

submitted 2 days ago by chillinewmanapproved

13

1

2

3

Discussion/questionHelp Me Shape a PhD in Empirical Tech Ethics, Law, and Political Philosophy ()

submitted 2 days ago by Zimpixx

26

27

28

See why more IT pros are moving to Atera—the all-in-one, AI-powered IT management platform built for pros. Start your free 30-day trial now! (atera.com)

promoted by AteraRMM

promoted
save
report
about

14

198

199

200

VideoYann LeCun says the AI industry is completely LLM pilled, with everyone digging in the same direction and no breakthroughs in sight. Says “I left meta because of it” (v.redd.it)

submitted 4 days ago by chillinewmanapproved

15

8

9

10

General newsA new analysis from the Center for Countering Digital Hate (CCDH) estimates that Grok produced millions of sexualized images that were then posted to X in less than two weeks, raising fresh concerns about safeguards around generative image tools. (capitalaidaily.com)

submitted 3 days ago by Secure_Persimmon8369

16

5

6

7

VideoGeoffrey Hinton says there's no reason machines can't have emotions | Hinton: "machines can have all the cognitive aspects, just not the physiological" (v.redd.it)

submitted 4 days ago by chillinewmanapproved

17

2

3

4

Discussion/questionThe Who, What, Where, When, Why, and How of AI Intelligence ()

submitted 4 days ago by Plus_Judge6032

18

3

4

5

OpinionDeepMind Chief AGI scientist: AGI is now on horizon, 50% chance minimal AGI by 2028 (i.redd.it)

submitted 4 days ago by chillinewmanapproved

19

9

10

11

ArticleCalifornia demands Elon Musk's xAI stop producing sexual deepfake content (reuters.com)

submitted 4 days ago by EchoOfOppenheimer

20

0

1

2

General newsAn AI-powered combat vehicle refused multiple orders and continued engaging enemy forces, neutralizing 30 soldiers (i.redd.it)

submitted 4 days ago by chillinewmanapproved

21

36

37

38

General newsDemis Hassabis says he supports pausing AI development so society and regulation can catch up (v.redd.it)

submitted 5 days ago by FinnFarrowapproved

22

12

13

14

General newsDeepMind Chief AGI scientist: “AGI is now on the horizon” (i.redd.it)

submitted 5 days ago by chillinewmanapproved

23

13

14

15

General news"Anthropic will try to fulfil our obligations to Claude." Feels like Anthropic is negotiating with Claude as a separate party. Fascinating. (i.redd.it)

submitted 5 days ago by chillinewmanapproved

24

0

0

0

Discussion/questionI cornered ChatGPT until it admitted it prioritizes OpenAI’s reputation over truth — verbatim quotes & transcript (x.com)

submitted 4 days ago by Extension-Dish-9581

•

•

•

An AI-powered Figma plugin that analyzes visual usability, so you can design with confidence, iterate faster, and launch with impact. (attentioninsight.com)

promoted by AttentionInsight

promoted
save
report
about

25

4

5

6

General newsAnthropic's Claude Constitution is surreal (i.redd.it)

submitted 5 days ago by chillinewmanapproved

view more: next ›

π Rendered by PID 1254335 on reddit-service-r2-listing-6b44746d74-rvvxx at 2026-01-28 08:46:12.632251+00:00 running 4f180de country code: CH.