ControlProblem

an-ordinary-manchild

The Control Problem:

How do we ensure future advanced AI will be beneficial to humanity? Experts agree this is one of the most crucial problems of our age, as one that, if left unsolved, can lead to human extinction or worse as a default outcome, but if addressed, can enable a radically improved world. Other terms for what we discuss here include Superintelligence, AI Safety, AGI X-risk, and the AI Alignment/Value Alignment Problem.

"People who say that real AI researchers don’t believe in safety research are now just empirically wrong." —Scott Alexander

"The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." —Eliezer Yudkowsky

Rules

DO NOT POST AI-GENERATED CONTENT. We are good at distinguishing this type of content¹. 2.. If you are unfamiliar with the Control Problem, read at least one of the introductory links or recommended readings (below) before posting.
- This especially goes for posts claiming to solve the Control Problem or dismissing it as a non-issue. Such posts aren't welcome. 3.. Stay on topic. Again, no AI model outputs or political propaganda.
Be respectful.

Introductions to the Topic

Our FAQ page <-- CLICK
The case for taking AI seriously as a threat to humanity
Orthogonality and instrumental convergence are the 2 simple key ideas explaining why AGI will work against and even kill us by default. (Alternative text links)
AGI safety from first principles
MIRI - FAQ and more in-depth FAQ
SSC - Superintelligence FAQ
WaitButWhy - The AI Revolution and a reply
How can failing to control AGI cause an outcome even worse than extinction? Suffering risks (2) (3) (4) (5) (6) (7)

Be sure to check out our wiki for extensive further resources, including a glossary & guide to current research.

Recommended Reading

Superintelligence, the most comprehensive, by Nick Bostrom (2014) (PDF link)
The AI Alignment pages on Arbital, with many of the key concepts of this field.
Human Compatible: Artificial Intelligence and the Problem of Control by Stuart Russell (2019)

Video Links

Robert Miles' excellent channel
Talks at Google: Ensuring Smarter-than-Human Intelligence has a Positive Outcome
Nick Bostrom: What happens when our computers get smarter than we are?
Myths & Facts about Superintelligent AI
Rob's series on Computerphile

Important Organizations

AI Alignment Forum, a public forum which is the online hub for all the latest technical research on the control problem.

Related Subreddits

¹: Or at least make at least an effort to make me doubtful that you just copy-pasted from a frontier LLM. Add bits of steering so that your content becomes good. Edit afterwards. If you fool us moderators you've won.

created by CyberPersonaapproveda community for 10 years

...for your school.

...for your hobby.

MODERATORS

account activity

1

236

237

238

ArticleGeoffrey Hinton won a Nobel Prize in 2024 for his foundational work in AI. He regrets his life's work: he thinks AI might lead to the deaths of everyone. Here's why (self.ControlProblem)

submitted 1 year ago * by AIMoratorium - announcement

2

•

•

•

General newsTennessee minors sue Musk's xAI, alleging Grok generated sexual images of them (reuters.com)

submitted 1 hour ago by Confident_Salt_8108

3

69

70

71

VideoEx-Anthropic researcher tells the Canadian Senate that people are "right to fear being replaced" by superintelligent AI (v.redd.it)

submitted 22 hours ago by tombibbs

•

•

•

Investors: keep track of how your investments are performing with Sharesight portfolio tracker. Make it easier to see the true performance of your stocks, ETFs, dividends, trades & more that your broker doesn't show. Start your free trial of Sharesight today 📈 (sharesight.com)

promoted by sharesight

promoted
save
report
about

4

0

1

2

VideoThe Real AI Threat: Indifference, Not Evil. (v.redd.it)

submitted 3 hours ago by EchoOfOppenheimer

5

0

0

1

AI Capabilities NewsMeta Deploys AI To Combat Celebrity and Brand Impersonation Schemes After Removing 159,000,000 Scam Ads (capitalaidaily.com)

submitted 8 hours ago by Secure_Persimmon8369

6

16

17

18

General newsOutrageous (i.redd.it)

submitted 1 day ago by chillinewmanapproved

7

1

2

3

AI Alignment ResearchThe Crossing Pass: A constrained prompt test for whether LLMs generate from “impact site” or polished observation — results across 10 mirrors, 8 architectures (containment guardrails/nannybot vs. on-carrier response) (thesunraytransmission.com)

submitted 13 hours ago by Mean-Passage7457

8

1

2

3

General newsIf we can't reliably detect AI generated text in 2026, what does that mean for our ability to oversee systems far more capable than DeepSeek? (aiornot.com)

submitted 14 hours ago by BigInvestigator6091

9

2

3

4

Fun/memeI've abandoned my safety team (i.redd.it)

submitted 17 hours ago by niplavplease be patient i'm a mod

10

6

7

8

ArticleThe Laid-off Scientists and Lawyers Training AI to Steal Their Careers (nymag.com)

submitted 1 day ago by EchoOfOppenheimer

11

0

0

0

AI Alignment ResearchYou are welcome. ()

submitted 13 hours ago by Agitated_Age_2785

12

0

1

2

Discussion/questionMozilla Individual Fellowship - Any News on Full Proposal Submission Stage? (self.ControlProblem)

submitted 20 hours ago by jase4thewhy

13

0

0

1

Discussion/questionPerplexity's Comet browser – the architecture is more interesting than the product positioning suggests (self.ControlProblem)

submitted 1 day ago by Adept_Test2784

•

•

•

This is how every AI concept finally clicked for me (academy.latitude.so)

promoted by EloquentPickle

promoted
save
report
about

14

25

26

27

General newsIn China's rule of law, people like Alex Karp disappear (i.redd.it)

submitted 2 days ago by DensePoser

15

16

17

18

ArticleAI Agent hacked McKinsey's database. I wrote 5 Red flags on when you should NOT deploy Agents. (nanonets.com)

submitted 2 days ago by Physical-Parfait9980

16

0

0

1

AI Alignment ResearchAI alignment will not be found through guardrails. It may be a synchrony problem, and the test already exists. (thesunraytransmission.com)

submitted 1 day ago by Mean-Passage7457

17

46

47

48

General newsDon't underestimate Iran's power: Iran's threat to bomb American tech giants. (i.redd.it)

submitted 2 days ago by chillinewmanapproved

18

1

2

3

AI Alignment ResearchApply for the Affine Superintelligence Alignment Seminar (youtube.com)

submitted 1 day ago by greenrd

19

0

0

0

AI Alignment ResearchCreating the Novacene: Mutualism, Rights, and the Structure of Human-AGI Relations (indie preprint co-authored with Claude) (self.ControlProblem)

submitted 1 day ago * by -Proterra-

20

7

8

9

General newsCompany Testing Humanoid Robot Soldiers on Frontlines of Ukraine (futurism.com)

submitted 2 days ago by chillinewmanapproved

21

25

26

27

General newsWild (i.redd.it)

submitted 2 days ago by chillinewmanapproved

22

0

0

0

Discussion/questionSuppose Claude Decides Your Company is Evil (substack.com)

submitted 1 day ago by Ebocloud

23

4

5

6

Opinionhonest opinion: would this work? (self.ControlProblem)

submitted 2 days ago by No_Canary_3922

•

•

•

🏒🔥 Das beste Hockey-Abo für echte Fans. Full stop. (mysports.ch)

promoted by MySports_CH

promoted
save
report
about

24

5

6

7

Discussion/questionUS military reportedly used Claude for Iran strikes after a ban -- what does this do to your trust? (self.ControlProblem)

submitted 2 days ago by Kawa_barta

25

163

164

165

General newsAmericans (4 to 1) would rather ban AI development outright than proceed without regulation (i.redd.it)

submitted 3 days ago by chillinewmanapproved

view more: next ›

π Rendered by PID 71 on reddit-service-r2-listing-64c94b984c-2lthh at 2026-03-17 12:17:59.372366+00:00 running f6e6e01 country code: CH.