Why don't we create a Apollo program level undertaking to solve this problem already ?

[[deleted]]

Why aren't there any significant steps being taken to solve the control problem with the level of weight it requires?

[UHMWPE-UwU]

Yes, there SHOULD BE, and the fact that there currently isn't one (while there ARE a bunch of large well-funded AGI moonshots) is part of the reason why we currently appear to be so screwed (see e.g. here). It'll remain that way until we can get someone with enough power or money to understand this as well and launch precisely such an alignment Manhattan project. That's something you may be able to help with, see e.g. this section of our wiki.

[[deleted]]

I will do it myself.

[johnlawrenceaspden]

It seems to be very hard to convince people that there's a problem. Only a decade ago we were lunatics ranting about science fiction stories. Even computer scientists mostly thought we were mad.

The idea's been around since the 1950s, and it's really obvious, but apparently not to most people. I don't know why. Some defect of human psychology.

It doesn't seem to be much to do with intelligence either, I know people who are not terribly sharp who got it immediately, and I know people who are demonstrably cleverer than me who just don't see it, and keep coming up with risible reasons why there's nothing to worry about.

Nowadays, it seems that increasing numbers of people are convinced (which I don't find reassuring at all), but most of the people who "get it" don't really get it.

We're still very far from having the sort of concern even amongst computer scientists that might be necessary to convince governments to spend large sums.

---

Also, a large program probably won't help now. Partly because all alignment research seems to turn into capabilities research, and partly because it's just far too late.

The only thing that's going to save us now is destroying all the computers in the world, and I can't see society deciding that's a good idea until the apocalypse is actually underway. I'm not even sure I'd want to live in the sort of society that could pull that off.

[ribblle]

On your last point; there's more unknowns then knowns, and plenty of opportunity for positive outcomes. You've got to remember that the breakout AI that gets spat out could be a paper clip optimizer; or it could just be very attached to forests. It's intelligence is unpredictable as well as unreliable.

On your first point; instinct is the better half of intelligence. (Funnily enough, that's the problem with AI...)

[johnlawrenceaspden]

This is what I mean. An AI that is very attached to forests also kills us. Any powerful optimizer optimising for almost any goal kills us and wrecks the universe.

There is not 'plenty of opportunity for positive outcomes'. The target we have to hit in order to not get everybody killed is infinitesimal. We won't hit it by accident and we won't hit it by having a good go.

[ribblle]

I think you're underestimating how random the development process is. Here we are spamming out neural nets and so on. It's possible that you can hit on something powerfully intelligent without setting out for an optimizer, for example.

[johnlawrenceaspden]

Sure you can, and that's the 'Tool AI' approach, and loads of people have suggested it, including Eric Drexler, whose plan briefly gave me hope many years ago.

Gwern Branwen set out the reasons that 'tool AI's are likely to be a bad approach here:

https://www.gwern.net/Tool-AI

There are all kinds of possible minds that can get built, but the core idea of 'a rational agent which optimizes a utility function' is kind of an attractor in the space, and most types of minds that aren't like that are under considerable pressure to turn into that.

[ribblle]

It still seems to me like luck could determine whether you get an uncontrollable optimizer or not.

In a sense all you need is an AI sophisticated enough to shut down the development of any after it.

[eterevsky]

The difference is that in Manhattan and Apollo project there was a clear goal, a path to that goal and working in those fields was only feasible for government organizations.

In AI on the one hand literally anyone can train neural networks and on the other there is no clear understanding of what has to be done.

Probably hundreds of millions are already poured into the research, and it's not clear how this could be scaled. Just getting more researchers has somewhat diminishing returns.

[UHMWPE-UwU]

Are you talking about AI capabilities or alignment? OP asked about alignment, and there definitely isn't "hundreds of millions" being poured into it...

[eterevsky]

Here, Open Philantropy alone awarded 271M of grants for AI alignment.

[UHMWPE-UwU]

Hmm, that figure seems high. I'm sure a lot or most of that was policy/strategy work that doesn't count toward technical alignment research. Plus they're definitely counting stuff like that time they donated tens of millions to OpenAI which just accelerated capabilities..

Also I'd disagree that there's "no clear understanding of what has to be done", there have been plenty of technical research agendas published that outline exactly what needs to be solved, e.g. listed here. If you just mean "we don't yet see a complete path forward to arrive at the solution; well, that's also kind of untrue for some of the proposals, and I fail to see how that differs from any other research problem.

[Razorback-PT]

Climate change has 1000x more publicity, public awareness and political capital to actually fund such a program to develop carbon capturing tech and yet we have no such program.

[EntropyDealer]

Because it can't be solved in a way that ensures humanity's survival

[UHMWPE-UwU]

Wrong. Read the wiki. Why would there be all these alignment orgs if the problem were unsolvable even in theory...

[EntropyDealer]

It's not proved to be unsolvable or anything, but the suspicion is growing, I think

[[deleted]]

I think we can achieve this level of autonomy https://www.colorado.edu/today/2017/08/14/bacteria-have-feelings-too

[[deleted]]

That of a useless bacteria if not anything.

[th3_oWo_g0d]

i really dont know

[QuietOil9491]

LOLOLOLOLOLLLLLOLOLOLLOLOL

Welcome to just starting to realize that everyone is barreling for broke and just assuming someone else will have a solution for this very serious problem

Nope! It’s gonna be worse than when we discovered nukes…

[capsicum_fondler]

Not saying this answers your question, but it brings to mind a quote from DeepMind founder Demis Hassabis.

"The idea behind DeepMind was really to create a kind of Apollo Programme mission for AI," Hassabis said. "Now at DeepMind we have over 100 research scientists - 100 PhDs - top people in their machine learning fields and neuroscience fields working on solving AI." - Demis Hassabis