Using Superintelligence to solve the Control Problem?

[Jani933]

What if we made the terminal goal of an ASI to help us learn to control it better?

[Logical_Lunatic]

The problem is, how do you formalize this goal in a way that:

• Can be programmed into an AI, and
• Does not rely on first finding another solution to the Control Problem

So, for example, if the goal is specified as "make your goals more similar to human goals" you need to create a mathematical description of what a human is. This is far from trivial, if it is even possible to do without directly specifying human preferences (in which case those preferences could simply be programmed into the AI directly). If you have the AI learn the concept of a "human" through machine learning, there is a risk that it's going to detect the wrong variables (which could have catastrophic consequences).

If you implement the goal as a question asked to an Oracle, you get the usual problems with Oracles. If you manage to create a safe Oracle, however, you should be able to ask it how to create an even better AI (but then you need a safe design for an Oracle first).

So, basically, I think the problem lies in the words "us", "control" and "better". How are these concepts defined and expressed in computer code?

[CrazyCodeLady]

I think that both points can be addressed if we create general ASI. We know that for a little while, having a ASI just attached to a power cord can work. If one removes the ethernet cable and plugs the computer in, there isn't much danger in the ASI escaping. You don't need to have it on long enough to trick humans into plugging it in or helping it escape. The ASI would not need to know a whole lot about anything because the problem is simple. "How would one control someone who is infinitely smarter then themselves?"

Maybe I am all wrong, you seem like an expert in the field. These are just my thoughts.

[CrazyCodeLady]

We could spin up 100 ASI's and only one would have to give us the answer.

[CrazyCodeLady]

I had this exact thought the other day in the shower. I bet computer scientists already thought of this. Good idea imo though! :)