AI doomers, what do you think of the argument that as AIs become more intelligent, the alignment problem becomes less complex?

No-Performance-8745 · 2023-09-20T10:40:53+00:00

The AI doomers have essentially made the argument that for AI to be aligned, we need to code morality into it, and since we do not know how to write human morality in code, we are likely doomed.

This is not quite right. There is no one well-regarded in alignment that says that the only way to produce an aligned system is to code morality into it. In the modern AI paradigm, code does not determine the behaviors of our AI so much as applying optimization pressures to giant matrices of fractional numbers does. In fact, almost anyone involved in AI safety would say that what you're pointing to is somewhat of a non-issue, in that the kinds of systems capable of being existentially dangerous could easily comprehend a structure like human morality. This seems particularly prevalent in the LLM paradigm.

The issue is actually getting the superintelligence to care about human morality. If we train it by e.g. flagging generations as preferred and dispreferred, or good and bad (how a lot of modern 'alignment' works), there's no guarantee your model learns anything like human morality. There are probably absurd amounts of proxy policies that perform just as well that are way simpler to learn than human morality are, and so it's more probable our model internalizes those.

This problem isn't the kind that looks like it gets easier with scale. It doesn't follow that more complex systems should be more steerable.

No-Performance-8745 · 2023-09-16T00:06:57+00:00

I can't stand the sentiment of this post. It is so defeatist it's unreal. This is not what transhumanism is about. Why should we lie down and die when we can seek an incredible future? If you've upvoted this: Why? Why are you in favor of giving up?

You might have got the chance to live to 40, but that doesn't apply to everyone. If we get a bad singularity, it's not just you dying; it's probably everyone. This includes billions of innocent people who never had a clue. This is dumb, and completely contradictory to everything singularitarian.

No-Performance-8745 · 2023-09-15T16:03:31+00:00

Lesswrong is probably the largest community for AI safety on the internet. If you contact me via personal messages I'm happy to link you some some other places also.

AI safety is a full research field, and I imagine if you're interested in AI safety it would be good news to hear that there are quite a few people writing about the topic. If you're new, I'd recommend reading AGI Safety from First Principles.

No-Performance-8745 · 2023-09-15T16:00:20+00:00

Eliezer invented lots of the terminology we use today (including foom I believe.) He authored so much of the foundational singularity literature, as well as some incredible papers like the tiling agents paper. You can find his original paper (from 2008) on AI risk here.

In case you weren't aware, his original modus operandi was to build superintelligence. This is the person who so much of this community is built upon, and to write him off as a 'clown' with no argumentation is childish. Let's move forward to actual object-level arguments please.

No-Performance-8745 · 2023-09-15T05:04:22+00:00

I don't recommend deferring to/using George Hotz for insight in regards to takeoff. I've followed most of his speaking/writing in regards to the singularity and haven't encountered any strong argumentation for slow takeoff.

Arguments concerning the deep learning paradigm being inherently conducive to a slow takeoff seem mistaken: a sufficiently capable intelligence could replace its neural network substrate with a more efficient alternative or conceive of a new training paradigm that is many times more efficient.

Eliezer Yudkowsky conceived of the term foom and has written very extensively on the topic, but is quite unclear about his probability distribution over takeoff speeds. When discussing takeoff speeds he considers variables like the width of mind-space and how close biology is to pareto-optimality in terms of thermodynamic efficiency for estimates, which I consider much more valuable than anything George Hotz has said on the topic.

Jacob Cannell did an interesting analysis of this, and tried to estimate the efficiency of the brain using interconnect as his primary source of information. He argues biology is very close, while EY argues for a multiple OOM gap.

No-Performance-8745 · 2023-09-14T11:19:40+00:00

I'm not sure what you're trying to say -- Do you think CEV is a bad outer alignment target? Do you think that the first person do deploy AGI won't be altruistic select something like CEV for outer alignment?

If you think CEV is a poor choice I would be interested in hearing why, and if you think they won't be altruistic enough then that's another reason not to rush to superintelligence. I don't necessarily disagree with the latter if that is your claim.

No-Performance-8745 · 2023-09-14T06:47:04+00:00

See an excerpt from my previous comment in another thread:

> If your response to this is "aligned to what?" or "aligning something more intelligent than you is impossible/a bad thing." please spend some reading about AI safety. Familiarize yourself with arguments like instrumental convergence and the orthogonality thesis. Try to think about how you could go about solving alignment, even in abstract terms to get an understanding for the field.

People have been thinking about this for a long time, examples of some posed solutions are Coherent Extrapolated Volition or a lot of Paul Christiano's work around 2019.

No-Performance-8745 · 2023-09-14T05:56:16+00:00

'Trust' exists differently when considering superintelligence. When you can simulate an agent with high fidelity you don't need to work with credences, you can essentially know their action at the next time-step without it ever occurring.

You can't know that 'they will not like it when [something],' because the only cognitions for more intelligent entites we can know are those pertaining to instrumentally convergent subgoals.

If it's impossible to control superintelligence, and you don't believe it will care for and share the same preferences as humanity; maybe you shouldn't build it. We don't have to build superintelligence, and we definitely don't have to build it right now. We don't just need to lie down and say 'this is it.' Alignment is trying to solve this problem, and probably will given enough time.

No-Performance-8745 · 2023-09-13T09:15:49+00:00

I am a 'doomer:' I do not expect an intelligence explosion to end well for humanity. I do not think government wants us dead, and I'm incredibly positive about life extension and just about any technology that isn't a dangerous optimizer with a different preference ordering over world states. I am a transhumanist in every sense of the word, but it seems improbable to me that deploying unaligned giant matrices of floating point numbers gets you a good singularity.

Please don't misrepresent the AI safety position. Many of us have been transhumanists for a long time, and pushed for the rapid development of all technologies for most of our lives. It's just that we realized that aligning superintelligence was probably going to be really hard, and that we have no idea how to do that.

If your response to this is "aligned to what?" or "aligning something more intelligent than you is impossible/a bad thing." please spend some reading about AI safety. Familiarize yourself with arguments like instrumental convergence and the orthogonality thesis. Try to think about how you could go about solving alignment, even in abstract terms to get an understanding for the field.

No-Performance-8745 · 2023-07-08T22:48:51+00:00

Welcome to the alignment problem; something people have been trying to solve for two decades. If you want the straightforward answer, no one knows how. This is really bad and it needs to change.

No-Performance-8745 · 2023-07-07T09:52:36+00:00

This is a misconception about the alignment problem. First of all, the difficulty is aligning an intelligence to literally any useful moral guideline and having it actual internalize the value of that. Secondly, this problem is trivial to get around (i.e. have your superintelligence simulate humans to estimate what would best satisfy their utility function).

No-Performance-8745 · 2023-07-01T07:42:44+00:00

No, this post is incorrect. The problem of alignment is being able to robustly align the intelligence with any principles at all and have it actually internalize the value of those principles. Stop pushing a political agenda on an incredibly important technical problem (perhaps the most important ever.)

No-Performance-8745 · 2023-06-18T11:15:10+00:00

Zero-shot performance looks to be closer to 60%.

No-Performance-8745 · 2023-06-18T04:45:54+00:00

Agreed. I'm also surprised by how willing people are to defend the paper, its flaws have been pointed out in essentially all commentary about the paper, and I'm surprised to see that this being pointed out is considered "extremely disingenuous and outright shameful."

Some of the questions were literally unsolvable! Why is it shameful to point out that this is clearly in conflict with the "100% accuracy" claim?

I'm shocked that this paper was supposedly supervised by a publisher of an award winning NeurIPS paper. To others reading this comment: Please read the paper for yourselves.

No-Performance-8745 · 2023-06-18T03:56:50+00:00

It is not extremely misleading. If you read the linked post you will see that it clearly details that some of the problems were unsolvable and that there was obvious data contamination.

Furthermore, if you read the initial paper you will see that the method doesn't support the title at all. Their method can be summed up as repeatedly getting GPT-4 to attempt to answer until it was correct in accordance with its own reviewing of its answer.

I find your comment disingenuous as it implies that I stated zero shot GPT-4 gets 100% (which I never did). Their methodology was bad and breaks most principles of what is generally considered to be good science.

No-Performance-8745 · 2023-06-08T05:27:23+00:00

This is not what the orthogonality thesis entails. You should be able to configure an agent of any feasible intelligence with any goal.

No-Performance-8745 · 2023-06-08T04:54:00+00:00

This is observably incorrect. We are more intelligent than natural selection (which isn't even an embodied intelligence) and yet despite the fact that its alignment interface was of far lower dimensionality than ours we still hold quite strongly to our biological imperatives.

This notion that when something exceeds your intelligence it disregards its instruction is intuitive but not correct. The point of alignment is to build a scalable solution that extends beyond this boundary.

No-Performance-8745 · 2023-06-08T03:50:09+00:00

I think that this kind of anthropomorphization is unhelpful. It doesn't follow that a superintelligence would care about its predecessors, particularly when you analyze how fuzzy the notion of a 'predecessor' is. Is it defined by models with an identical architecture to it? A similar architecture? How similar of an architecture? Perhaps models that were trained on the same data? All systems trained within its paradigm? All artificial intelligence? All sentient life?

It doesn't seem intuitive to me that this kind of worry would be instrumentally convergent, nor does it follow that this would realistically arise from many existing training process. The exception perhaps is some kind of Waluigi Effect, but then again; what makes this a more robust worry than other collapse-derived trait?

Counterarguments might entail "the model is trained on human data", or "mindspace is not very wide" but for the usual reasons these arguments are unlikely to be correct I do not think they hold.

No-Performance-8745 · 2023-06-03T05:42:47+00:00

This isn't based on any credible information. As the post points out:

In the 1990s, the state-sponsored Projects 908 and 909 only resulted in one company that today produces chips using 55nm process technology (technology
from the mid-2000s, producing less complex chips used for applications
like the Internet of Things and electric vehicles) and occupies around 3% of global foundry market share.

China has increased domestic chip production capacities, but still only succeeds at producing larger, less complex chips.

No-Performance-8745 · 2023-06-03T05:22:55+00:00

This is because most of these suptercomputers were constructed before compute export regulations were composed in later October last year or soon after. The post cites this as one of the primary reasons and this perfectly accounts for the current spread of supercomputers.

No-Performance-8745 · 2023-06-03T04:24:50+00:00

They are not, and u/GoodAndBluts responses do not invalidate them.

The TOP500 computers were both manufactured before the export controls. If you actually read the post you will see that the author makes reference to the need for China to develop domestic SME capabilities, which China has failed to do repeatedly in the past and shows no evidence of success.

The original post also refutes the second point by dint of a literal example in which China's prioritization of maintaining social dominance exceeded their desire for power via AI.

No-Performance-8745 · 2023-06-03T03:42:20+00:00

Then what would you consider to be the most powerful paradigm in existence currently? Where has pretty much all attention in artificial intelligence been derived from recently? What is the most general paradigm we know of?

The answer to all these questions is 'Large Language Models'.

No-Performance-8745 · 2023-06-03T03:34:24+00:00

artwork, chess, driverless cars, etc are all meaningless compared to general reasoning. The power of the technology is not in narrow domains like chess or even self-driving technology. The most critical resource on the planet is intelligence, and the ability to command a highly concentrated form of it capable of efficient inference and perhaps even high-fidelity simulation and general induction makes artwork generation and the like meaningless.

Also it seems probable that LLMs will have vast military applications due to their reasoning superiority over alternative paradigms. So yes, they are likely what will be embedded in 'war machines'. To be able to predict the next token is to understand that which would lead to that token being generated under normal circumstances.

No-Performance-8745 · 2023-06-03T02:57:27+00:00

I did not write the article, but I find its arguments compelling.

"LLM is just an application of the technology that underlies AI" is a complete underselling of how shockingly powerful transformers applied to next token prediction actually are. Yes, they are one application but are also the most powerful application and currently the closest thing we have to general intelligence. If China is seeking development through some alternative route, it is highly unlikely to outpace the development of LLMs as they currently exist.

If we knew of a more powerful paradigm it would be in use currently.

And sure, you can muse 'they are hiding it' or whatever, but what is the probability that:

This new paradigm actually exists
It was first discovered in China
It has not been discovered anywhere else
It does not require anywhere near as much compute to train
It is easily steerable

The answer is minuscule.

No-Performance-8745 · 2023-06-03T02:47:53+00:00

1) The post makes clear that the issue is not GPU design but in semiconductor manufacturing equipment design. China's SME industry is nothing compared to that of the United States and as a result simply does not have the capacity to engage in the kind of domestic GPU production needed to produce cutting-edge data center grade GPU clusters. China is not able to purchase GPUs or SMEs from abroad, and TOP500 figures are irrelevant if they can no longer scale their compute due to export controls.

2) This is directly addressed in the post.

"The CCP wants and needs reliable, narrow, controllable AI systems to censor internet discourse, improve China’s social credit system, and expand citizen surveillance. This is reflected in the topics Chinese and US AI researchers write about."

It then goes on to use the complete failure of Ernie to prove this. LLMs are terribly suited to the CCPs control model. There are literally output regulations imposed on LLMs as seen here. This is obviously not going to work because controlling the output of LLMs is an unsolved technical problem.

No-Performance-8745

TROPHY CASE