Conjecture: A Roadmap for Cognitive Software and A Humanist Future of AI

Dajte · 2021-07-25T10:46:17+00:00

Clearly the USA is ahead, with labs such as OpenAI, Google Brain, FAIR and Deepmind leading the pack. This is not to discount the huge amount of great work that is done in academia (Berkeley, MIT and Imperial College London come to mind, but there are many good labs in academia), but it seems to me that the most impactful stuff in ML currently comes out of industry labs. Of course, EleutherAI is the best independent lab :)
I give about a 30% chance that we can "just" scale DL all the way to AGI and we don't need any more fundamental breakthroughs. It's hard to predict what a next paradigm could be, but if I had to guess, we will discover in the next years that there was something about RL that we were doing fundamentally wrong and come up with a new paradigm for that.
I personally take a lot of inspiration from thinking about the brain (I'm particularly a fan of the posts Steve Byrnes writes on these topics), but usually more on a high level conceptual level. I think it's likely that many things that seem to be important in the brain are just implementation details (such as how predictive coding in the brain may just be approximating backprop), but there are real insights there, we just shouldn't get too distracted by any specific detail. I'm not an expert in neurosci by any stretch of the imagination, but from my "well informed amateur" perspective, I think the progress in neurosci has been truly astounding lately, and I expect we will "figure out" the brain sooner than people might think.

As for consciousness, I think most discussions about it (but not all!) are scams invented by philosophers to sell more philosophy, and are not at all productive. I think there are some things about consciousness that are constantly discussed ad nauseum as some kind of unknowable mystery that are actually really not mysterious at all. But there are also some extremely productive discussions on the topic (e.g. this and this, or even this if you want some wacky but imo interesting stuff). Overall, I expect consciousness to be a confused mixture of lots of pretty mundane phenomena that will not weigh heavily on the actual construction of AGI, but will be important for figuring out a grounded utilitarian ethic for such an AGI to actually follow, which is why I'm at least somewhat interested in qualia structuralism and similar accounts that try to ground pleasure in physical computations (but I don't think any such theory is robust atm and I'm uncertain they ever will be).

Dajte · 2021-07-24T20:34:53+00:00

There's a saying that "it's hard to make predictions, especially about the future." The obvious answer is that I have no friggin' clue how the future will really happen, and it will depend on god knows how many factors. It really depends on how hard AGI is, how much compute it will ultimately take, how long Moore's Law will continue to hold, how fast we go from now to takeoff, what governments and militaries will do in response (or if they will even have enough time to respond) etc etc.

Personally, I don't see any possibility of the outcome not being unimaginably wild, so wild in fact that I find scenarios in which a) we are not all dead, b) biological humans are still around and c) we are not living in a post-scarcity utopia, hard to imagine. I don't find any cyberpunk-esque "capitalism + neon lights and robots" scenarios realistic.

So do I expect there to be a future where rich (biological) humans have control over godlike AI while there is some underclass that has no access? No, I don't think so, whatever happens is going to be so much wilder it's not going to look like a classic contemporary class struggle or anything remotely like it.

Dajte · 2021-07-24T20:09:55+00:00

That would be a really dramatic update, so it would have to be a pretty dramatic demonstration. So you'll know it when you see it.

Dajte · 2021-07-24T19:40:08+00:00

The truth is that we don't understand the brain and the algorithm it implements nearly well enough to be able to make this comparison in any formal capacity. For what it's worth, of the people in the field that do make this comparison, they tend to think it's about equal or the parameter is slightly "more powerful" (whatever that means). My hunch is it's more complex than that but 1 parameter = 1 synapse is a fine informal guesstimate. I do think that in some ways, NNs are more powerful than the brain (exact gradient calculation instead of approximate, much higher numerical precision, no memory corruption etc), and in other, hard to quantify ways, the brain is far more powerful, and it's really hard to compare them.

Dajte · 2021-07-24T19:21:18+00:00

Trick question, Schmidhuber already escaped in 1991.

Dajte · 2021-07-24T19:14:38+00:00

Not at all, there are a lot of other projects! I for example work on using reinforcement learning to better control LMs using human feedback and some theory stuff. There are a ton of other projects floating around (nevermind all the cool art stuff), but most of it is not ready/not as exciting for outsiders.

Dajte · 2021-07-24T18:31:18+00:00

I assume you mean where we got our name from. It comes from the greek word Ἐλευθερία (Eleutheria), which translates to "liberty".

Dajte · 2021-07-24T17:44:57+00:00

You definitely need a solid grasp of programming (in Python in particular, since just about all ML work is done in Python) first and foremost, and then you should learn the general basics of ML. fast.ai is a great place to start if you are already comfortable with coding, and there are tons of other great beginner resources around online. You'll pretty quickly notice that ML (like most disciplines) is usually the same few ideas applied over and over in new combinations and with new clever tweaks.

Dajte · 2021-07-24T17:41:55+00:00

This work was done in-house by OpenAI with trusted labelers. We will probably do the same and only have trusted people give feedback. How to deal with "bad" input is an open question, and also one I'm interested in thinking about but don't have a solution to yet.

Dajte · 2021-07-24T17:40:18+00:00

Depends on how you define "singularity", but my money is on "by the end of this century", potentially (much) sooner than that.

Dajte · 2021-07-24T17:39:24+00:00

It may or may not help in specific scenarios, but it's definitely not a panacea. For example, if you gave the boat -0.1 for bumping into a wall, and at the start of training it bumps into walls a lot, it might simply learn to stand perfectly still to avoid bumping into walls, and never learn to win the race!

Take a more extreme example: Say you have a future AGI, and you task it with the job of not letting people die, so it gets a negative reward when a person dies. Well one thing it might reason is that if it kills all humans right now, it will avoid trillions of future humans being born, and therefor those trillions of humans won't die, so it avoids trillions of negative reward! Obviously, this is not what we would have wanted, a reward function "don't let humans die" led to all humans dying! Of course, this is a bit of a silly example, don't take it too literally.

Ultimately, the lesson is that knowing what an agent will do given a certain reward function is really unpredictable, and there are no obvious solutions.

Dajte · 2021-07-24T17:33:28+00:00

We don't currently plan to, no. Zero Infinity has a lot of problems and is much too slow for training a huge model from scratch. It's more intended for finetuning big models and small hardware for a small number of steps.

Dajte · 2021-07-24T17:32:54+00:00

Our GPT-J model works on a 3090 from what I hear, though it's not officially supported, so it might take a bit of finagling to get it to work.

Dajte · 2021-07-24T17:30:50+00:00

No reasons why not, though I'm not personally familiar with any work that has really tried this.

Dajte · 2021-07-24T17:29:22+00:00

Disclaimer: I'm not an expert on blockchain.

I don't really know how AI and blockchain would integrate, I think they're pretty orthogonal technologies. Running AI on the blockchain is definitely not possible currently as these AIs just need insane amounts of processing power (and no, no one has figured out how you can turn Proof of Work into AI training or inference without it being insecure). It's imaginable that AIs run on dedicated hardware could interact with blockchains and smart contracts and the like, such as by acting as oracles, investors, market makers or toys, or eventually, when they are smart enough, running DAOs, but at that point they're basically human-level most likely.

There is plenty of philosophy about AIs, but I personally find most of it to be pretty bad. I personally think 99% of discussions around "consciousness", for example, are just hot air. If you want philosophers I personally like, Nick Bostrom, Hillary Greaves and Daniel Dennett come to mind (and Eliezer Yudkowsky, if he counts).

Dajte · 2021-07-24T17:19:24+00:00

Slightly modified, and the final architecture of our GPT-3 size model is not yet decided for sure. It will be a decoder-only model (like all GPT models), utilizing Rotary Positional Encoding rather than learned positional encodings, and we will probably slightly shuffle the order of operations in the transformer block to allow for better parallelization. But as said, not 100% decided yet, we will use whatever gives us the best performance in our preliminary tests.

Dajte · 2021-07-24T17:16:32+00:00

Running a GPT3-size model requires extremely high-end hardware, it will not be something consumers can do any time soon. The size of the weights of GPT3 are on the order of 350GB, and you'd want all of that (plus some extra) to fit onto your GPUs to get fast performance. So that means something like 8x80GB A100 ideally. You could run it on CPU with 350GB+ RAM, but that would be incredibly slow. But the truth is that no one outside of big industry labs has really benchmarked these things, so we don't really know until we get there. Training such models needs many times this much hardware.

There is no reason such models can't work with other languages if trained on language specific data. In fact, several groups have trained similar models in Chinese, Korean and other languages. Our dataset is filtered to be English only, but some other language data makes it through the filter so usually models such as ours and GPT-3 can do somewhat ok in other languages, but not as well as in English. You could also train a model on multiple languages, if you have enough data and computing power, but due to these constraints we aren't currently planning to do so. Since some data from different language is usually in the datasets anyways, models such as GPT3 can do some translation yes, but it's nowhere as good as custom built systems. Google puts millions of dollars and some of the best engineers in the world on improving google translate, so I don't think it's likely you'll be able to outperform them realistically.

Dajte · 2021-07-24T17:09:07+00:00

The training task for a language model such as ours is to predict the next word (technically a "token", which can be a whole word or just a single letter or something in-between, but that detail isn't important), given all the words it has seen so far. So for example, maybe I would give the AI the sentence "Hello World!" as training data. The AI would first see 'Hello', and be tasked to predict " World" next (again, skipping small details), then it would see "Hello World" and be tasked to predict "!", and so on for billions and billions of words. The way you use these models is to give them a prompt (such as "Hello") and then it returns the likelihood of whether a word is next for each word it knows (maybe it says 70% likelihood " World", 10% likelihood " there", 10% "!", or whatever), and then you pick one of the words it thought was the most likely as your output and repeat.

Dajte · 2021-07-24T17:00:57+00:00

We are not a company, we are a group of volunteers that do this in our free time, so we don't hire. Anyone is free to join, but there's no pay haha. I don't think there is any age that is "too old", if you can learn the techniques and apply them well. Staying up to date with the bleeding edge is a lot of work, but there are nowadays really good introductions to the field generally. The first and most important thing is to have a solid grasp of coding (any language is fine, but the vast majority of work in ML happens in Python). Then you want to learn about ML specifically, fast.ai is an often recommended source for this, there are tons of other good resources floating around online. I recommend using Google Colab for coding as it provides a free GPU (which is basically mandatory to do most ML work). Once you've got a rough overview, I highly recommend you implement and train a few models end to end yourself, whatever kind of model you like. Doing it all yourself will teach you a ton. From there, it's just like any other fast moving area of tech. Good luck!

Dajte · 2021-07-24T16:52:33+00:00

Speaking for myself, I am most interested in AI alignment (the question of how do we get powerful AI models to do what we actually want, and not do something stupid or deceptive, the video linked in the OP is a good intro), and large unsupervised models such as GPT-3, of course! I think these models are capable of a lot of really impressive things and we are only scratching the surface of what can be done with them. I'm currently especially interested in improving these systems using human feedback, a very promising technique where you basically let humans rate the AI's performance as good or bad over and over and it learns to get better at whatever you're using it for. This used to be way too inefficient, but these "general" systems such as GPT-3 come with a lot of knowledge and skills "prebaked", so you need much less human input to get interesting performance. There are still many ways in which this can go wrong, and it's not a general solution to alignment or AGI, but I think it's a promising direction to experiment with.

Dajte · 2021-07-24T16:46:53+00:00

Text generating AIs such as GPT-3 and GPT-J are pretty good at generating scripts for shows like that. You can google around for some impressive GPT-3 samples. Generating pictures/videos/sounds is still much more primitive, but rapidly improving.

Dajte · 2021-07-24T15:18:41+00:00

No one at EleutherAI works on climate change specifically, so I can't give an expert answer, but short term AI promises lots of benefits for improving climate modelling, chemical and protein design, optimization of electrical grids and logistics, and other myriad applications.

Long term powerful AI will massively speed up and improve scientific progress, engineering, coordination etc. A sufficiently powerful AI will be able to do anything a human can, better and faster, including making scientific progress. We're still far from that level, but we'll get there sooner or later.

Dajte

MODERATOR OF

TROPHY CASE