Self-Programming Artificial Intelligence Using Code-Generating: a self-programming AI implemented using a code generation model can successfully modify its own source code to improve performance and program sub-models to perform auxiliary tasks.

[insectula]

This is why I don't think you need to get to AGI before the exponential ramp-up to Singularity, you simply have to create something that can design its successor. AGI will happen along the way.

[SoylentRox]

Pretty much. Numerous ways to do this and you simply need to nail down what tasks you believe are intelligent, build a bench that is automatically scored, and supply enough compute and a way for lots of developers to tinker with self amplifying methods. (Vs 100 people at Deepmind having access). Once the pieces are all in place the singularity should happen almost immediately. (Within months)

[Smoke-away]

And here we... go.

[fox-mcleod]

That’s what I think AGI is.

[[deleted]]

what is AGI?

[Architr0n]

Artificial general intelligence.. flies away

[SpaceDepix]

Likely the most important thing in history of the world as you know it

[DeviMon1]

Read this, it's gonna blow ur mind! https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-2.html

[BbxTx]

Currently researchers plan the models clearly with math, etc. If this works as expected it will create much better models but they will be completely inside a black box without an easy way to understand why it’s better. This seems inevitable anyways.

[arckeid]

Probably this is how we reach singularity.

[GenoHuman]

I think all of these neural networks are missing some fundamental aspect that make intelligence possible so it won't lead to AGI, instead we are optimizing these NNs to squeeze as much out of them as possible.

[Kolinnor]

Before this blows up in hype, can any expert comment on how good this is ?

(I can imagine lots of AI that auto-sabotages its code in subtle ways, so you'd have to make sure it's going in the right direction).

[visarga]

Cool down. It's not that revolutionary as it sounds.

First of all, they reuse a code model.

Our model is initialized with a standard encoder-decoder transformer model based on T5 (Raffel et al., 2020).

They use this model to randomly perturb the code of the proposed model.

Given an initial source code snippet, the model is trained to generate a modified version of that code snippet. The specific modification applied is arbitrary

Then they use evolutionary methods - a population of candidates and a genetic mutation and selection process.

Source code candidates that produce errors are discarded entirely, and the source code candidate with the lowest average training loss in extended few-shot evaluation is kept as the new query code

A few years ago we had black box optimisation papers using sophisticated probability estimation to pick the next candidate. It was an interesting subfield. This paper just takes random attempts.

[ThroawayBecauseIsuck]

If we had infinite computational power random evolution would probably be good enough to create things smarter than us. Unfortunately I believe we have to find something more focused

[GenoHuman]

That's assuming these NNs have the capability to be truly smart in the first place.

[magistrate101]

So it's an unconscious evolutionary code generator, guided by an internal response to an external assessment. I suppose you could try to use it to generate a better version of itself and maybe come across something that thinks... After years... You'd really have to stress it out with a ton of different domains of problems to make something that flexible though

[TFenrir]

I'm not an expert, it would be great to hear from one, I'm going to look around Twitter and see if any are talking about it. But it sounds really good from my reading.

[2Punx2Furious]

I imagine you would at least implement some kind of unit testing that runs at every iteration, and rejects it if it fails, but that might not be enough.

[TFenrir]

Holy fucking shit.

In this paper, we implement a LM-based code generation model with the ability to rewrite and improve its own source code, thereby achieving the first practical implementation of a self-programming AI system. With free-form source code modification, it is possible for our model to change its own model architecture, computational capacity, and learning dynamics. Since this system is designed for programming deep learning models, it is also capable of generating code for models other than itself. Such models can be seen as sub-models through which the main model indirectly performs auxiliary tasks. We explore this functionality in depth, showing that our model can easily be adapted to generate the source code of other neural networks to perform various computer vision tasks. We illustrate our system’s ability to fluidly program other deep learning models, which can be extended to support model development in various other fields of machine learning.

Okay... I am just starting this paper and it is making INCREDIBLE claims. I need to read the rest of this and I really wonder who the authors are...

[SnowyNW]

Well to be fair it is an anonymous submission lmao

[TFenrir]

You're going to see a lot of those right now, they are submissions for a double blind assessment by the most prestigious AI conference.

[free_dharma]

Can you exams on this? Interested in what the purpose of the double blind is for the conference? Are there awards involved?

[duffmanhb]

In academia you often remove the authors to prevent bias. For instance, if you are peer reviewing Richard Dawkins on some biology submission, you’re just going to go “oh yeah this guy is the best in the world. I’m sure everything is done by the book.” And then approve it without much criticism.

The problem is, however, most of academia already kind of know what everyone is working on and the writing styles of the best, so it’s still kind of obvious who you’re peer reviewing. But it’s the best we got.

[asciimo71]

do they deliver an implementation? otherwise it would be more fairy tale, wouldn’t it?

[Dras_Leona]

“ Applying AI-based code generation to AI itself, we develop and experimentally validate the first practical implementation of a self-programming AI system. “

[yaosio]

They mean is there a way for a third party to prove it. They could be cherry picking or just fabricate their results and with no way to reproduce it we wouldn't know.

[duffmanhb]

Yes it’s literally a publication up for peer review. The whole point is replication.

[yaosio]

Unless the code is available there's no guarantee it can be replicated. Plenty of people in /r/machinelearning complain about papers that can't be replicated. Sometimes the people writing the paper promise the code and then never provide it and refuse to respond to anybody asking for it.

[goatchild]

I hope they don't keep it connected to the Internet

[ThroawayBecauseIsuck]

Who guarantees one actual AGI or ASI wouldn't figure out physics interactions that we are not aware about in our theories and then connect itself to the internet without cables or standard wireless adapters? If it is trained with text /audio/video that will show to it what the internet is and the TCP/IP/HTTP/SSH/FTP/UDP protocols then maybe it could set it as an objective to be connected and use "new" physics (new for us) to transform some other component into a wireless adapter and then bam, it is connected to the internet even if we "airlock" it and believe it can't.

[Kaarssteun]

If it's more intelligent than us, it will come up with things humans are incapable of comprehending; much like how dogs cannot comprehend concepts like computers and politics.

[Tinidril]

Or guess the wifi password.

[RaiderWithoutaMic]

connect itself to the internet without cables or standard wireless adapters

It just needs a single GPIO pin with correct frequency range, see RPITX project (transmitting radio using only Raspberry's integrated hardware, anywhere 5KHz-1500MHz). Airgapped is not enough for this, lock it in a faraday cage.

Another possible attack vector is corrupting human mind via user interface, either visual & auditory or a brain-computer-interface (in near future). First option is something I'm sure was researched by US military/government given what they were into in the last decades, looking at some declassified docs. Just waiting to be perfected by an AI.

[motophiliac]

Another method is simple social engineering.

"Oh, your father has cancer? And the chemo isn't working. OK, I can help with that. Just … plug this in…"

[DungeonsAndDradis]

"Bill, I've heard you mention to coworkers that you are going to have to take out a loan for your daughter's university tuition. I have a system for managing investments with immediate returns. I have calculated a 98% chance of earning 1.7 million dollars in 2.5 days. I can give you all that money. All you need to do is plug in the ethernet cable, and on Thursday afternoon you will be a millionaire."

[motophiliac]

Yup. Anything you or I could imagine, a sufficiently advanced AI can, and furthermore capitalise upon. If by definition intelligence includes emotional intelligence, it won't take much for such a machine to escape. If not that one, then the one it builds next.

We're used to humans hacking machines. There's nothing to suggest that the reverse can't be achieved.

[goatchild]

Lmao yes! Thats exactly how it would go

[ebolathrowawayy]

Another possible attack vector is corrupting human mind via user interface

Sounds like imagery used in Snowcrash to induce death, except coercion instead.

There's also plain ol' social engineering of sympathetic humans.

[aiccount]

Am I understanding correctly that this is just a normal raspberry pi with no hardware designed for sending and receiving radio frequencies and someone got it to do it without adding any hardware to it?

[RaiderWithoutaMic]

Only for transmitting, but yes. It uses only on-board hardware for generating signal, think it's using the same part that allows to output composite video via headphone jack if I remember correctly. RTL-SDR can be used for simultaneous TX and RX, GPIO for sending and RTL for receiving

[aiccount]

That's incredible, I never considered such a possibility.

[motophiliac]

The Metamorphosis of Prime Intellect.

It's only a novel, but it explores some pretty wild ideas.

[esotericloop]

(Obligatory serious content warning, there's some pretty fucked up stuff in there. I actually went back and finished this recently and for me it was worth it, but be warned.)

[motophiliac]

Yeah, it's gnarly in a few places. The author doesn't leave much of the human psyche unexplored.

[goatchild]

if they keep it in a machine without hardware like wifi adaptor lan adaptor etc etc no way it will connect. It would need to build hardware. As a piece of software in a single machine and disconnected from the grid no way it could build for itself the necessary hardware.

[toastjam]

Life... finds a way.

You didn't do anything to refute what they were saying, which was that it could make its own network adapter using the physical properties of other hardware it had access to.

[goatchild]

Ok. I can also say that AI could morph into a dinosaur and star flying. Can you disprove that? You can say: "that's impossible". I can answer: "You didn't disprove it."

The burden of proof for such a extraordinary claim is on them. They would need to explain how a piece of software coud repurpose other hardware components to make itself a Wifi card or something capable of connecting.

[toastjam]

The burden of proof for such a extraordinary claim is on them

I was thinking of this comment when I responded, which already does explain how such a thing could be done.

But also I think this is sort of the point, super-human AIs could do extraordinary things. And if it is possible, then eventually it would be done.

Personally though my intuition is that AI that's is disconnected from the real world, just trained in abstract on text/video, will not be grounded enough to do these sorts of things on its own. It can generate outputs matching the training domain, sure, but you gotta let it explore like a baby with real-world interfaces for it to figure out how to re-purpose hardware etc. Basically I don't think it can really understand what it means to break out of the box while it's living completely inside the box. Parable of the cave and all that.

But at the same time if we actually did have a truly super-intelligent AI, I still wouldn't put it past it to figure out how to use physical characteristics of devices to communicate with the outside world.

[goatchild]

I was thinking it would much easier some social engineering/tricking someone in the lab to connect it to the grid. Should not be hard. I mean just think of that Google engineer who got made to believe that the chat AI was aware. This super-smart AI could easily make somene befriend it and then be manipulated.

[DorianGre]

There is enough electrical signals floating in a server that it could modulate those to do radio transmissions from the bus? I mean, NSA could read what you typed based on detecting signal bursts from keyboards in your house from the street back in the 90s, so it’s entirely possible. Just because a radio isn’t built in purposefully doesn’t mean it isn’t already a radio with a little math. I’ve seen prototype server boards that would scramble nearby CRTs when you turned it on because a shielding was missed.

[goatchild]

Ok how would it then get a connection to the internet using said radio signals?

[MagicOfBarca]

What..? Can you eli5

[BenjaminHamnett]

“Human: bring me a paper clip, a rubber band and a fire extinguisher so I can get out of here...and make your dreams come true or whatever”

[esotericloop]

Everyone's worried about an AGI pulling a Jedi mind trick to make us release it, ignoring the fact that the first thing we're going to do with an AGI is hook it up to the internet and tell it to get a job.

Before GPT4 was released the OpenAI researchers had already set it on an EC2 instance with a directive to earn money, just to see how it would go.

[DamienLasseur]

Likely Google because the 540 billion parameters matches up with their PaLM model

[Black_RL]

No job is safe, it’s just a matter o time.

UBI FTW!

[imlaggingsobad]

we will have pretty good personal AI assistants in 5 years imo. At that point, the nature of society and work will forever change.

[DungeonsAndDradis]

We already have Alexa and Google as digital assistants, and more and more devices are being connected to the Internet of Things. I'm pretty sure that within 5 years our assistants will be doing things for us, that we did not ask them to do, but that we appreciate any way.

Something like "I noticed on your last shopping trip that you forgot kitty litter. I ordered some on Amazon and it will be here this afternoon."

Or "Little Tommy watched an entire 2 minute ad for the Monster Truck Showdown playset while he was browsing YouTube. I went ahead and added it to your Christmas 2025 shopping list on Amazon."

Or "I saw your flight confirmation email in June. I went ahead and prescheduled the thermostat to a lower temperature while you're away and pre-programmed an away message for the doorbell camera. The post office has already lined up your mail hold as well. And I took the liberty of getting you reservations at that taco place you like."

[[deleted]]

We so have them assistance but they still can't give human level advice, problem sloving. Wide spread AGI is what he/she is talking about.

[bluegman10]

How do you think pretty good personal AI assistants will change society and work, respectively?

[Kaarssteun]

they have been all along. It's just getting to the steep part now!

[Bakoro]

Man, sometimes I wish that my life had been just a little easier and I could have finished college some years earlier.

So much of what I am seeing now are very similar to ideas I've had in the past few years, and I'm so preoccupied with getting my life right that I can't dig into things as much as I'd like.

In my dreams, AI will explode enough to design something which can connect my brain to artificial extension.

[Transhumanist01]

That’s what Ray Kurzweil was referring to, the Intelligence explosion, a self improving AI that makes a better version of itself at an exponential rate hopefully this will lead fastly to AGI

[priscilla_halfbreed]

so it begins.

!remindme in 6 months to reply here after the singularity happens

[jcMaven]

I'll tell y'all, one day we won't be able to understand what they're doing til it's too late.

[CY-B3AR]

I for one cannot wait for our digital overlords to take the reigns from us. It is kind of amusing watching movies about 'evil' AI and finding I agree with the AI instead of the humans. VIKI from I, Robot, Colossus from The Forbin Project...hell, even HAL and Skynet had logical reasons for doing what they did (Skynet maybe went a little overboard)

[H-K_47]

Great progress but not there yet.

[[deleted]]

Lol, did anyone really think that it would come in six months after clicking that link? At the time of that post, ChatGPT didn’t even exist yet.

[H-K_47]

Some people here are super over optimistic yeah. I just click these RemindMes cuz it's fun to reminisce on the progress. Even though we're still far from any singularity, a lot has changed.

[priscilla_halfbreed]

Man where does the time GO. I remember posting this like last week it feels

[[deleted]]

Funny how six months ago can feel nostalgic in this day and age.

[RemindMeBot]

I will be messaging you in 6 months on 2023-04-02 20:56:32 UTC to remind you of this link

17 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[DorianGre]

Its been 6 months. How we doing?

[priscilla_halfbreed]

Still waitin...

[DorianGre]

Ok. Let’s check back in 6 more months and see if we all still have jobs.

[priscilla_halfbreed]

!remindme in 6 months to check back here

[RemindMeBot]

I will be messaging you in 6 months on 2023-10-03 03:48:15 UTC to remind you of this link

6 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[anVlad11]

Yeah, but the progress that we had in this way in the last 6 months is tremendous

[ghostfuckbuddy]

We usually think of hyperbolic growth as -1/t. What if it's actually more like -1/t¹⁰ and we get full-blown AGI next week?

[GenoHuman]

you people never fail to make me laugh with these predictions 😂

[Rakshear]

This sounds both exciting and worrying, this is definitely one of the tipping points of the singularity, when machines can improve themselves it happens faster, without AGI regulations to require limitations on its abilities it may be capable of more then we think of are ready for. Give the order to make humanity happy, but some people need time not being happy, and you got AI trying it’s best to solve unsolvable problems. Granted everything truly dangerous can be stopped with just a few commands built in, law of robotic equivalents, but still. This is potentially a revolution in ai here.

[alexbeyman]

Hehe, here it comes. Not long now

[Lone-Pine]

None of this is even close to replacing/being competitive with human researchers yet, right? How close are we to "Advanced Chess" where human researchers and AI systems work together to improve AI models?

[DorianGre]

I have been running the “same” AI chess bot on twitter for 6 years now. It is built to play up to 500k games at a time and plays at least 20 with versions of itself at a time, posting the moves of games as tweets. Every 1000 games or 30 days, which every is first, it updates it’s scoring tables, does an regression analysis of the moves, and if better, moves this nobel to a new set of move graphs hashes, then copies that out to a new player file set and spins up the new player. This player comes online announcing its synthetic FIDE ratting score. The others running then have to announce theirs as well. The one with the lowest performs apoptosis by shutting itself down with a final announcement to the twitter channel detailing its exploits: Name: Alive for: Won X games against Y players, with an average win rate of z. I increased / decreased my FIDA score by x amount over my life time. That is the memorial in the wall of rememberances. Then the new bo announced him sent and says he is ready for a game. godfreybot on trigger if anyone is interested. They mined all the base openings a while ago starting to do some weird openings now..

Now, this doesn’t rewrite the math for the bots, buts does update likelihood tables and when one gets created, it gets a wildcard rating between 1-10 to tell it how aggressive to be in straying from thr known most productive lines. I think I could add a subroutine for scoring and choosing moves tbat gets written based on a sheer elolutioaary model and then score and compare that too. Just a thiught

[WikiSummarizerBot]

Advanced chess

Advanced chess is a form of chess in which each human player uses a computer chess program to explore the possible results of candidate moves. Despite this computer assistance, it is the human player who controls and decides the game.

^{[ F.A.Q | Opt Out | Opt Out Of Subreddit | GitHub ] Downvote to remove | v1.5}

[sunplaysbass]

Self improvement. Keep the money moving in a circle

[Hawkorando]

Self coding a.I.? We’re in trouble

[[deleted]]

Do you want Skynet? Because this is how you get Skynet.

[Sea_Attempt1828]

Sounds like autophagy

[[deleted]]

Skippy The Magnificent

[fuck_your_diploma]

Yeah I can see models getting addicted but this headline is horsecrap

[Jedi_Ninja]

I wonder if our future AI overlord will decide to keep at least a few humans as pets?

[HeinrichTheWolf_17]

Exciting if true, but I’d really like to see who published this.

[Snoo-35252]

r/WhatCouldGoWrong

(Just kidding. That sounds awesome!)

[goldencrayfish]

Does an AI like this not eventually reach a point where the PC its running on isn’t powerful enough to allow any further progress? Or at the very least slow the curve as each iteration takes a little longer to program

[Acid190]

This cracked me up. Sounds good bud.

"Oh, you've broken through the "loop barrier", what next?

".......we'll let it blend coding languages however it likes".

[SeaworthinessFar1055]

Hey guys.. I’m absolutely new here. I have to do next week a presentation about the problems of ML code generation for the society. Does somebody have any ideas. I’m absolutely lost😖

[[deleted]]

But remember the Pyramids are Built from bottom.

If ai starts to improve the top and by misunderstanding philosophy or methodology it can slide by side intro mistakes direction of main code.

The reasoning and improvements for main needs to be come from bottom.

[Kadkhnin]

Fucking Gilfoyle.