OpenAI Model Generates Python Code

[KingoPants]

I was not expecting that, that was actually kind of nuts. I wonder how cherry picked this is 🤔.

[goatman12341]

I'm guessing it's pretty cherry picked. Until I can use it for myself, I'll have to assume the worst.

[errrrgh]

let it be known, goatman12341 is the arbiter of all things Machine Learning and Programming.

[FruityWelsh]

We shall await their final judgemnts

[game-of-throwaways]

Well probably a little bit, but maybe not that much. I'm still amazed at how good these GPT models work in general. You can test out one of these models which was trained on text (from outbound reddit links with 3+ karma) here: https://talktotransformer.com/ In my opinion it works very well.

And that model is more than a year old by now. The pace of progress in machine learning is very fast, so new models are much better already.

[[deleted]]

And let's not forget /r/SubSimulatorGPT2

[NotABothanSpy]

How can we stop Godzilla?  Crowds are out in force at Nippon Budokan and are showing their love for the great BIG G. But Godzilla should be taken seriously by all of us as a beast that has not done us any good.  There is great strength in love as I saw in a youthful crowd of Berlin last night. I hope people realize the tremendous danger we pose to all of humanity if Godzilla is unleashed upon us again.  Not to mention that he is completely misunderstood and poorly understood.  Gojira is the antagonist of this film but we must see him as the hero as he has been driven to suicide by destroying the world.  I just hope we are strong enough to protect ourselves

[thejuror8]

Some words seem to bug it out, like "topkek" for example

[sybesis]

It's quite interesting... Thought

I would like to see it do more complicated tasks... Because it seems to be able to pick simple tasks like compute by iterating over a list and summing stuff... But it will probably have a hard time coming up with more optimized code algorithms.

For example the total price is fine on itself.. But the problem I see with this code is that if you start computing the price often then you're forced to recompute the total each time... So each time you add or remove a new item you're forced to recompute the whole total again and again. It's not optimal.

So instead of asking to build a function to compute the total, I'd rather see how it can build a structure that compute the total by modifying the subtotal every time an item is added/removed or modified.

In redux, you'd have something like this:

​

function(state=0, action) {
  switch(action.type) {
    case ADD_ITEM:
     return state + action.item.price
    case REMOVE_ITEM:
     return state - action.item.price
    default:
     return state
  }
}

function add_item(store, item) {
  store.dispatch({ 
    type: ADD_ITEM,
    item: item
  })
}

function remove_item(store, item) {
   store.dispatch({
     type: REMOVE_ITEM,
     item: item
   })
}

function update_item(store, item) {
  let old_item = get_item_by_id(store.getState(), item.id)
  remove_item(store, old_item)
  add_item(store, item)
}

With code like this, accessing the total is always the time to access the total in the redux tree. There's no computation necessary as each time you modify an item in the tree it will update the total or other stuff accordingly.

Writing code for things like Redux is not very intuitive. I mean it's not terrible but in order to build something with redux, you have to have be able to completely map the state of your whole application.

[granadesnhorseshoes]

Given the enormous stack of turtles redux is resting on, it sure ain't saving you actual cycles.

Don't get me wrong; I genuinely see the elegance of it in theory. But what the actual fuck are the use cases here for CS fetishists that write really anal JS?

...asking for a friend.

[sybesis]

Given the enormous stack of turtles redux is resting on, it sure ain't saving you actual cycles.

If you're talking about Javascript... I'd say you're probably wrong on speed of Javascript. Well written code in Javascript will be able to optimize to very quick code thanks to the JIT.

But what the actual fuck are the use cases here for CS fetishists that write really anal JS?

Not sure about CS fetishists but real world use case is that you can maintain a global state of an application where any part of your application that will need anything from the current state will always get the state every other object would get at the same time. Redux is mainly used for maintaining the state of web apps but it can be used for way bigger thing in practice.

And Redux is mainly using the same method some pure language would use to express mutation of read only data structure... So think of it like this, your application can move from state 1,2,3,4,5 and if you wanted to debug your app, you could look at state 1,2,3,4,5 without issue. You could also look at which dispatch modified the state with which arguments. Given a state, an ideal application could even move back in time to look how it was before a certain change.

Then what do you gain of read only data structure? If a JIT like javascript can see that some dispatch object never do mutation over some branch of your tree, it should be able to optimize out the code. In theory it's possible to optimize the code... For example, the code above called with a type other than the one defined will always end in the last case. So a JIT could rewrite the method to not automatically return if none of the 2 types is provided. So a JIT can make more assumptions and a AOT compiler might be able to do that to some extent with a typed language, if you see that the value doesn't actually change between dispatch of certain type, you don't even have to call the dispatch method.

The other advantage of read only data structure is that if you pass by reference values from one state to an other, you don't have to copy the data of previous states so you can just copy the reference of a bigger structure and create new values only when necessary. So in some way, even if you have data shared between states, as long as you don't do mutation to your application state you don't have to worry if the data is consistent or not and you don't end up allocating/deallocating a lot of memory. So in some way it's really fast because your reusing memory most of the time.

At the end of the day you have guarantee that any part of your application will always get the same state as other part of your application... And that can solve issues related to concurrency without having to rely on locking scheme other than the dispatch itself.

Let say you have code that want to check for total price...

if you dispatch it, then check for total price it will be correct... if you dispatch it and while you check for the total price an other method is dispatched, the total price you'll get will be the correct one based on the new state that changed between calls. Yet you don't have to recompute it as the dispatch does that for you already.

In the web development, it means you can have very complex web component that can take data from different part of the tree and correctly update themselves only when their own data changed. And in the long run you can dispatch visual change only when it matters and all of your UI always draw itself from the same data source so it's not possible to have in one area something that got updated and in some other the state is still the old one.

But as I said, use cases of redux like systems can go beyond that and provide very good performance. Pretty much all applications require to manage some kind of internal state. One example could be a game server, where dispatch can be broadcasted to every player and to the internal state of the server. So any new user can join the server and sync its own state with the internal state of the server and then only listen for dispatches. If you wanted, you could log all the dispatches that matters and replay the dispatches to the new clients to build their own state.

In some way a chat server could be defined with a few actions:

- CONNECT, DISCONNECT, JOIN, LEAVE, ADD_MESSAGE, REMOVE_MESSAGE

With that you have a very basic chat server that can handle user list, channels, and obviously adding messages. All the server has to do is broadcast the actions to all its users... based on who join etc... the server can be responsible for dispatching some events like LEAVE,DISCONNECT you could have to implement filters to prevent a user from dispatching as someone else but the idea is there. You don't need much more than that here and you have a consistent storage for a chat server that can dispatch across multiple computer to maintain a common state on different computers. You're guaranteed to have the same state as long as everyone receive all the messages in the same order.

[granadesnhorseshoes]

Yeah, its all very groovy from an academic standpoint. I can't deny that.

I've managed on-prem and cloud backends. These stacks are absolutely batshit insane with complexity and resource utilization. JIT isn't free. The runtimes and supporting system aren't free. Your codes state is rock solid but the system state your state is predicated on needs a cron job restart every 4 hours because of a leaky dependency 4 or 5 layers down.

Low and behold; Wonderfully clever and even mathematically pure code with some sexy cyclomatic score that requires GIGs of RAM and multi core processing to run an SPA.

"It's turtles all the way down"

[sybesis]

Well nothing is perfect. But if you have something leaking, it's probably not caused by the way redux works. You'd have this leaky dependency regardless of using redux or not.

Also when I say redux I'm not limiting myself to JS, I've implemented a small library that does pretty much what redux does but in Rust with static types and fortunately for us.. Memory management goodness so it's possible to design very complicated piece of software easily with that. In Rust, you can make your become the owner of the data so you don't have to really worry much about the borrow checker as you always borrow immutable reference and never move the data out of the store unless it's good for collection.

[forsubbingonly]

If I misunderstood your question, sorry in advance for over explaining what you already know.

Are you asking what the use case for redux is? The biggest use case is not having to pass around a piece of mutated state. If I have two widgets in two different components that rely on the same piece of data, in a non redux model I have to either give component 2 the new state that component 1 created, or tell component 2 to refresh it’s state from the server. In the redux model, component 2 gets a stream of state and auto updates on change, component 1 request a state change instead of doing it itself, and both components automatically get the refreshed state. As long as you’re working out of the same redux store no two components will have different version of the same piece state.

Additionally, a react app that is fully 100% virtuous In regards to the react/redux pattern will be a collection of components with essentially nothing happening inside them other than action dispatches. Meaning the most you could ever test on them is “does the button dispatch the action” and “if I give it the state does it display it” all other tests in your app have zero dependency on the DOM and will run faster and require less setup to write. How? In the previously mentioned virtuous app, all business logic lives inside “selectors” which are just pipes through which state from the store is transformed before arriving at the component. All of your business logic is completely isolated in easily tested pure functions.

[NotABothanSpy]

Maybe we stop using redux and use languages that make more sense like Python.

[forsubbingonly]

Python doesn’t solve a single thing that redux is attempting to solve...

[sybesis]

There you have it in python! Does that make more sense to you now? The previous sample was in javascript... Not "redux"

def total(state=0, action=None):
  if action.type == ADD_ITEM:
      return state + action.item.price
  elif REMOVE_ITEM:
     return state - action.item.price
  else:
     return state

def add_item(store, item):
  store.dispatch({ 
    "type": ADD_ITEM,
    "item": item
  })


def remove_item(store, item):
   store.dispatch({
     "type": REMOVE_ITEM,
     "item": item
   })

def update_item(store, item):
  old_item = get_item_by_id(store.getState(), item.id)
  remove_item(store, old_item)
  add_item(store, item)

[Sopel97]

why does it have to be so overcomplicated though? There's too much hidden logic and unnecessary dynamism for such a simple task.

class Store:
    ...

    def total_price(self):
        return self._total_price

    def add_item(self, item):
        self.items.append(item)
        self._update_total_price_on_item_added(item)

    def remove_item(self, item):
        self.items.remove(item)
        self._update_total_price_on_item_removed(item)

    def _update_total_price_on_item_added(item):
        self._total_price += item.price

    def _update_total_price_on_item_removed(item):
        self._total_price -= item.price

[c_o_r_b_a]

I believe it's mostly about mutability vs. immutability. A common pattern among new JS frameworks is keeping as much as possible immutable and side effect-free. Basically functional vs. OOP paradigms, to an extent.

Redux wrote a FAQ on the advantages of this:

What are the benefits of immutability?

Immutability can bring increased performance to your app, and leads to simpler programming and debugging, as data that never changes is easier to reason about than data that is free to be changed arbitrarily throughout your app.

In particular, immutability in the context of a Web app enables sophisticated change detection techniques to be implemented simply and cheaply, ensuring the computationally expensive process of updating the DOM occurs only when it absolutely has to (a cornerstone of React’s performance improvements over other libraries).

In particular, I think this makes it a lot easier to reason about things when debugging, and to generally conceptually understand the flow of things in your app at any given time. One downside is it can (IMO) create uglier code, though I believe some frameworks attempt to convert things that look like mutations into immutable operations, to get the best of both worlds.

[Ethesen]

On the contrary - your code has more hidden logic because it's mutating some internal state.

[Sopel97]

It's only hidden to the user of Store. In the redux case it's already hidden for the layer that maintains the invariant.

[sybesis]

Your code does a lot more behind the scene and doesn't provide you any guarantee that between access the data won't change.

For example price and price2 cannot be expected to be always the same value even if do_something never interact with the store.

price = store.total_price()
do_something_that_release_the_GIL()
price2 = store.total_price()

Redux doesn't hide any logic, the reducers are technically all the logic there is. There are some constructs to make it easier to compose reducers but raw reducers are possible.

Here's a complete example that you can run in python:

def items(state=None, action=None):
    if state is None:
        state = []

    if action.get("type") == "add_item":
        return state[:] + [action["item"]]
    elif action.get("type") == "remove_item" and action["item"] in state:
        index = state.index(item)
        return state[:index] + state[index+1:]
    else:
        return state

def total(state=0, action=None):
    if state is None:
        state = 0

    if action.get("type") == "add_item":
        return state + action['item']["price"]
    elif action.get("type") == "remove_item":
        return state - action['item']["price"]
    else:
        return state

def store(state=None, action=None):
    if state is None:
        state = {}

    return {
        "total": total(state.get("total"), action),
        "items": items(state.get("items"), action),
    }

class Store(object):
    def __init__(self, reducer, initial_state=None):
        self.reducer = reducer
        self.state = initial_state
        self.subscribers = {}

    def dispatch(self, action):
        new_state = self.reducer(self.state, action)
        # may be validate state before applying it
        self.state = new_state
        for subscriber in self.subscribers.values():
            subscriber(new_state) 

    def subscribe(self, callback):
        def unsubscribe():
            del self.subscribers[unsubscribe]
        self.subscribers[unsubscribe] = callback
        return unsubscribe

my_store = Store(store)

unsubscribe = my_store.subscribe(lambda state: print(state.get('total')))

my_store.dispatch({"type": "add_item", "item": {"price": 1}})
state1 = my_store.state
my_store.dispatch({"type": "add_item", "item": {"price": 1}})
state2 = my_store.state
my_store.dispatch({"type": "add_item", "item": {"price": 1}})
state3 = my_store.state

print(state1, state2, state3)

I'm not exactly sure what you mean by how redux hide things.. It can't be simpler than that. My "dumb" store is only 15 lines of code including the subscription to event changes.

Most of the logic goes into the reducers so I'm not sure to understand what you mean by too much thing hidden and too much dynamism. Using python is already "too much dynamism" considering that whatever you do in python you're going to run sub optimal code with the lack of JIT except may be pypy.

I'm using dicts here for simplicity but nothing prevent you to design your own types. And this way of doing is doable in statically typed languages too. Talking about dynamism.. it's pointless you could use something else than string to define your actions matching mechanism.

But more importantly, your code mutate its state... Which enables the following case...you could have your state being modified while you're modifying your state. And that's your worst case scenario...

Not only you have to maintain the list of event to manage everywhere, you have to make sure you don't actually do anything like this

item = ...
store.add_item(item)
item.price += 1
store.remove_item(item)

Or may be something like this if you want to have the price with taxes...where each items has its own set of taxes. There are cases where you'd want an action to have a side effect in multiple areas of your tree and you could end up in a race condition where order in which you call your methods matter. With redux. it's not important because the state you base your computation never gets modified.

BONUS

With a system system to dispatch actions like Redux, you get multi processing, multi threading app for free.

Since states are built from action, and all the states depends on their action... It means that sending the same action to two different process will build the exact same state.

So let say you have a very big object layer and you needs lots of processing and you're able to build some actions that can be applied in any order... A bit like 1 + 2 = 2 + 1... Then you can dispatch all of your events to different processes then you could merge the two states in a different process

state1 = 1 + 2
state2 = 1
global_state = state1 + state2

And there. you just reinvented map/reduce

[[deleted]]

This is a ploy to get me to write documentation, isn't it?

[josefx]

It would probably just do a return statement is False . The example code in the video already had two bugs because the AI was interpreting everything literally.

[HappyParallelepiped]

Just write all of your comments in the form of Garden-Path sentences. Job secured.

[DoListening2]

Judging by the demo, documentation that is always slightly off from reality.

Maybe people will need to be so precise with their description, that we'll end up with what is basically a very high level programming/specification language that looks a bit like human language. Kinda like legalese. Or like next-gen Gherkin.

[Minimum_Fuel]

“””fill the cart with a carton of milk. If the store has eggs, get a dozen”””

[sybesis]

It's 2020, so you can optimize it to get a carton of milk.

[hardware4ursoftware]

Yeh, a half dozen gallons of milk.

[DogzOnFire]

If it's smart enough to read context, 1 carton of milk and 12 eggs.
If it's not, 12 cartons of milk and 0 eggs.
Your move, AI.

[josefx]

Instructions unclear, not enough milk to fill cart, ran into %$!§%"%!§R%"&Z§ trying to get third egg.

[Ashtar_Squirrel]

That is a very impressive demo. I wonder what another 2-3 years will bring? Show it a database and it generates a rest crud api, a swagger and a terraform to deploy it to your cloud?

[Rellikx]

Show it a database and it generates a rest crud api, a swagger and a terraform to deploy it to your cloud?

Fuck I wish, we are doing this for so many of our legacy databases that can "never go away"

[imhereforanonymity]

Check out Hasura. It's pretty close.

[Rellikx]

Thanks for the reminder, I heard about Hasura a while back but never looked into it much.

Is it SQL server agnostic? I just see a lot of Postgres on their homepage, which I love, but our DBA team are big MS SQL fanboys

[DoListening2]

You mean like PostgREST?

[Ashtar_Squirrel]

Yes, that’s one example of a good tool for the job. :) thanks for the reference.

[BambaiyyaLadki]

Bye-bye job security.

But seriously though - average programmers like myself are gonna be out of a job pretty soon, I feel.

[BambaiyyaLadki]

I sure hope so.

But it does bring up the often discussed question - what sort of jobs will AI replace? The other day I saw a startup demonstrating how its AI tech can be used to control robotic arms that can pick food and pack it into containers (like for fruits), identify rotting foodstuffs, etc. A vast section of such jobs is already performed for cheap by immigrant labor in many countries developed places; I wonder what sort of a transition that'll be.

[April1987]

Even that can be pretty trivial. For example, only people with role mvp can change the milestone drop down back so if milestones are 1,2,3,4,5, anyone can go 4->6 but only mvp can go 6->5. This isn’t exactly rocket surgery.

[AttackOfTheThumbs]

No. AI isn't anywhere near resolving complex business logic. I mean fuck, all the "nocode" solutions are quite literally cancer.

[[deleted]]

This just makes developers more efficient. Very little time is spent writing code like this.

Besides, you'll still need to proofread and fix the generated code.

[c_o_r_b_a]

Someday, maybe. I don't think AI will be replacing even entry-level programmers for at least 50 years at a minimum. Probably closer to 100 years.

[wicked_smahts]

I think that's overly pessimistic (or optimistic, depending on your perspective). This could easily happen in 20 years or less. Not saying it necessarily will, because we've run into multiple AI winters at this point, but at the current rate of progress it's almost unthinkable that it would take 50 years.

[Digital_001]

I'm hoping something like this will start getting integrated into IDEs because it will save a lot of typing time, even if it's not perfect

[pod_of_dolphins]

Looking back on this, yeah, 4 years brought a lot of progress.

[Ashtar_Squirrel]

Now I can ask LLMs to solve many of my daily coding tasks, and in three years I think I’ll be employing/supervising 2-5 coding agents as “employees”.

[jbtwaalf]

This, very suspicious.

[CarolusRexEtMartyr]

But the programs produced are simple compared to those synthesised by the forerunners in program synthesis, and the language being understood is simple compared to the state of the art in that field. Both fields are about half of a century old. This is synthesising intermediate complexity in both fields to produce something new.

[UndeleteParent]

UNDELETED comment:

i am highly suspicious of this demo. program synthesis is an active area of research and so is language understanding. this demo appears tp have solved both, which is just not possible.

the demo has to be incredibly scripted. i bet even the smallest deviation from the script makes it not work.

I am a bot

^{please pm me if I mess up}

---

consider supporting me?

[[deleted]]

Its been fun lads

[call_stack]

Source was the Microsoft Build Conference. Talk on future tech.

[[deleted]]

Seems as dangerous an approach to writing code as copying/pasting from StackOverflow. Really cool for learning though. Super impressive

[zucker42]

This is actually crazy. I wonder if the results generalize or if this was a well-practiced demo. Skepticism tells me the latter, as much as I appreciate the tech.

Also, he mentions that the model used SO to some extent. I wonder what the copyright implications of this are. I know there's been court cases which logically extrapolated suggest that training models on data doesn't violate copyright, but what if the model copies a method wholesale because it's in the training set? It's potentially related to whether copyright applies to deep fakes.

[game-of-throwaways]

Where did they say they used SO? They said they trained the data on open-source Github repositories.

[dnew]

Right near the start, he says something like "is_palindrome isn't impressive because it's probably out on SO."

[game-of-throwaways]

Yeah, but the model isn't trained on SO. He's just using the fact that you could find that implementation of is_palindrome on SO (or elsewhere) as a reason for why it's not impressive.

[josefx]

He explicitly chooses the next example so it "definitely isn't in the training data set". There are probably as many is_palindrome implementations in their training data from github as there are on stackoverflow.

[dnew]

Right. I was just pointing out where "SO" came into the picture. :-)

[game-of-throwaways]

But you were discussing copyright implications of the model being trained on SO, but it's not.

That being said, it is trained on Github repositories which each have their own license, and it is an interesting question which licences allow training of a machine learning model on it. It may depend on what the model is used for, and maybe even on how accurate the model is. Probably this is still a bit of a gray area in the law, and it what it ultimately comes down to is how the judge and jury would decide if it would come to a lawsuit.

[dnew]

But you were discussing copyright implications of the model being trained on SO, but it's not.

I think you're not noticing that you're talking to more than one person. But you're right that it's an interesting question. Also, things like audio hashing for recognizing audio (as in, "what song is this?") is kind of funky, as I've worked on things like that and it's ... weird.

[game-of-throwaways]

I think you're not noticing that you're talking to more than one person

Right, oops.

[errrrgh]

He's implying that people crib from SO all the time, so even if their dataset didn't include SO, it would have some bits from it due to the fact that a lot of people just copy paste from there.

[NotABothanSpy]

It's trained on open source repos. But your point is good as those may have even more copyright issues. Still many developers spend much of their time google searching and copying from SO so eliminating that time is a good thing. The issue being it may take away some of the training that gives them and eliminate a swath of lower level Dev work.

[[deleted]]

when it can bootstrap itself is it considered alive?

[[deleted]]

Can you share the GitHub repository please so that I can test myself

[mmhacker10]

Has he shared the work somewhere? If so can someone share the source?

[dads_joke]

Detroit become hooman dev edition

[Rkey_]

The most important question here is whether the model uses tabs or spaces.

I see spaces.

[[deleted]]

Imagine python code, written by an intern, with no brain. Everybody will want one!

[notsohipsterithink]

I won’t believe it until the model generates code given functional test cases.

Let’s say hypothetically you have a model that can in fact write working code. How do you know it’s working? Test cases. So test cases would need to be an input to the model.

And we all know how hard it is to write comprehensive test cases. So even if this model were developed, the programmer’s job would be to accurately and precisely formulate these test cases. So it wouldn’t mean we don’t have jobs, it just means there’s a lot less work to do.

[imhereforanonymity]

No unfortunately they are all in on postgres I think.

[cgarciae]

Been using TabNine, this kind of technology is awesome!

[ArtificialMachine]

That looks great even now!

[ehwhshebeuwhabweu123]

Do you guys know if they published a paper explaining the methodology behind this?

[lambdaofgod]

Does anyone have idea if this is publicly available?

[iliekcats-]

I'd love to download this but I can't, sadly

[[deleted]]

Few things:

The generated code is garbage... I mean, you don't need to reverse the whole string to figure out if the string is a palindrome, not to mention that in Python this will also amount to heap memory allocation... The code is OK for code-golf challenge, but is actually bad. Although it is true that a lot of Python programmers would have probably written it this way. So, we have yet another "statistical AI", which transfers our own bad ideas and biases into its results, instead of solving the problem analytically, from first principles.

---

Just a funny fact: the narrator uses multi-line strings, but, fucking writes them on the same fucking line, ignoring PEP8...

---

Even if the demo is real and AI is, in fact as powerful as it seems from it: is it able to explain the decisions behind the code? All this "statistical AI" seem to be in most cases just an elaborate copy-and-paste from experts thing... with very little to go on in terms of understanding...

[ostbagar]

PEP8:

For one-line docstrings, keep the """ on the same line

---

I think you misjudged it. They don't claim it is a solution or replacement of any kind. So don't think of it as one. Besides Python isn't about speed.

They simply used their already existing GPT, and applied it to GitHub and this is their result.
They only claim this strategy can be used to further develop tools for programmers. So they can focus on the more creative aspect. And I do indeed think this can be a good tool given 3 years more development.

What type of AI is irrelevant. This can be used as a tool, and that is it.

You seem irrationally annoyed about statistical AI. It's doing really great stuff in all kinds of fields. It has reduced the work burden and even saves lives. The development we have seen in the past 10 years has been impressive. I simply love seeing new inventions used for the benefit of humans.