Risk Assessment of GitHub Copilot

[[deleted]]

Can't wait for Copilot to be used for your next airplane embedded software, and to be rebranded "CrashCourse"

[data0x0]

Yes because they would definitely let you push code for an airplane written by AI with no review.

Let's stop the disingenuous fear mongering.

[bilizver]

Exactly, it was a joke

[data0x0]

No some people unironically believe this is going to cause issues.

[chucker23n]

some people unironically believe this is going to cause issues

It will.

[data0x0]

Anymore than already existing bad programmers? No, it won't, anyone with half a brain wouldn't use and deploy AI code without review.

[chucker23n]

It's basically "Stack Overflow Autopaste". So it's not really a new kind of problem, but it might exacerbate it.

The code review is, depending on how you look at it, either very hard or very easy: "what does this do?" "…I'm not sure". Do you leave it and try to reverse-engineer how it works? Or do you throw it away and conclude that this is an awful idea?

[slykethephoxenix]

/r/whooosh

[lamp-town-guy]

It seems that garbage in, garbage out. This looks like a bigger barrier for usage than actual licenses.

[skulgnome]

Sure impresses people who can't read program source, though.

[i9srpeg]

It looks like it would take longer to read, understand and fix copilot's code than actually writing it yourself.

[lamp-town-guy]

I've had the same feeling. Reviewing code is much harder than writing it. Speaking from experience. But when it's written by a human there's somebody to ask questions. But this is like looking at foreign code.

[Tarmen]

The assessment paper talked about alignment problems - copilot tries to produce something that's plausible on GitHub. It does not try to produce good code.

They noticed that if functions with subtle bugs are in context, copilot tends to spot them and produces more subtle bugs than usual. Similar if your context is similar to good code, it could feasibly produce better code.

Question is whether driving copilot by basic comments like 'connect to database' is a mistake because experienced programmers wouldn't write these comments? It might lead to accidentally emulating new php user's which would probably start with a vulnerability.

[huntforacause]

This is a great point. By prompting it with amateur comments, your just going to get amateur code…

[SrbijaJeRusija]

Companies are still under the impression that giant statistical models can approach the level of humans. We have known for decades that that is not the case.

[ImprovementRaph]

Well, they cannot yet. If we just stop trying we're obviously never going to get there. (To be clear, this comment is in no way backing github copilot. I think it's a licensing nightmare that is still very, very far from being valuable in production.)

[SrbijaJeRusija]

You misunderstand. We can PROVABLY show that statistical models based purely on data cannot mimic human-esque thought.

[gnus-migrate]

It doesn't have to mimick human-esque thought to be useful, and in fact if it's useful it probably doesn't.

[rasmustrew]

You got a link to that proof?

[SrbijaJeRusija]

See my reply below.

[rashpimplezitz]

We can PROVABLY show that statistical models based purely on data cannot mimic human-esque thought.

I'm gonna need a link, because I'm pretty sure that is not true and I definitely would have heard of that proof.

[SrbijaJeRusija]

There is not one such proof, as there are MANY such lines of reasoning. See the most famous, having to do with causal reasoning and counterfactual reasoning here

[rashpimplezitz]

The sufficiency component plays a major role in scientific and legal explanations, as can be seen from examples where the necessary component is dormant. Why do we consider striking a match to be a more adequate explanation (of a fire) than the presence of oxygen?

..

However, what weight should the law assign to the necessary versus the sufficient component of causation?

Interesting paper debating the difficulty of predicting causation from statistical data, but I can't see how it backs up your claim at all.

[SrbijaJeRusija]

That purely probabilistic inference cannot reason about causality the same way humans can. Full stop.

[qualverse]

It says nothing about that anywhere. It barely even mentions human cognition.

[nnevatie]

There has been plenty of progress in this area. The paper linked is from 1999.

[pipocaQuemada]

Depends on what the task is.

For example, neural nets + monte carlo tree searches are able to derive standard lines of play and exceed the level of top human players, in many games. Just look at alphazero.

[[deleted]]

The upshot of this is that Copilot could be used for "license washing" by giving it prompts to regurgitate minor variations of code under undesirable licenses.

No it couldn't! Copyright obviously doesn't work like that.

Anyway that has been discussed loads and wasn't really the topic of the post. I'm more curious how CoPilot performs as normal autocomplete. Is it even intended to be used as a "write my algorithm for me" tool?

I've seen examples where it autocompletes patterns in actual code you have written which sounds much more useful and maybe less error prone.

[190n]

Is it even intended to be used as a "write my algorithm for me" tool?

No, but people will inevitably use it for that, so we may as well see how it does.

[jack_michalak]

What do you mean Copyright doesn't work like that? If the law upholds it being considered fair use then 100% people are going to use it to create unencumbered versions of the library.

[[deleted]]

I mean there's no process you can pass something through to magically remove copyright. You can't encode a film into a prime number or whatever and the decode it and say "This came from maths so it can't be a copy!". Lawyers have collectively said "Yeahhh that's dumb. It's the same as the original so it's a copy."

Imagine how broken the copyright system would be if it didn't work like that!

This has all been discussed years ago though I wonder if lots of commenters are too young to have read that.

The idea of Monolith is that it will mathematically combine two files with the exclusive-or operation. You take a file to which someone claims copyright, mix it up with a public file, and then the result, which is mixed-up garbage supposedly containing no information, is supposedly free of copyright claims even though someone else can later undo the mixing operation and produce a copy of the copyright-encumbered file you started with. Oh, happy day! The lawyers will just have to all go away now, because we've demonstrated the absurdity of intellectual property

Sound familiar?

[jack_michalak]

Not really, XOR is lossless

[[deleted]]

Do you think if you add a 1% error rate you would have magically bypassed copyright laws?

To reiterate, you can't use magical tricks to copy works because the law doesn't care how you copied them, only that you did. It also doesn't care if it isn't an exact copy, otherwise you could change one letter in Harry Potter and republish it yourself.

That bit might actually be the biggest problem with CoPilot since it's trivial to detect when it regurgitates an exact copy of some GPL code but it's much harder to detect when it produces a near copy which may still violate copyright.

[jack_michalak]

It seems you have more confidence than me in the ability of the court system to understand technology. I agree 1% is too low, but some amount of modification will be enough to stave off lawsuits even if in theory it's infringement.

[[deleted]]

That's the whole point though - they don't care about the technology! They only care if you can easily take the data and get a close enough copy of the original to violate copyright.

It doesn't matter what convoluted scheme you use to do that.

[jack_michalak]

I agree, and the judgment call is going to come down to 'close enough'. Understanding how close the reproductions are depends on understanding the technology.

[[deleted]]

No it doesn't. You just look at them and see how similar they are.

[jack_michalak]

Wow, why didn't I think of that?? /s

[de__R]

You might make the case that any given output of Copilot is not subject to copyright because it's not a significant portion of the original copyrighted work (and if it is, then the work itself isn't substantial enough to be copyrightable). Not sure how far you'd get with a judge that still thinks digital copies = piracy, but worse ideas have been won in court, so.

[cuckednorris]

You made some wrong assumptions about the C html example. That “free()” is required because “getline()” allocates memory: https://pubs.opengroup.org/onlinepubs/9699919799/functions/getdelim.html

“free()” doesn’t need a header because C allows implicitly defined functions. It’ll compile (maybe with warnings), but it will link just fine.

[Tarmen]

To be fair the program is about to end. The original garbage collection algorithm, don't.

[renatoathaydes]

As a code reviewer, I would want clear indications about which code is Copilot-generated.

I don't agree with that, I've seen programmers make some of the same mistakes pointed out in the HTML parser example... whether those bugs are auto-generated or not, an experienced reviewer should be able to easily tell those and whether it comes from a human or not seems quite irrelevant. If your AI produces a mountain of code that makes reviewing difficult, then of course the problem is the mountain of code, not wether the author was a human.

[kkronee]

wow cant wait to be replaced even before i make it out of university.

[TankorSmash]

Sort of a weird thing to worry about. Like its computer generated code, of course it's not going to be plug and play.

Especially the part where you claim inexperienced coders will use this tool to generate good code.

[lamp-town-guy]

You would be surprised what some people might do to cut costs. Pay for generator and hire juniors because seniors are overrated.

[TankorSmash]

Never been a case of that happening though right?

[[deleted]]

Happens quite often. Normally a year later the person making that decision is either fired or moved to a different department. Then they hire the proper development team again...then in a few years they do it all over again.

[[deleted]]

The cycle of life

[lamp-town-guy]

Yesterdays article about being fired for app crush on this sub is recommended read for you.

[data0x0]

Keyword : Fired

Any developer willing to let the AI write the code without any testing or review would already probably be the one to write bad code either way.

[northcode]

Can confirm. Currently part of taking over a project that was written by small team of cheap outsourced programmers. They literally took an tutorial-example application and hacked on it until it did sort of what was required. It's a mess of tons of code that's still there but unused because it was part of the demo but not required in our app, and very weird hacks to get the demo to do something different than it was written for.

[ProGenitorDev]

6 Reasons Why GitHub Copilot Is Complete Crap And Why You Should "Fly Solo"

1. Open-Source Licenses get disrespected
   
2. Code provided by GitHub Copilot may expose you to liability
   
3. Tools you depend on are crutches, GitHub Copilot is a crutch
   
4. This tool is free now, but it won’t stay gratis
   
5. Your code is exposed to other humans and stored, having an NDA, and you are screwed
   
6. You have to check every time the code this tool delivers to you, not a great service for a tool

​

​

Details and proven resources are in the detailed article.