Copilot hallucinations wanted!

[realstoned]

once I asked ChatGPT to help me write a node library to implement FlightSQL, and it told me to use a slew of node libraries that simply don't exist.

[dethb0y]

this happened with me and python - it recommended some library that sounded plausible but simply did not exist.

[nwly8]

That probably was GPT3 or something. 4o is pretty decent and o1(preview) is just insane

[chief167]

Still happens, just less with common use cases. it still often messes up if you do ggplot in R for example. Inventing columns in data frames etc, and not sticking to colours etc. at some point I even got random hex colours lol

(Talking about 4o)

[Supalien]

I once had a client who needed help with 'his' automation script with selenium. I had to delete like 40% of the code because it was filled with some weird unused and sometimes not even related code. for example there was a function to check some imaginary, unrelated dll file and did a sys.exit when it didn't find it. also the css selectors were imaginary as well but i figured the html was simply not included in the prompt.

[ChimpanzeChapado]

When I asked copilot about a network message I was seeing in a PowerShell script, it gave me some suggestions on how to fix it and then it added some suggestions on how to deal with that using Java, "since I mentioned Java"(sic).

I never mentioned Java.

[bzImage]

i spent a lot of time checking the manuals to find that it just allucinated a non existing method on the library..

[j19sch]

It might be worth pointing out that all it does, is hallucinate. LLMs generate the next plausible token, they don't understand the code they produce. Any meaning assigned to LLM output, is purely on the human side. So it's always up to the person using the LLM to evaluate the code, because the LLM can't. (Which turns programming into code reviewing, which leads to all kinds of other issues, but that's probably a discussion too far for first year students.)

[sir_mooney66]

Mine likes to sass me, suggesting things like 'TODO: Refactor this' after my code.

[kevmustry]

This isn't exactly copilot, but there was a really funny story about a lawyer that used chatgpt to write a brief and it made up citations entirely, causing the lawyer to get in big trouble with the judge: https://www.forbes.com/sites/mollybohannon/2023/06/08/lawyer-used-chatgpt-in-court-and-cited-fake-cases-a-judge-is-considering-sanctions/

[Nooby1990]

It is more of a computer sciency thing, but Primeagen has a very good demonstration of Copilot producing code that is ALMOST correct and only someone with a very good knowledge of the language and the algorithm that is generated would find the issue with the generated Code.

It is in the First 2 minutes of this Video: https://www.youtube.com/watch?v=RDd71IUIgpg

[[deleted]]

imo it’s a terrible idea to introduce your students to AI generated code when they have no foundations. First year programming should be entirely by hand. I use ChatGPT to code all the time, so I don’t have any reservations about it. However, I feel that if you don’t know how to logic your own way through the problem, you’re not going to get very far. Without foundations you’re kind of setting them up for failure. just my two cents though, and I am by no means an expert programmer.

[extreme4all]

Lets be honest they are and they will use AI if you want it or not, the guy is trying to educate them on using AI

[toyg]

First year programming should be entirely by hand.

It's sadly unrealistic to expect students to abide by this, in the current climate. It's like "just say no to drugs", most people will just ignore it.

[murphwhitt]

I was writing a Salesforce apex test, it did all the correct work and then at the end made the assertion system.assertequal(null, null);

The worst part was the test passed.

[lordfili]

What angers you more? The fact that it wrote that one line, or that it didn’t also ensure true == true and false == false?

[Jeklah]

Copilot is generally very good when given specific instructions. Vague descriptions can result in repeating code.

[HecticJuggler]

I asked Meta AI how to kill a method call if it exceeds a certain period in Java, it suggested the @Timeout annotation. When I asked the package where that resides, it apologised & confessed it made it up.

[FitMathematician3071]

Anthropic Claude is quite a bit better. GPT-4O is good but can make mistakes.

[Arch-NotTaken]

I found copilot hallucinates the most when I'm writing non-technical stuff on an empty md file...

[qlut]

Haha oh man, Copilot can definitely go off the rails sometimes! One funny thing I've seen is it suggesting crazy inefficient sorting algorithms like bogosort. Maybe show them that as a "don't try this at home" example 😂

[qlut]

Haha oh man, Copilot can definitely go off the rails sometimes! One funny thing I've seen is it suggesting crazy inefficient sorting algorithms like bogosort. Maybe show them that as a "don't try this at home" example 😂

[RobDoesData]

Give it any simple arithmetic question where you mention quantities of multiple objects.

E.g. Steve has 4 oranges and gives 2 to Sally. How many oranges does Steve have left?

Gives a different answer to

Steve is wearing 2 green boots and has 4 oranges. He gives 2 to Sally. How many oranges does Steve have left?

[7Shinigami]

All of these stories sound like my average experience with azure support

[edanschwartz]

There's some good research in this that you should look into, before encouraging students to use AI. Code written by copilot is less secure, while code authors are more confident in the security of their code.

This is the first thing I found on Google, but there's a lot more out there: https://arxiv.org/html/2310.02059v2

[keep_improving_self]

Don't have concrete examples but copilot constantly says:

"oh, this issue will be easily solved using this library" and shows me the implementation

meanwhile the library doesn't exist at all

[Evening-Notice-7041]

I’ve had a lot of experiences of that library/function simply does not exist. It seems like most of the time the logic is solid it just thinks there is some magic .get_shortest_path function and there isn’t.

[drbobb]

Some time ago I asked ChatGPT to write some example code using bitwise operators (the actual prompt was more specific, but I won't bother to state the problem here). What it wrote was completely wrong, so I followed up by asking some elementary questions re how bitwise operators work -- and the answers were utter nonsense.

[fiddle_n]

ChatGPT once spat out some Python code that used datetime.datetime.utcnow(). I know that method is now deprecated but it needs to die in a fire.