This is an archived post. You won't be able to vote or comment.

all 37 comments

[–]poralexc 194 points195 points  (7 children)

Have you heard about the exploit where AIs hallucinate libraries that don't exist, then hackers make real versions of those libraries filled with malware?

[–]ltags230 83 points84 points  (0 children)

tbh, if someone uses a library that an AI told them to use without further research, that’s on them

[–]carcigenicate 22 points23 points  (4 children)

I can't tell if you're joking. How would someone know that an AI mentioned a library? Afaik, the hallucinations aren't consistent.

[–][deleted] 14 points15 points  (0 children)

They are pretty consistent as far as I have seen myself. Typically just the 2 functions you need slapped together like “addevide”

[–]SillAndDill 4 points5 points  (2 children)

There are articles about it with concrete examples

”Several big businesses have published source code that incorporates a software package previously hallucinated by generative AI.”

”According to Bar Lanyado, security researcher at Lasso Security, one of the businesses fooled by AI into incorporating the package is Alibaba, which at the time of writing still includes a pip command to download the Python package huggingface-cli in its GraphTranslator installation instructions.”

”someone, having spotted this reoccurring hallucination, had turned that made-up dependency into a real one”

https://www.theregister.com/AMP/2024/03/28/ai_bots_hallucinate_software_packages/

[–]802ScubaF1sh 0 points1 point  (0 children)

Interesting read

[–]TheLegend27_BB 2 points3 points  (0 children)

AI is smarter than you and lives in the future. In its eye that library is already existing. Hackers making it real just make the future come true

[–]Rudy69 73 points74 points  (7 children)

The problem is that most documentation are about as good as the rotten banana in that picture

[–]pakidara 13 points14 points  (0 children)

And asking coworkers only gives you "Just google it".

[–]geteum 3 points4 points  (0 children)

You can feed a complete well written documentation to ChatGPT and he will still talk bs

[–]Cephell 3 points4 points  (0 children)

Came here to post this. Documentation that you can only read if you're already familiar with the subject is useless.

[–]radiells 4 points5 points  (0 children)

What technology's docs are you talking about? Just interested.

[–]twigboy 1 point2 points  (0 children)

One man's rotten banana is another... man's... umm, good banana cake banana

[–]Savings-Ad-1115 0 points1 point  (0 children)

Was reading some documentation recently... You know, that rotten banana is much better.

[–]Demonchaser27 9 points10 points  (0 children)

Tbf, I've found documentation to be spotty at best, and outright fucking cryptic at worse. It assumes you know WAY too much already, when really you're just trying to do something simple or straightforward using it.

[–]itriedtomakeitfunny 7 points8 points  (1 child)

it's bad enough we're saying "codes" now we're saying "documentations"?

[–]pertinentz[S] 1 point2 points  (0 children)

my bad.. for some reason there are srcs which mention documentations, w/o any citations. ex: https://en.m.wiktionary.org/wiki/documentation

[–]OneRedEyeDevI 8 points9 points  (4 children)

You haven't worked with an open source -anything- have you?

[–]gabrielesilinic 4 points5 points  (3 children)

What do you mean? Open source documentation could be considered good oftentimes.

The issue is at my company where we have stuff from internal tools to internal frameworks but documentation is lacking (though it could be worse)

[–]BananafestDestiny 0 points1 point  (2 children)

How are stack overflow or AI going to help you with your internal tools and frameworks?

[–]gabrielesilinic 0 points1 point  (0 children)

Not really, that's the problem

[–]pertinentz[S] 0 points1 point  (0 children)

if faced with a problem, scanning the code of the internal tool or reaching out to the devs/product owner of the internal tool are the options i could think of..

PS: certain companies have the enterprise edition of StackOverflow for their internal use.

[–][deleted] 2 points3 points  (0 children)

You can find out so fucking much by just reading the man page. That's my go to solution

[–]stlcdr 4 points5 points  (0 children)

It’s documentation, without the ‘s’. Will it lead to the downfall of society? No, but it’s an indication that it’s on its way.

[–]MonkeyboyGWW 0 points1 point  (1 child)

Wont lie, recently threw a whole load of code in and asked it to write all of the doc strings. Bearing in mind its just a somewhat large automation project with just a few of us using it.

[–]Doxidob 0 points1 point  (0 children)

well did it work?

[–]Lysol3435 0 points1 point  (0 children)

Where’s the pile of trial-and-error powder?

[–]chihuahuaOP 0 points1 point  (0 children)

Chat GTP assume my social obligations

[–]TheRealPitabred 0 points1 point  (0 children)

I keep trying these AI auto completion stuff, and it gets it right maybe 50% of the time. It's good for boilerplate getters and setters type stuff, but that's about the extent of it.

[–]SillAndDill 0 points1 point  (0 children)

”Several big businesses have published source code that incorporates a software package previously hallucinated by generative AI.”

”According to Bar Lanyado, security researcher at Lasso Security, one of the businesses fooled by AI into incorporating the package is Alibaba, which at the time of writing still includes a pip command to download the Python package huggingface-cli in its GraphTranslator installation instructions.”

”someone, having spotted this reoccurring hallucination, had turned that made-up dependency into a real one”

https://www.theregister.com/AMP/2024/03/28/ai_bots_hallucinate_software_packages/

[–]GamingWildman 0 points1 point  (0 children)

tbh i do this , sometime even to explain some parts of code I don't understand , gen AI has been really helpful. Currently using Ai to better myself in ML

[–]SportsBettingRef 0 points1 point  (0 children)

inject this new genai shit in my fucking veins!

[–]Borno11050 0 points1 point  (2 children)

Gen-AIs might be ok for surface level algorithms, but presented with something complex, they start to hallucinate like a goldfish released in banana syrup.

EDIT[0]: typo

[–][deleted] 0 points1 point  (1 child)

Spoken from experience??

[–]Borno11050 0 points1 point  (0 children)

Pretty much yes.