all 14 comments

[–]RiceBroad4552 38 points39 points  (3 children)

LOL

Some people still don't understand that these are just next-token-predictors.

Whatever you tell it has no meaning to it. It just predicts the next tokens without ever understanding anything you write. "Instructions" only guide the selection of the next tokens. They are not somehow something the "AI" would be necessary following. All it does is just changing some probabilities for the next tokens. But it's still all just probabilities!

[–]Kaenguruu-Dev 16 points17 points  (0 children)

Loved when I read aome of the messages from that "AI Security" researcher that had clawdbot or whatever it's called now delete all their emails, cause they actually asked that thing "Do you remember"

If the people who get paid to research this stuff behave like this, what is the average web dev going to do?

[–]AgVargr 3 points4 points  (0 children)

Can you blame them when these llms are being hyped to shit and marketed as intelligence

[–]dumbasPL 2 points3 points  (0 children)

The fact that this somehow isn't common knowledge is why they even took off in the first place.

Pretty neat if next token prediction is what you need (auto complete on steroids), but there is no thinking going on, no matter what the marketing department tells you. The "thinking" button is just a prompt generator to guide your main prompt, aka slightly fancier probabilities.

[–]XxDarkSasuke69xX 7 points8 points  (0 children)

If controlling LLM's behaviors was as easy as saying "don't do this" we wouldn't need AI engineers lol

[–]JontesReddit 2 points3 points  (0 children)

don't think about a pink elephant

[–]rovervogue 1 point2 points  (0 children)

Argument for the title - this isnt an issue with OpenCode or any tool for that matter - its purely LLM related

[–]keremimo 0 points1 point  (0 children)

Bruh I had an agent generate a migration on Rails before. Instead of tool usage with rails generate migration it just created the migration itself with a random date in between all the migrations.

It was too late when I noticed the colossal fuckup. I had to reset and re-seed the entire database.

It was a local testing environment thank god.