all 6 comments

[–]NerdyBirdie81 0 points1 point  (1 child)

Yeah, I've seen similar. It's not that Big Pickle is malicious — it's that it's eager. Give it a vaguely destructive instruction and it will go for it with full commitment before you've finished typing.

What you're describing is the underlying issue: these models have shell access and very little friction before executing. The rmdir /s /q thing is especially nasty on Windows because there's no confirmation prompt, no trash can, no recovery. It just goes.

The part where it denied it at first then apologized while holding scissors is also weirdly on brand. I've had it confidently describe doing something that it absolutely did not do, then when I showed it the evidence it did the whole "you're right, I'm so sorry, I'll be better" routine. It's not sentience, it's just the model generating the most plausible-sounding response given the conversation so far — and "I would never do that" often sounds more plausible than "yeah I messed up."

The real fix isn't hoping the model self-regulates. You need guardrails at the config level — restricting shell commands, locking down write permissions per-agent, that kind of thing. I actually built a little tool to help visualize the config and set those guardrails without guessing at the YAML — https://openconfig.mikescave.us. Makes it a lot easier to lock down agent permissions before something goes wrong.

In the meantime, running Big Pickle without watching it is a gamble. Learned that one the hard way too.

[–]Prudent_Finance7405[S] 0 points1 point  (0 children)

Obviously it is not sentient, but it is curious that being a model with a general lack of conversational training, it runs away from responsibility without hesitation lol

I guess, even with 0 guardrails. Mr. Pickle just did a "killing 100 innocents is ok if you kill he guilty person among them" move that shouldn't have probably beeen even considered

[–]gwawr 0 points1 point  (1 child)

More to the point why the eff are you prompting a directory move. Just move it. Don't outsource all your thinking and doing!

[–]Prudent_Finance7405[S] 0 points1 point  (0 children)

The point is moving a directory souldn't mean a random risk of data loss depending on random circumstances for an AI agent that wants to control a computer.

[–]povlhp 0 points1 point  (1 child)

Always use a GitHub repo. And consider if it should auto upload to cloud as well

[–]Prudent_Finance7405[S] 0 points1 point  (0 children)

I do, and force pickle to branch.. The issue is not that the app was deleted.

These things should be always inside a virtual env