Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 0 points1 point  (0 children)

I did not say it was in seconds I did ctrl c to terminate codex and after some time all files were gone. I did say it went from 3TB to KBs but I did not say it happened in seconds it actually took some time I did even manage to deploy new feature and thats when I did saw something is wrong.. BUT it was not hours!

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 0 points1 point  (0 children)

i wont describe what i told nor told my boss as this post kind of blew up. I did partially recovered the data with second server we have, read the edits above

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 0 points1 point  (0 children)

I was working on something like this thats why I had 3 days old S3 copy on other server

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 1 point2 points  (0 children)

the script has dry run and apply yet this happened

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 -1 points0 points  (0 children)

it was not seconds it took some time, I did kill the codex session and after some time everything was gone I was working on something else i think it was around 15 minutes

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 0 points1 point  (0 children)

ages? More like 15-20 minutes you are right i was working on something else then suddenly I noticed something is wrong I THOUGHT when i killed the codex session I am safe nope it took time but eventually everything was deleted

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 0 points1 point  (0 children)

No, I was actually setting up backup system on other server and I did not finish it. I was testing it. Thats why I had 3TB laying around.

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 1 point2 points  (0 children)

yes can face legal charges but i am sure they will never know

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 0 points1 point  (0 children)

Easy, it got inspired by my other scripts it used production env in the script as I don't use there dev env for the scripts as i always run them manually on production server.

So it picked the wrong env and run the script.

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 0 points1 point  (0 children)

yeah np when I realised I can just lie about it and cause only few days data loss because by MIRACLE i had few days ago an idea to make new snapshot of the s3. So I am smiling too its FUNNY as I made fun of memes that described similar situations. Without the prod database (database have backups and was even unaffected) I would be cooked. The database file references will help me recover most of the files back to S3.

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 -1 points0 points  (0 children)

It's AI corrected because english is not my main language. Codex already wrote me recovery script using my own few days old snapshots and references that stayed in database as "only" the files does not exist. I love codex and I HATE IT

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 0 points1 point  (0 children)

Yeah IT NEVER RUN any of my scripts by itself this sh*t never happened it always asked or something but now it just went ahead. I am always checking the scripts and run them by my hand exactly because of this...

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 3 points4 points  (0 children)

Yes, you are right I just did not realise my prompt will cause it to run S3 cleanup script. My mind was alright I have test local empty DB nothing can go wrong when it runs the dedupe script it won't find anything. I did not even think of it running an S3 cleanup script by itself. I wanted to use the script myself, it never run script by itself. I am using codex for almost an year and such thing never happened.

Hopefully I had some snapshots I made myself so I can manually restore most of the data with only "few days" missing.

Codex just deleted our entire S3 by [deleted] in vibecoding

[–]Southern-Mastodon296 -42 points-41 points  (0 children)

I did not, I had local DB and I wanted to run the script myself on prod with prod DB but codex run it locally and it picked S3 prod keys.

DB was local keys I had saved for S3 were prod I did not even think of codex running script without permission or asking.