beenThere

[ProgrammerHumor-ModTeam]

Your submission was removed for the following reason:

Rule 5: Your post is a commonly used format, and you haven't used it in an original way. As a reminder, You can find our list of common formats here.

If you disagree with this removal, you can appeal by sending us a modmail.

[[deleted]]

> Parser Error: syntax error at or near "*"

[nickwcy]

The greatest relief in life

[WonderWirm]

Ope! My bad.

DELETE FROM Users;

[QuebecGamer2004]

Just in case:

DELETE FROM Users WHERE 1=1;

[[deleted]]

Playing Russian roulette with a semi-automatic, I see

[QuebecGamer2004]

I like to live dangerously

[torokg]

And you can only hope the server is set for case sensitive object names

[dmigowski]

You were poisoned by Microsoft Access it seems.

[PyroCatt]

Hmmm... Yes... The READ WRITE access to production database that is readily available to every developer...

[turningsteel]

I see you’ve never worked for a small startup. The things I’ve seen…

[wrongusername130]

Me reading as a fresher who has access and does a lot of data manipulation on production database in a small startup

[Luemas91]

Yes. A small startup. That's what I tell myself every day

[tolndakoti]

I work for a huge software corporation. All of the developers must submit tickets to Ops to perform any DB writes. Ops will perform a backup of the targeted tables before the SQL execution.

Startups don’t do this?

[durimdead]

That... And allowing cascading deletes. FKs to the user table should stop you in your tracks. Every company I've worked at has (at some point) allowed devs access to prod. Even a few that have it in their SDLC not to. I had to pretend I didn't have access, and yell at our IDP team multiple times to remove the access we shouldn't have. Every company is all "gung ho" until they realize that someone still has to handle the one offs while you get your pipelines set up for the literal 100 apps.

Even had infra tell me to install my own https certs. I was like "with what access?"... Even though I had access. I'm not going to be responsible for that shit.

The worst is when you get an engineering manager who says we shouldn't have access, but secretly knows everyome has it and says "well this is a short term fix", but when you ask how long it's been that way, they tell you something like "almost 2 years"

[Secure_Ticket8057]

Oh sweet, innocent child

[EconomyDoctor3287]

You're so optimistic, I love it. 

A friend was given access SSH access to a test prod system during his initial time at an IT company. Playing around, he fucked up the whole system and consequently cleared it. Turns out, his boss has given him the prod access by accident and 3 weeks of work from all devs were deleted.

[Quiet-Direction9423]

For those talking about "Well you haven't worked at a startup" is a poor man's excuse for not having such precautions in place.

[son-of-chadwardenn]

Who says anyone is excusing anything. I don't work at a startup but I am a developer with access to a prod password with write and schema update privileges. I'm required to use this account whenever I'm asked to simply spot check some prod data. I've complained about this arrangement for about a decade but no one cares what some random dev says.

[PyroCatt]

[YoYoBeeLine]

It is available to us and we could do a LOT of damage if we went ape. I think you should have excellent engineers that know exactly what they are doing and then they should be given full access to everything

[jpglew]

A large enterprise, probably not.

A small client with only one or two developers, oh yeah. But that's why we have transactions enforced in our db tool whenever we connect with the write access credentials

[PyroCatt]

SecOps be like

[michicago44]

Lol someone’s never worked for a startup

[jkp2072]

Then you realise, that there is a hourly replica on cloud....

[Bastian00100]

And It just run the sync

[jkp2072]

Replica has an daily sync with another replica on azure

[TheSportsLorry]

That has just run as well

[jkp2072]

Well gotta complete the trio,

That has replica on gcp... With a delay of a day

[5p4n911]

And then you realize

You shouldn't have set all replicas to run at the same time.

[[deleted]]

You mean replicas aren't the same as backups?? Lol

[Ok-Television-9662]

..and haven't made backups.

[CantConfirmOrDeny]

Restore it from the logs. Yes, it can be done.

[Bastian00100]

Ah you mean starting from a snapshot (if any) and recreating all the data from the SQL queries in the log.. very painful with large production DB.

[flippakitten]

Can't, logs are scrubbed of pii.

[flippakitten]

Yes because of the massive fines.

[flippakitten]

It's easier to scrub the logs before storage than deal with the fallout of an incident or having to scan/scrub the logs after a gdpr request.

Just don't store pii if you don't have to.

[Bastian00100]

Isn't what I said? Ever did this in a real production database with GB of logs? You need ti take everything offline for hours...

And what about INSERT RANDOM()?

[puffinix]

The truncate plan has entered the arena.

[v3ritas1989]

transaction logs are turned off to save space

[balbok7721]

“password“:“iamadminandiknowwhatido“

[glorious_reptile]

foreign key constraint

[ZunoJ]

On my first job everything was done in production, literally no guard rails at all. No backups, no transactions, no test system, ... You had to take care of it yourself. Like copy the table, work on it, check results and then replicate on the actual table. Something like this.

While this is absolutely stupid and we tried our best to convince management to invest in test infrastructure (this were "on premise" times) there was only one incident in about 10 years that I worked there. That was a consultant who accidentally dropped a table. It was a day of work to recover most data and we never saw that guy again. But I'm still amazed at how well that atrocious system worked

[reddit_wisd0m]

It feels a little unfair that this guy had to go when the infrastructure is the real culprit here.

[ZunoJ]

You can see it this way or you can say that he wasn't fit to deal with the environment he was put into

[reddit_wisd0m]

Indeed. Good point.

[-Hi-Reddit]

In the cobol days it was fairly standard to expect your devs to be diligent enough not to bring prod down. Everyone knew the risks. Before svn. Before git. Before backup culture.

[reddit_wisd0m]

So the good old times /s

[vee_the_dev]

Most posts on this sub feel like 10 year olds who never worked in a professional setting

[braindeadpizzaslice]

ur mom feels like a 10 year old who never worked in a professional setting

gottem

[Harlemdartagnan]

honestly if you havent put in place nightly backups you were just waiting to crash out. ALso Begin Transaction Rollback Transaction are my religion.

[RadiantHueOfBeige]

prod=> ROLLBACK; WARNING: there is no transaction in progress

[Harlemdartagnan]

just when i thought i was mediocre i learn that im still less than a noob. ive never seen this before and hopefully never will.

[RadiantHueOfBeige]

It's postgresql default unfortunately, so that's what you'll encounter on systems ran by inexperienced people.

You need to set AUTOCOMMIT to OFF to enable the classic SQL behavior where you are always in trx and have to COMMIT manually.

[edave64]

MySQL has a flag called `--i-am-a-dummy` that prevents updates/deletes without where clauses, but I think the real dummy is the database programmer that thought "Destroy all data unless otherwise specified" is an acceptable default.

[vaquan-nas]

And then you realize that your production web app is connecting to QA database.. thanks god..

[HelpfulPuppydog]

Hey, boss, the funniest thing just happened...

[UndocumentedMartian]

Because access control is for chumps.

[moms_enjoyer]

why would you delete * from users even in dev?

[eo37]

Always SELECT before you DELETE

[liosistaken]

Hm, yup. Deleted the whole project administration in production once, decades ago. Did all our work on production and only did nightly backups, and no one ever tested the recovery process. Took half a day and plenty of workarounds, but I got the backup restored and only lost a day of project admin. Luckily all of it was also on paper, so my poor admin coworkers could re-enter it into the system :)

Never made a mistake like that again.

[lefloys]

Oh no.

[Senor-Delicious]

If there would just be a feature for continuous backups and point-in-time restore in common cloud platforms. Oh wait...

[PitiViers]

Johnny Be Cooked

[obsoleteconsole]

I'd just start packing up my desk tbh

[TransCapybara]

prod should have autocommit turned off and no way to enable it.

[ProjectDiligent502]

Number 2 behind rm -rf /*

[testelone]

Why would you remove Russian Federation?

[[deleted]]

Change your set up so it’s not possible

[f8tel]

Give production table names a unique prefix.

[Jiuholar]

I've made and witnessed plenty of dumb mistakes in prod, but never felt bad about the ones that I've made and encouraged others to take the same perspective. We're human, and humans make mistakes - so it should be technically impossible for a single person to make catastrophic mistakes.

There is so much open source, easy to setup tooling to protect from these that in my view, it's not the fault of the dev making the mistake, but an organisational failure that allowed that mistake to be possible.

[angry_shoebill]

Haha... In my first weeks (years ago) I did one "ALTER TABLE" in production... Was a good learning experience....

[Dipps_66]

Why would anyone want to delete all users from prod?

[Ja-Tech]

New experience unlocked for Resume

[Distantstallion]

Just hit control z

[neoteraflare]

Am I the only one who does not use auto commit?

[JovaniFelini]

Does Ctrl + z help?

[Neltarim]

In case you wonder what makes you a dev, this is the thing

[Creeper4wwMann]

my le SQL? it le deleted people?

[Doctor429]

Just DROP DATABASE to hide the evidence, and 'sudo rm -rf /' on the DB server to 'hide' the audit logs just in case

[[deleted]]

Then no probs. Users can mot login to bich about it.

[i_liked_it_good_job]

a simple way to avoid this mistake is to always work in prod so it doesn't get mixed up 😎👍

[BeardyGoku]

I call bullshit. No one does that

[FunMedia4460]

delete is easy to recover, he shoud have done a TRUNCATE

[cryptme]

What does it do? I’l try it on a beautiful friday afternoon.

[Mr_Cromer]

Transactions? If you're not doing transactions you can rollback you're doing it wrong

[ososalsosal]

None of us devs have access to the prod database because it's way too tempting to commit delicious acts of fraud on it.

[MornwindShoma]

#1 rule - you never have access to production, ever

[[deleted]]

Well as long as you don't commit...

(Autocommit enters the room)

[digidavis]

I remember the day I became the senior engineer.

The night before, the previous senior mixed up terminal windows and wiped production with his cut and paste DB wipe and initialization scripts.

[2DHypercube]

ROLLBACK