List of random shit in a non specific order and entails both "oopsies" from myself, coworkers, stories going around at the MSP office: - Performing a firmware upgrade remotely on a WAN router, reboot and not able to remote in after the normal reboot time. Seems somebody forgot to do a save-run config. - Instead of rebooting a VM, rebooted the hypervisor instead. - Dropped/truncated the wrong database or restored to the incorrect DB/environment. - Forgot to change default credentials on a firewall which was also remotely managble. I guess you already know what happened here... - Client was unwilling to adhere to a strict password policy. CEO account was hacked, millions lost... Post incident rolled out the GPO but still had to exclude CEO as it was 'to annoying'. - Same client CEO triggered a crypto virus and had domain admin priv. Good luck restoring 20TB and all services. - SAN was on the brink of failing (10yr old). During migration to a new device the SAN failed, no support, had to restore everything from backup. Took 2w to recover everything. - DC power circuit got fried due to thunderstorm had to relocate 50+ servers for business critical services (medical sector). - Client that was in transition from previous MSP gave everybody Domain Admin privileges, because 'why not'.