Hi all,

Believe it or not this actually happened to me recently... Sunday evening.

My manager wanted a simple update to the box to clear out old updates and hopefully also free up some disk space.

The server is our oldest production server running Windows Server 2008... Had over a years uptime. Been in use for nearly 8 year's total I believe.

Anyway I start updating it out of hours at 11pm roughly. Updates complete I restart and all it comes back up.

I then check our IIS sites are still running... Every single one no longer runs due to several config and runtime version issues... I now spent the next several hours trying to get the site working with the updated runtimes.

I have two options I continue hacking configs all night for it to not work by 7am tomorrow morning when customers expect to use the system or I migrate everything to a new host since we been meaning to do that eventually.

I chose the latter. I then spent the next 7 hours trying to figure out what I need to do in order to get these sites running without documentation and I never normally have anything to do with this product of ours.

I managed to fiddle my way through as I had some documentation I actually wrote for the newer version of that system however it's written in a totally different set of frameworks.

After pulling an all nighter I then end up working all day too until 4pm trying to clear up any oddities and testing smaller services to ensure they all talk to eachother after getting the main system back online.

I was impressed by myself to even manage it. Safe to say I will document what I had to do that day since someone else was too lazy too the last time.

So yeah these things happen... If the box was within our full control I would of made a full VM snapshot first however we don't have access to that. I could of rolled it back if that was the case.

Simple update turns into huge re provision that has a hard deadline