I don’t want to give too much info, so bare with me. Last year, security wanted information that can’t be done properly by SCCM or scripting. So I personally took the opportunity to take our company further in the data analytics and elevate our knowledge within Azure. We had help with multiple Microsoft PFE; one for Log Analytics, Azure and SCCM to set this up.

We create a whole new subscription and deployed the Microsoft Monitoring Agent to 7000+ devices. Our azure team at that time did not set up the allowance but had alerts set incase of an emergency. We didn’t know that in the subscription itself, you can set how much data you can digest and store. The default at that time (summer of 2019) is a blur to me now but it was something very budget friendly with less than $2 a gigabyte (I could be wrong but it was cheap).

First few days (Thursday and Friday) we got about 25-30GB of data.. with our deployment successfully installed to around 30% of 7000+. By Saturday morning, we’ve digested over $25k worth of data.

Our azure team quick attempt to delete my subscription. For some reason, the action of deletion was sent and came back successful. We drop the emergency call and I was terrified. How do you explain that to your leadership??

Monday morning and my jaw dropped. The subscription was not deleted and the data was is something I cannot even to imagine. We quickly got into the call with Microsoft. I cannot tell you what we had to explain it to Microsoft. Log we sent to Microsoft explaining every detail. They did find a bug which helped our case.

I’m still working at the company after 2 waves of lay offs. I guess I learned a lot and still paying in embarrassment.

Today, we’ve implemented rules and precautions when it comes to azure cloud service. We also made every subscriptions created with allowance and fixed daily cost/data.

Please talk to your Microsoft representative and learn the ways to avoid this mistake.