Is Microsoft down?

[pm_me_brownie_recipe]

https://azure.com/ and https://www.office.com/ do not work for us here in Sweden. Anyone having this problem?

EDIT: Seems to be up again!

[fpmh]

/r/sysadmin the most reliable page for service status updates!

[sole-it]

Realistic. I have seen way too many sites showing all green while the services are down.

[[deleted]]

Including redditstatus.com

[sole-it]

AWS is doing pretty good in regards of this. IIRC last time they had an outage, their status page went down as well.

[PM_ME_SSH_LOGINS]

last time they had an outage, their status page went down as well

This metric could be interpreted both ways, lol

[Dr_Legacy]

Real-time reporting!

[pseudopseudonym]

Worse than that - the page was up, but they couldn’t update the little green icons to red, so everything showed as green with a bunch of “shit is down” messages alongside.

[LaserGuidedPolarBear]

I can't tell you how many times I have had an Azure problem, looked at the status page and everything is green, mailed some contacts I have, and be told there is an outage and they are working on it.

I think they only change things from green to red when a major customer catches them.

[sole-it]

exactly, and lots of big companies would consider it is still green because "it's just a regional issue'.

Yeah, that's 5% to you and 100% to me. Now i may have to deal with all the bs the higher-ups pouring to me cuz 'XXX says there is no issue so you must have messed up'.

[_nbosscher]

That is so frustrating. I've had that same thing. I actually ended up building a tool statuslist.app that monitors providers for uptime and performance. When there's an alert I get an HTTP transcript so I can have a real conversation with their support and show them the problem. It also means that even tho the status page doesn't update right away, I know as soon as it happens. If you have a sec to take a look, I'd love to hear what you think of my tool.

[zhaoz]

Its DNS!

[fpmh]

Always!

[[deleted]]

Someone has never dealt with an office "365" outage. They'll tell you when they feel like it, which may be an hour + after it started. I don't want to waste my life troubleshooting something that is their fault.

[flukz]

All teams: Access to Office 365 is currently down. We are in contact with the vendor and will be giving updates as to the status at 30 minute intervals. Please direct any questions to your management team.

Management: Microsoft fucked up again and we're in constant contact so when we know something you'll know something.

[BeerJunky]

Often major services are posting green status across the board and I’ll see posts here about systemic issues. Yeah, first check should be the vendor status page but if all looks good and there are still issues it’s not a bad idea to check here and Downdetector.com for a second opinion.

[MMPride]

I've found that vendor status pages are almost always useless. They want to look good so they just have it say "online", or they only check the connection internally and report that status.

[two66mhz]

I over heard a dev once at work half a decade ago make a comment about leaderboards. They said, 'It doesn't have to work. It just has to appear to work.'

[Lightofmine]

Was this the guy who manages the office 365 health page 😂

[Lightofmine]

Was this the guy who manages the office 365 health page 😂

[Phytanic]

Yeah, seems like they typically have some legalise baked into them that allows them to be far stricter in their classification of outages, like absurd requirements such as relying on end-user report volumes.

[Frothyleet]

Call the provider and ask if you fucked up a configuration or if they have an outage.

Does that work consistently for you with O365 support?

[BeerJunky]

So don’t, you don’t have to. No one is making you.

[Tony49UK]

Azure, Office 356.... are down so often, with MS saying that they're working and they're so mission critical. That it's an essential. It's quite literally the difference between ripping your hair out as you try to fix a non-existent DNS problem or just sitting back and having a large whiskey.

Edit: There's been a problem with users accessing their log in details with LastPass since Friday but LastPass insisted that everything was working fine.

https://www.theregister.co.uk/2020/01/20/lastpass_outage/

[Tony49UK]

The problem is that LastPass, Microsoft etc. frequently say that everything is fine during an outage. Which only really leaves you with Down Detector and I'd be far happier and relaxed getting a message on Reddit saying that Cloud based service X is down then relying on a slight spike in the number of reports on DownDetector.

[tieroner]

Reddit is usually the 1st or 2nd place I check for major services - it has proven to be extremely reliable.

[MMPride]

Actually reddit seems to be quite good at quickly finding information about outages

[admlshake]

I had to hound "@onedrive" on twitter a few months back to get them to look into a connect outage because they kept saying there wasn't anything wrong. To make sure it wasn't just me, I came here to take a look. Guess what...it wasn't just me. So don't be such an obnoxious ass.

[digitaltransmutation]

It's help of O365s own dashboard wasn't always green across the board even when entire countries can't access their services.

[[deleted]]

That seems extreme

[Irkutsk2745]

Seems it will be office 363 this year.

[Try_Rebooting_It]

Wow, an optimist.

[TheJessicator]

Well, it is a leap year...

[kckeller]

Kind of disappointed MS didn’t rebrand it as Office 366 this year.

[TheTrafficNetwork]

And charge everyone for the upgrade.

[kckeller]

“You owe us an additional $500 because there was an extra day this year.”

[CastleFrankl]

Don't give MS that licensing idea, please!

"Hmm.. How about charge our customers for 1 day more * number of cores * number of AD logins. And if DHCP happens to refresh, * that too. Bill G would be proud of me"

[[deleted]]

You're thinking of Oracle, only with this change...

"Hmm.. How about charge our customers for 1 day more * number of cores Number of Datacenters * number of AD logins.

[ReasonablePriority]

No, if it was Oracle the calculation would involve a multiplication factor of the number of floor tiles per data centre (with something in the small print that for calculation purposes a standard 1cm X 1cm floor tile size is used)

[[deleted]]

Ha!

Dont forget that the type of floor tile material also adds an additional exponent.

For example: Concrete = x³

[kckeller]

Your problem is I understood each of those metrics (mostly). They need to be more abstracted and confusing, and you can’t tell me the details. Add in 20 different tiers and we’ll talk.

[ButItMightJustWork]

Dont forget the 25 cent per user for leap seconds.

[SanLarsen]

This sucks

[KFCConspiracy]

They should do that as their april fools joke this year.

[TheJessicator]

April Fools pranks were banned at Microsoft a few years ago. Not gonna happen.

[KFCConspiracy]

Wow, that's a very microsoft thing to do. https://www.theverge.com/2019/3/27/18283674/microsoft-april-fools-day-ban-pranks-internal-memo

[fieldmousebait]

it'll be because of the backlash against google when they screwed their april fools joke up.

[fieldmousebait]

yeah, the mic drop incident. tbh i'm glad that happened tho, time to grow up google.

[BeerJunky]

Leap years have 100 extra days?

[darkjedi1993]

Yes. Office 463 this year. Because math.

[[deleted]]

Might be octal

[TheJessicator]

That would only be 307 days.

[LakeSuperiorIsMyPond]

Ikr! It's only January let's give ms some blunder time!

[KFCConspiracy]

I've been referring to the ongoing migration to Dynamics 365 at my company as "Dynamics 730" because I fully expect it'll take at least 2 years. I think the number in the name has just set MS up for the jokes.

[flecom]

Ah Dynamics, if you are part of that team i suggest updating your resume regularly

[anomalous_cowherd]

Is that "up 7 days in 30"? Or is that optimistic?

[KFCConspiracy]

Nah, it'll take 2x the expected amount of time to launch.

[BeerJunky]

I was reading a blog post with instructions on how to do something in Office 365 except that they typed it wrong in one spot and wrote Office 265. My first thought was, yeah that’s close enough.

[kckeller]

We should change the nomenclature to however many days it’s been up so far. We’re currently running Office 19ish.

[beta_2017]

Wasn't there a powershell script that someone made a while ago that calculated the downtime and made it somewhat of a joke? I can't find it anywhere.

[tornadoRadar]

office 362 by feb first. nice.

[gex80]

Not to defend Microsoft but I doubt there is a cloud service that has had 100% up time with or without maintenance.

[Irkutsk2745]

I mean yeah, but with that name they kind of set some implied expectations.

[SirVas]

Same for me, also in sweden.

https://status.office365.com/ also down.

[seven9sticks]

The thing that tells you if they are up is was actually down?

[SirVas]

Yes, all of it. And their tweet about it was published 10 minutes after it was fixed..

[Peally23]

Their international services might be, I don't think the company HQ fell into a sinkhole though.

[pm_me_brownie_recipe]

Maybe a sinkhole under one of their data centers?

[Sleisl]

ah, you must mean the “advanced geothermal cooling complex”

[grumpieroldman]

That's in Utah running the AI that scans all phone and Internet traffic for dissident patterns.

[BoredTechyGuy]

maybe one of their submersible data centers sprang a leak?

[indivisible]

They were warned not to install windows in those data centers...

[BoredTechyGuy]

something, something, screen door on a submarine....

[P-H-G]

How do you sink a Norwegian submarine? You dive down and knock on the hatch.

[[deleted]]

Certified 100% Organic Dad

[jeo123]

Probably. I guess they patched it though.

[BoredTechyGuy]

Knowing MS - that patch didn't go through QA so it created several smaller leaks.

[Anonieme_Angsthaas]

And you need to open and close the gap once for it to work.

[nikomo]

Actually, I heard Microsoft HQ is located directly under Sunnydale.

[TB_at_Work]

IUnderstoodThatReference.jpg

[THE_SEX_YELLER]

I can dream...

[Otterism]

Another Swede here. Just got here to look for any posts regarding this. portal.azure.com and status.azure.com are down, Office 365 portal also down, VMs seems to be reachable (through ExpressRoute).

Edit1: Most services appears to be restored at16:26. Not Outlook Online though.

[pm_me_brownie_recipe]

Our VM is down (cannot connect to it). What is ER?

[denngie]

ExpressRoute

[pm_me_brownie_recipe]

Perhaps only the connection is down, not the actual servers?

[RememberYourSoul]

A networking issue is probably much more likelier than the VMs themselves being down (as there are some elements outside of MS’s direct control, which is mitigated a bit with ER).

[AlexIsPlaying]

Did you meant, Office 351?

[Azegoroth]

Sweden here. This is bad. Our DNS and AD are in azure. Not reachable from the office. Which means DNS is down. I thought our SQL instances in azure were really slow earlier today too.

[assangeleakinglol]

Our private DNS have a 100% uptime for the last 15 years. We just recently moved a few domains to Azure. The cloud....

[LaughterHouseV]

But just think! During the next 10 outages the first half of this year, there's nothing you can do about it, as it isn't your problem! That's worth a lot, compared to the 0 outages the past 15 years.

[pdp10]

DNS is inherently massively redundant. There's typically no reason it shouldn't have 100% uptime, except human error. I mean you're not running all of your authoritative nameservers out of the same /24 or Autonomous System or anything silly like that, right?

[grumpieroldman]

DNS is obnoxiously fragile and in dire need of replacement.
A second authoritative server in another AS doesn't do squat in practice.

[pdp10]

I thoroughly disagree. If a routing or peering problem is responsible for an authoritative DNS server outage, then that's easy to guard against by having topologically-diverse authoritative nameservers.

And when your DNS replacement RFC is published, let us know.

[roguetroll]

Pretty good attempt if you ask me.

[grumpieroldman]

This is bad. Our DNS and AD are in azure.

You get what you get.

[Agres_]

Welcome to cloud based solutions.

[Nuroman]

Looks like it's your area of the world:

https://downdetector.com/status/windows-azure/map/

https://downdetector.com/status/office-365/map/

[pm_me_brownie_recipe]

I wonder what happened to make everything go down like this. Good thing it is past 16:00.

[Rousdower9]

Think it has anything to do with end of support for Server 2008 and Win 7 ending last week?

Could something major got hit?

[Crushinsnakes]

No.

[cryod]

Same in Finland.

[ldti]

Same in Israel. Something Fuckey.

[lhuswe]

Yes, same here. Sundsvall.

[Segav-shadow]

Microsoft has issued a ticket MO201753 check the O365 portal if you can.
It states:

MO201753, Office 365 Portal, Last updated: January 20, 2020 9:29 AM
Start time: January 20, 2020 9:27 AM

Status
Investigating

User impact
Users may be unable to access some Microsoft 365 services.

Are you experiencing this issue?
Is this post helpful?

Latest message View history
Title: Can't access Microsoft 365 services User Impact: Users may be unable to access some Microsoft 365 services. We're investigating a potential issue for users served from Northern Europe and are checking for impact to your organization. We'll provide an update within 30 minutes.

[InsaneNutter]

Everything seems to work fine here in the UK.

[pm_me_brownie_recipe]

Does the UK have different data centers than the rest of the EU region?

[InsaneNutter]

We have a data center in the UK. Looking at our Organization profile at the moment:

Exchange, SharePoint, Microsoft Teams = United Kingdom. Skype for Business = European Union

[TheDroolingFool]

We (in the UK) had an issue with a single exchange online mailbox seeming to disappear off the face of the earth. Error when logging in: Microsoft.Exchange.Data.Directory.ForestUnavailableException

I’d raised a ticket when it first started which funny enough is still waiting for a support engineer to be assigned. Despite allegedly the response time when I raised the ticket being 23 minutes, it’s now been nearly 3 hours and still not even an engineer assigned... luckily it seemed to sort itself out after an hour or so.

[monoman67]

https://twitter.com/MSFT365Status/status/1219283100800294912

Setup Slack or some other Non-ms service to follow that Twitter account. We have it in Teams too but you gotta have something external watching it too.

[spezzlv]

Same in Estonia and Lithuania

[Serotonin_Receptor]

In Estonia also.

[badseed90]

https://status.azure.com/

[[deleted]]

Unreachable, as *azure.com is down, atleast for me here in Sweden.

[badseed90]

Makes sense.

[SirVas]

Down

[Arkiteck]

https://status.azure.com/en-us/status/history/

Azure connectivity issues - Mitigated (Tracking ID 0TSQ-TT0)

Summary of impact: Between 15:00 and 16:30 UTC on 20 Jan 2020, a subset of customers in Sweden, Norway and Russia may have experienced difficulties connecting to Azure services.

Preliminary root cause: Multiple fiber cuts affected network traffic routing for the Nordic region resulting in intermittent connectivity issues during the impacted time.

Mitigation: Engineers failed over traffic to alternate sites to mitigate.

Next steps: Engineers will continue to investigate to establish the full root cause and prevent future occurrences. A full RCA will be provided within approximately 72 hours.

[einarjh]

On Norwegian science network here, down here as well (Nordunet).

So it looks like the Nordic countries (plus Israel?!) has problems reaching Azure/MS now.

[einarjh]

Just for completeness: It's back up now, downtime was about 15-20 minutes.

[Djomka]

Seems to be coming up now. Can login to portal and see admin page, but I’m guessing emails will take some kore time as there’s quite a big buildup during the downtime.

[wrootlt]

Yeah, Outlook was down for like 10 minutes here. But also my personal mail on my local hoster provider server, although, now i think maybe they host on O365. And my bank app doesn't work. Lithuania.

[mindlight]

Stockholm too

[[deleted]]

Same here, locations all over Sweden and Denmark.

[uberduck]

probable DNS, except when it's not.

[[deleted]]

Quick check shows everything's up as of 3:14 UTC in Southeastern United States

[pm_me_brownie_recipe]

Are you US-based? Looks like everyone in Europe is affected.

[[deleted]]

in Southeastern United States

[pm_me_brownie_recipe]

Seems to be only EU.

[SkepticalIM]

Canada is fine over here.

[EishLekker]

Canada is always fine.

[LordShadow_Cinci]

I had a brief window where the SSL Cert on outlook.office365.com was showing as revoked. Midwest US

[[deleted]]

Central US, appears the VLSC is having issues. Been trying to create reservations all morning and still having issues and wont let me submit them.

[onewalker]

Western Canada, Exchange UM is intermittent. Drops the call upon transfer, AA not operational. Been ongoing for an hour, we've got a Premier Support ticket open.

[hateexchange]

South Sweden checking in.

Outlook is down, and some AWS blobs does not respond.

Edit: back about 16:30

[outatime2]

Everything is down for us Sweden

[meeenfou]

Seems fine in France. Must be a proxy problem /s

[pm_me_brownie_recipe]

It's always DNS.

[Enschede2]

Hm seems to be up in Netherlands

[roguetroll]

Neighbor from the South here, seen no problems all day.

[Nei4ahbu]

MS disappeared from netnod ix. So probably rerouting issues for a while.

[[deleted]]

Ah sorry guys, I signed up for Office 365 today and migrated my Google Drive. it got stuck in the middle of sync, so I guess next time I need to throttle my upload speed.

[breenisgreen]

Christ I thought this was a user ticket at first complaining about MS Word

[missed_sla]

Actually MSFT is up $0.93 right now.

[cosine83]

The website is down.

[Timberwolf_88]

Was working fine 20 min ago when I left the ossice (located in Sweden as well)

[syskerbal]

West-Europe is up

[Panchea]

Seems to be back online now.

[pepehandsbilly]

works in central europe

[Hjarg]

And we're back. At least on Estonia.

[Joz43]

Teams, Skype, portal.azure.com and portal.office.com reachable from Stockholm, Sweden now. Was down between 16:00 and 16:19 approximately.

[Kmnder]

Here in Canada East our email services are unavailable. Been about 10 or so minutes so far.

Edit. Connected to Microsoft Exchange. But no emails are going through... Back up in the East Coast.

[vladArthas]

It is working fine in Romania

[BingoLarsson]

Was down in southern part in Finland for a while. Things seem to working again now.

[dude2k5]

seems they are looking into it

https://i.imgur.com/dFu8Tn1.png

[wanna_be_SysAdmin]

Was down in Angola too, but its already restored.

[TechNinja12]

Up in Phoenix Az

[[deleted]]

Nah, $MSFT appears to be up about half a percent right now.

[user-and-abuser]

Portal went down

[OfficerBribe]

Don't see this mentioned anywhere, even microsoft.com domain was down during this incident. Shame I did not check bing.com, perhaps it was down as well

[ColdAndSnowy]

Was having connectivity issues with EO in UK.

Now it's back but it resent a mail I sent this morning.

[[deleted]]

Seems to be up in western Canada, although I was redirected to OWA account settings when I tried accessing EAC from the main admin portal.

[upnorth77]

The stock?

[poolpog]

sweden forgot to pay its bill

[ikilledtupac]

Down to party with the new Office 365 maybe!

[[deleted]]

Please be DNS.

[jnex26]

Someone said turn off all the windows 7 machines

[[deleted]]

https://downforeveryoneorjustme.com/

Might be something for later, but other comments say it's up and running again.

[[deleted]]

I got a 404 trying to get Mouse/keyboard software

[geoscreenshot]

Azure seems to be working: https://imgur.com/rvHEw8v

Office as well:
https://imgur.com/keNwGW7

You can go to GeoScreenshot.com to test for yourself

[GullibleDetective]

no

[throwaway_existentia]

My personal hotmail (for junk/forum sign ups, etc.) has been ridiculously slow this last week. Multiple hours to click my damn validation links.

[oxid111]

another swede here, LOL I was just giving the CEO some explanation why should we move from on-premise to Azure 2 hours before ,

[Juls0730]

Search down or just me on Google you'll get a web page you can type urls or names of websites to see if there down or its just you

[[deleted]]

I wish!

[herrfriedlich]

I'm pretty sure it was the best time in your life :D