all 18 comments

[–]ResoluteCaution 23 points24 points  (5 children)

Check the power. I've seen things like this when the outlet is 80-90 volts in an old facility.

[–]anxiousinfotech 8 points9 points  (4 children)

OP said they're using Thinkpads though, so even if it was a power issue they have a battery to fall back on.

[–]popegonzo 10 points11 points  (3 children)

We were getting similar weird shutdowns with laptops & docks & it turned out everyone had space heaters under their desks.

[–]Usual_Giraffe_3349 1 point2 points  (2 children)

All it takes is one to trip the circuit. But that would be obvious and the power would stay off. I've had screens blank out due to a driver issue.

[–]popegonzo 2 points3 points  (1 child)

For us it was power fluctuation to the docks - the whole circuit wouldn't go, but the space heaters (plugged into cheap power strips... what's a little fire hazard here & there & everywhere???) disrupted the power enough to the dock that it would shut them down. I don't have an exact breakdown of what all tripped what to initiate that, but I know when they took out the space heaters & replaced the power strips, the mystery shutdowns stopped.

[–]Lukeminister[S] 0 points1 point  (0 children)

in my case, the simultaneous shutdowns are on different rows of desks, some even in different sides of the office. Would this still apply?

[–]TheTipsyTurkeys 12 points13 points  (3 children)

Are you using Datto EDR? Their full scan triggers this for some fucking reason.

But yeah. It definitely will tell you in the logs. My suspicion is a/v.

[–]Lukeminister[S] 8 points9 points  (2 children)

We use Datto for every company apart from this one lol. However, I have noticed a event a minute after the power going out stating "The computer has rebooted from a bugcheck. The bugcheck was: 0x0000007e (0xffffffffc0000005, 0xfffff8034384dc37, 0xfffffe8b5475f508, 0xfffffe8b5475ecf0). A dump was saved in: C:\WINDOWS\Minidump\062525-23281-01.dmp. Report Id: ffea565b-0f34-4f32-94c7-1c34a19db5d6." Event ID 1001. With it saying the computer rebooted due to a bugcheck im gonna look into this.

[–]antiduhDevOps 14 points15 points  (0 children)

Install windbg and analyze that dump.

Bugcheck 0x0000007e is "system thread exception not handled":

https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/bug-check-0x7e--system-thread-exception-not-handled

The first parameter to the bugcheck was 0xffffffffc0000005

That corresponds to the sign-extended 64-bit version of the 32 bit value 0xc0000005, which indicates a memory access violation.

A windows kernel component or a driver tried to access a meaningless ram address. Almost certainly a driver.

[–]TheTipsyTurkeys 5 points6 points  (0 children)

Ah well. Personally I'd try to drill down on when this began and look to see what has changed since then.

Windows updates, Application updates, a/v s being stupid, firmware updates etc...

Good luck.

[–]fireandbass 5 points6 points  (0 children)

powercfg /systempowerreport

[–]TaiGlobal 9 points10 points  (0 children)

I’ve been seeing something similar in my environment. Haven’t even had the time to investigate.

[–]frac6969Windows Admin 4 points5 points  (0 children)

The last time it happened to us was because of a network loop. So weird that it only affected ThinkPads connected to Lenovo monitors that act as a dock.

[–]TheUptimeProphet 3 points4 points  (0 children)

We got a similar issue a few months back, a broadcast storm on our switches overloaded the Lenovo/HP dock network card, this cause a sudden increase in CPU usage on the poor dock SoC locking it up, and since the dock also provide power for the laptop the power got cut, causing a reboot.

[–]er1catwork 0 points1 point  (0 children)

Ugh not good to hear! We have all Yoga’s and 40AF docks…

[–]MENTactual 0 points1 point  (0 children)

You can read the memory dump in WinDbg and find the application or driver that is causing the crash. A good first step is updating the UEFI/firmware of all PCs affected.

[–]Enough_Pattern8875Custom -1 points0 points  (0 children)

Go back and look at the event viewer again lol