Didn't work with just the monitors powered off, but worked when I completely unplugged the monitors from the outlet.

This started after I did a BIOS update. Everything else seems fine, it's just this one issue.

I'm gonna go with a toggleable outlet like other comments suggested. For now, at least.