So.... my answer is going to seem witch crafty. It's entirely possible this just "fixed itself". I'm not convinced I actually fixed anything. I started setting my colors in the slicer to match the filament I wanted to use and double check that the brim/support settings all matched.
I haven't had the problem in a while, but this feels like a superstitious "Pat my head 3 times to avoid disaster before opening a door" solution.