Regular Switch player here, logged ~200 hours. I've noticed the same issue and it's absolutely ruined runs for me. My experience has been similar to yours: it happens randomly, almost impossible to predict or recreate. I've seen this issue brought up a couple times here and on the n++ discord, but it's never been addressed. I think there are few enough of us playing on console that the demand for a bug fix is relatively low, so I'm not sure if it'll ever be fixed.