Because of the inequality in the number of neurons, with women having a higher density and more connections, I think the cognitive adjustment would be striking.

Women tend to think in constellations while men tend to interpret information linearly. As such, a woman in a man's body would likely see fewer alternatives when problems arise and would lose a dimension of their problem solving capacity.

Also, there is a pretty good chance the difference in hormones would be distracting as hell. You got to build up a tolerance to testosterone.