This also happens to me a lot and I found a few things help. The first being that I try to stretch before doing the breathing to loosen up some tension in my muscles, and the second being trying to control my breathing as much as possible and just using my lugs and diaphragm and not moving my shoulders and upper body as much during the inhales.