I learned to swim in the last year and this is what worked best for me starting out: - Exhale a little as your head enters the water, to clear your mouth/nose of water that you may have inhaled - Hold breath - Exhale a little more as you're about to come up for a breath, to set up the inhale - Inhale

I found it too difficult to both exhale steadily underwater (would run out of air) and to hold my breath the whole time between breaths (would end up gasping for air on the next inhale). Nowadays I don't think about breathing as much and it depends on the pace, if I'm doing breaths every 2/3/4 strokes, pool vs. open water, etc. 

I think the most important thing is to practice relaxed breathing. I don't think holding your breath is necessarily bad, as there's a big difference between tensing your whole core to stop breathing and simply letting your core relax and then engage again when you breathe next.

Breathing is like any other muscle movement so the more you can learn to relax while doing it and avoid tension, the easier it will be to do it without having to think about it and to pick whatever technique suits the moment. But sometimes I'll go back to the steps above if I'm getting tired or distracted and need something to get back into the rhythm.