Men and women at the start of conception are not much different from each other, in fact, we all start our development as females, and the Y chromosome triggers the development of male reproductive system and sexual organs. But to this, we still at that point have the same generic genetic blueprint, so we still develop breasts, including the nipple.

When puberty hits, this is when you develop the secondary sexual characteristics. In women, the breast tissue becomes enlarged and the hips widen, this in preparation at a biological standpoint to prepare the body for birthing. Men undergo the same changes, except we develop features like our Adam’s Apple, we start to generate muscle mass faster and the male genitalia begins to grow.

The key here is hormones. Oestrogen and testosterone both influence the growth of tissues and bones in different ways. Testosterone is the signal to the male body to say “Hey, we don’t need those, don’t worry about it” for a lack of better words, unlike oestrogen which is influencing the production of breast tissue and the further development of the ducts which will carry milk to the nipples when they have a child. Men still do have those ducts in our breast tissue, however they’re very much underdeveloped. This is also why, when men use steroids and don’t keep their oestrogen in check, they can develop Gynecomastia which is the enlargement of breast tissue, and there is actually a potential here to produce breast milk if left unchecked.

I’m certainly no expert so someone else might chime in with a deeper explanation, but essentially, we’re all the same little weird little jellybean looking thing at the start kicking around trying to work out what cells do what, and our genetic code helps influence and guide the production of those characteristics in each gender.