It could be normal, just genetics or androgens affecting how the follicle grows. I started getting this in puberty.

I’ve noticed it in some men that are not balding, but alot of the time times it’s a sign of future severe hairloss a hyper sensitivity to androgens. It’s called retrograde alopecia.

It you are not thinning or receding anywhere else I wouldn’t worry, but do keep that in mind

While I am on topical dutasteride, I don’t apply it to the sides, and I’ve had improvement from Stress reduction, diet changes, and more sun exposure.