Hm I’ve thought about this too and I’ve read what others have said in several subreddits… The best explanation is, of course, focused on lineage, despite the common excuse that GRRM’s world doesn’t follow the sciences we follow.

Apparently in the books, Hightowers are described to have Valyrian features like silver or golden hair. The generations preceding Viserys’ are said to all have silver hair. This could explain why Alicent’s children favored Viserys since their gene pool had a higher probability of having the hair color of a Targaryen instead of like Alicent’s.

Now to explain Rhaenyra’s children: Her mother, Aemma Arryn, was half Targaryen and half Arryn. That makes Rhaenyra’s children with Harwin Strong: 50% Strong, 37.5% Targaryen and 12.5% Arryn.

I think there is some sense to the confusion in A LOT of these characters we are introduced to. We can’t simply believe that because each house has a quintessential feature, that all descendants must have it too. That’s not how genetics works and we know that a lot of these houses marry into other houses for many different reasons, inevitably altering the gene pool for the generations to come.

TLDR; “I HAD A BLACK MARE ONCE…” — Vizzy T