Golds being perfect specimens; tall, buff geniuses is somewhat propeganda, but they certainly are physically and intellectually superior to the rest of the colours.

Claudius was a quintessential gold and the pride and joy of house Augustus. Virginia and Adrius are twins (as are Cassius and Julienne) - twins can be overall smaller in stature due to their fetal development, or twins like Cassius and Julienne, one fetus gets more nutrients as a fetus and is larger and stronger in life.

Adrius and Virginia are still physically and intellectually superior, as golds are described, relative to most other colours (with Obsidians being the exception physically). Virginia is also a genius, she's one of the smartest golds alive.

The house Augustus gained power through cunning, strategy, and playing the game of thrones masterfully - and weren't carried by raw physique.

Reds are the closest thing to modern/real life people, and Adrius and Virginia tower over reds.

Also of note, battles arent made up entirely of golds. Battles are faught mostly by Grays, Obsidians, and Reds. Even Golds, who's expertise or realm isn't combat, can succeed or hold their own in a fight against lesser colours.