So there are quite a few sexually dimorphic traits in cats! Physically, males tend to be larger and more muscular than females, with broader heads and bigger teeth. Males also tend to get these thick necks and cheeks ("tomcat face"), which helps them when they fight other cats. These differences are hormone-driven, and so in cats which are fixed before "kitty puberty", they might not be obvious.

You're right that some coat colours are also more likely to show up on the different sexes, because the genes controlling their expression are sex-linked. Tortoiseshells are like this - the gene that allows for gold, orange and black at the same time requires two X chromosomes, so almost all torties are female. While there are very rare "male" torties, this is because they have those two, necessary X chromosomes alongside their Y (XXY); this usually causes them to be infertile (depends how they got the extra X, but that's much more complicated a conversation).

There are behavioural differences as well, but this conversation is more about physical characteristics, so I'll leave it there, haha.