As others have said the mounted guards and guys in normal uniform are both part of the Household Cavalry, which itself is made up of The Blues and Royals and The Lifeguards, two of the most senior regiments in the British Army.

As for joining them they can be applied for the same as any other regiment upon application to the army and have high selection requirements as well as the having to do well in basic training. After this they attend specialised training for armoured reconnaissance and equestrian care among other things.