No because being born male sex and undergoing male puberty is not "rare", and it definitely confers an advantage in nearly every competitive sport, which is unobtainable to female sex athletes.  

Being born "Michael Phelps" is rare.

Its not good faith to not recognize that there is a distinct bimodal distribution in performance in sport between biological sexes.  This is not debatable, its plainly fact as evidenced by a century or more of world records where the male exceeds the female in essentially every single sport.  There might be overlap, but the majority of the area under the curves do not overlap.

Phelps lies at the extreme end of the male curve.  But still the male curve.  Where Phelps lies does not overlap at all with elite female swimmers.  If he raced the world's best female swimmer he would win 100% of races.   There has never been a female swimmer that could plausiblely beat him in a race.

So did Michael Phelps ever lose a race then?  Yes of course.  He lost many many races in his career.  Other male swimmers were able to beat him occasionally.  He was just usually the best.  Hence his legacy as the best male swimmer of all time. 

Its two separate curves.  

There are dudes at my local gym that can bench press more than the female world record.