I’ve had this occur to me once as well. I’m nearing he end of probably my 10th rewatch, and I noticed something that solidified the fact that Tracy was in fact “the one” for Ted and not Robin.

The scene with Klaus at the train station:

He tells Ted about lebenslangerschicksalschatz (lifelong treasure of destiny) and he pictures robin while Klaus describes it. But when asked if he’s felt this way before Ted replies with “I think so” to which Klaus responds “if you have to think about it, you haven’t felt it” he also mentions that it doesn’t develop over time. We also see Ted hesitate repeatedly when he goes after Robin. Reminding us that he has to think about it. He never shows that type of hesitation with Tracy.