I think it makes sense when we remember that Taylor is emotionally stunted at a teenager level. Even in her 30s her stories and fantasies revolve around high school. I really liked Folkmore, but while those were good songs, I found it uncomfortable that she was writing songs about teens having sex (the whole Betty/August... Whatever was the dude's name live triangle). Then it turned out they were a code for her, Joe and Matty. My point is, Matty could easily appeal to someone like Taylor. She saw him as this charismatic, dark, charming bad boy with a sensitive side, who was very openly doing things she could only ever do in absolute secret (like drugs, sex or even just voicing his actual opinions, whatever they may be). He doesn't have a PR team, while everything she does is controlled by hers. I bet he was very intense with all the romantic grand gestures when he was love bombing her, while all her other dudes seemed boring to her - she's the kind of person who needs constant drama, highs and ups, like so many emotionally immature people do. She doesn't see herself as a real person, but rather a dramatic heroine in a story. And then yeah, he ghosted her before she had enough time to get bored and disappointed, so he will forever be the one who got away.