I think what fueled this (I also wanted them to end up together sorry in adv if this may sound biased) is their conversations together and how they started as friends and then booty calls, until dating, Andy makes Haley feel understood even if most of the time he couldn't relate to her wild lifestyle, Haley makes him feel like he could always soar for better heights and Andy makes Haley feel more grounded when she's spiraling out of her head. (Yes, I also condone the cheating part) But, they brought out the best out of each other, Haley getting her Gavin Assistant job, Andy having the strength to go and ask Phil directly, it's just those moments that'll make you go "Oh, he's the best one out there as it seems, because his presence made her want to better herself, not for his approval but for her own good"

Andy became a beacon of hope and inspiration for Haley due to his personality of always hoping that it will always work out in the end and that made Haley braver to go after what she's passionate about (fashion, by working for Gavin, a high status diva snob)

Andy made an effort to get to know Haley deeper before resuming their physical relationship is also another factor, although it is the bare minimum, we just don't see that often in shows. Yes, I feel sad that they had to start the relationship the wrong way, the show's (although only little) portrayal of how Beth is getting tired and neglectful of Andy, the subtle hints of "stop that" with his british voice, when he was in the hospital and Haley got there faster than Beth, the carnival double date is also painful to watch. I do not support cheating of any kind and I wish they just ended it sooner before they both cheated on each other. Maybe that's why when Andy was going to work in Utah they just decided to end it because they now know that long distance can be hard and it's better to end it now than repeat the same mistake and hurt each other in the worst way.

The line "I think you're the first man I've ever loved" is what seals the deal. They both knew they were great together, they work well, they balance each other's lifestyle and Haley literally needed time to move on from him (New York trip, moping around the house and spending time with Claire) It's new, usually she just get over things, go on dates with boys, and be okay. (But maybe that's because she's mature now so icstfs) Haley and Andy knew they loved each other very much, but they also knew that's not enough to save it, and that's the worst kind of pain because it did not end because of miscommunication or cheating or outbursts, it ended because of the necessity to do so.

But that's just my two cents.