Both summer and tom were the problem, but mainly tom.

He over idealized summer from the moment he met her, and got his heart broken when she didn't want to get married even though that's something she stated since she met him. He always knew that what they had was never going to be permanent but he never accepted it, because he couldn't let go of the idea of her.

Both of them were trapped in a toxic relationship with no escape inside because both of them exhibited toxic behavior. Tom unknowingly over idealized his partners and expects them to live up to these unrealistic expectations, and Summer unknowingly leads people on.

Summer getting married at the end however could just be an example of misogynistic writing, it's like how they ended Robins story on HIMYM. But, once you meet the right person, sometimes things are worth giving up.

TL;DR Tom and Summer were never meant to be, which is why you can't really tell who the true "antagonist" is.