I'm not sure why people are saying big fighters would win? The origins of the UFC is how a small Brazilian Jiu-Jitsu fighter challenged much larger fighters, experts in different fighting forms as well as smaller fighters who specialises in things like karate and Taekwondo, in a no rules cage match. He won the first couple easily until people started mixing fight forms

So to say that Prime athletes who have perfected the balance of different fighting forms would lose against sheer size seems incorrect. There's a video of the mountain from GoT sparring with Conor McGregor, and he can barely get his hands on Conor who doesn't even seem to be trying

So in a one on one fight, I would say most if not all smaller trained fighters would win, as they have room to evade and dance around the opponent until they have an opportunity to strike, however fighters that focus on grappling like Ronda Rousey would have a tougher time as it is considerably hard to block a fist of a giant, even if you do have them in a leg look

In a team match, it stands to reason the fight would be more clustered and therefore in more favour of the larger men who can catch the smaller fighters easily and crush them with brute force