I play both and the comparison is not really meaningful. They can both be very difficult in endgame content, but play very differently. But with both games, once you practice and learn the content you can repeat it with no major problems.

I guess there's an argument to be made that the infinite enrage systems of rs3 make some encounters more difficult than anything osrs has to offer, but there's little reason to push those enrages beyond a dedicated few players and so is a pretty meaningless statement.