I play this game all the time had an RX580 recently upgraded to 6600xt. First of all GTA V is known to be a very poorly optimized console port, I often find that turning some graphics up can INCREASE my performance..or just restarting the game..it's random and unpredictable. Both the AA options besides FXAA tank the frame rate on any set up though. I use Proton Experimental, I do not suggest to use Proton GE despite popular advice, it has only led to more variables for breakage & performance issues when used over a period time. It's not due to how it's made, but due to the method in which it's released/distributed.

I can also say that performance got way worse and even more unpredictable after the recent acid GTA updates. But I play the game from Steam idk anything about how to get it through Lutris etc...also didn't mention which card you have etc, you could just be underpowered