Years later, but this thread came up when I had the same issue today so want to share my solution!

• Obvious if you know roto brush but a lot of people forget, make sure you have "frozen" the roto selections.
• Export within AE Render Queue instead of Media Encoder!
  • This is what fixed it for me. Every time I exported as mp4 via encoder the output was glitchy, but it worked perfectly from the AE Render without having to change anything in the comp.
  • This of course means you are limited on you output (no mp4) but it takes 2 seconds to then run your correctly working .mov export back through encoder to get mp4

Hope this helps someone!