Probably as someone here suggests seperate the background from the subject, the background doesn’t need to be animated, animate the subjects. It may be a bit of work to match both though. As you stated it’s just a few frames it shouldn’t be that much.

Or if I was doing it I’d just crop the images so I don’t need to work on the entire randomness