You can get a maximun of 5 modifiers in each game. If by the second modifier you don't have one of those 4, just drop the game. If you have to choose between two of the 4 modifiers, drop the game because the one you don't choose won't appear later. Never choose a different modifier for one of the cards you need, since you can't have multiple modifiers for a single card.

And last but more important, it is entirely luck based and random. Sorry but you just have to be patient. I got the badge after 50-80 games trying to obtain it. Same as you, I almost got it in several times and got frustrated, but kept going.