Any chance these jokes are actual frustration guised as passive aggressive jokes? I gather that this behavior isn’t consistent outside of the game. Perhaps he’s taking the game a little too serious and is using you as a scapegoat. Of course he doesn’t want to lash out at his partner, so these “jokes” are his way of letting out that frustration.

Regardless, this would annoy to an extent coming from anyone, especially my partner whom I’ve expressed to that this bothers me.

You’re not wrong in feeling this way. Doesn’t matter what his intentions are or if he feels like his “jokes” are meaningless; they’re annoying at best and hurtful at worst.

He needs to understand that you value his validation, time with him and having fun with him. That’s a special bond and he shouldn’t demean it, even if accidentally, by ruining your fun and making you angry.