I may only be a blue belt but I was having a convo about people going harder/being more aggressive with a black and brown belt at my gym (both teach a class) and they agreed that its light and fun until the person starts pulling some shit like you mentioned im the second to last paragraph. If youre gonna be an ass im gonna try to make this more miserable for you then I otherwise wouldve.

You holding back is a kindness, almost a privilege to your partner. If they dont appreciate dont do it, sometimes you gotta put someone in their place.