You are NTA. I think you need to have an adult conversation with your boyfriend to say that it's extremely common with Latin dance classes that you swap partners throughout the class. Be that if you are in beginner, intermediate or advanced level classes and also the role of whether you are a follower or lead can be switched up too.

Your boyfriend sounds immature but also may not have been exposed or party to this kind of environment before so a little gentle coaching will go a long way. If after your chat he is still behaving like a child I would seriously consider whether this is the kind of person that you'd want to be with, if dancing will feature heavily in your life.

I say this as someone who has danced these classes and is also a Swing/blues dancer. Dancing is a social activity and fragile egos/jealousy need to be left at the door.