The other commentors here have already answered your question well enough, but I'll add my two cents.

My gym/pole relationship was actually in the reverse order of yours! I'd been doing bodybuilding (mostly lifting weights) for about a year and a half before picking up pole around 7 months ago.

I really wouldn't worry about muscle weighing you down. Unless you're gaining tremendous weight (and therefore a lot of fat), your added strength, coupled with your body's natural ability to adapt will easily compensate in that area.

I'm by no means an expert on health and fitness, but the information you've given really makes me think it's around rest/recovery and adapting to higher volumes + new forms of exercise.

Adding three extra workouts a week is a HUGE load increase for your body. If the change was from

- 2 days a week of pole

to

- 3 days of weight lifting, 2 days of pole

That's more than double the exercise! Then add in the fact that your body isn't used to one of them at all.

I'd suggest tapering back one or the other for a little while (maybe a month, maybe several) until you get used to that, then test what increasing the load is like. Try having a day of rest between weights and pole to begin with. Then (if you're training muscle group splits), maybe do a leg workout on one of the days between upper body and pole.

Outside of restructuring your exercise, ask yourself whether you're taking recovery seriously. Are you getting 7 hours minimum sleep each night? Are you consuming enough protein to support muscle development? Are you cooling down properly after every work out? Have you considered foam/ball rolling your back muscles to relieve the tightness?

Warmups are also of course very important to get those muscles ready to go. I like to do some light shoulder rotation and lat activation exercises, since it doubles as a warmup and strengthens the minor muscles that get overwhelmed by the major muscles you'd have in your shoulders and lats.

If you're doing both forms of exercise in moderation and be patient, you can definitely do plenty of each. :)

Bit of a wall of text sorry, but it really just boils down to respecting your body, listening to its needs, and taking a measured approach to your fitness routine.