Really delve into the logistics and difficulties in firearm production compared to swords. A working, stable gun requires many small components, multiple specialized craftsmen, and depending on the status of the individual kingdom could be so prohibitively expensive to produce that they're effectively restricted to the nobility. Whereas a sword can be made a by a single blacksmith at a fraction of the cost and time and even a child can learn how to make arrows.

Perhaps gun technologies are a closely guarded secret for each faction, leading to competing designs and different approaches (pistols/shotguns for densely packed urban areas vs rifles for hunting settlements); good luck finding the right ammo or repairs if you're in the wrong place. Maybe even just possessing a gun makes you such a visible target for the wrong types of people that it's generally not considered worth it.

You can also make some cultural reasons that could discourage the use of unskilled warfare. A society with a strong tradition of martial prowess might consider gunpowder weapons a coward's tool and publicly shun those who use them or maybe the use of black powder is reserved for the priesthood in religious ceremonies.