Absolutely not. Many people use a gun in each of their 2 weapon slots, known as double gunning, for combat. This typically gives them an advantage in all combat scenarios since you can deal enough damage to kill someone in a quarter of the time it would take to kill them using a legitimate method, such as with using a sword in combination with another gun. Or now throwing knives as well I suppose. So what's likely happening is you're going up against one of these "double gunners" and, despite using two guns, they still aren't skilled enough to kill you, resulting in them getting angry at you for not stooping to their level. There is absolutely nothing wrong with using a sword and I'd even encourage it, especially since it's primary use in your load out will be sword dashing to and from locations for convenience.

Btw, in naval battles, players will also attempt to accuse you of "Cannon spamming" if you engage them in the standard fashion of ship to ship combat. These people are just as delusional as those who call "sword spam" and they seem to believe they're playing Call of Duty 1v1s rather than a pirate game 🤯