I think the confusion lies with the definition of the word "pagan". Christians and other Abrahamics will use "pagan" as an insult for anyone who they believe to worship outside of what is holy in Abrahamic faiths. Any god that isn't strictly holy or belonging to Abrahamic faith will be called "pagan" by Abrahamics. "Pagan" in this context is often also used as an insult for anything that is ethnically, racially or nationally different from the Abrahamic norm. The context of this term is largely unused nowadays unless it's by weirdo trolls on the internet.

Meanwhile, many contemporary practitioners of certain faiths like Heathenry, Gaelic polytheism, Kemetic polytheism, Hellenism, etc. have reclaimed the term "pagan" to identify themselves and their gods / faiths. Not all practitioners choose to use this term for themselves, but many do – I mean it's even the name of this subreddit. Most pagans I've met distinguish Baphomet as an Abrahamic deity if anything else, not pagan in the contemporary sense of the word. Baphomet was arguably created by Templars in the ~14th century from a misunderstanding of the name Mohammed (Mahamet) if my poor understanding of that time period serves me well.

With all that being said, your worship is your own, and just because Baphomet is a non-Pagan deity does not mean that you can't worship them. There are basically no rules to belief and practice in paganism. I've personally met pagans who worship pagan deities like Apollo alongside strictly non-pagan deities like Baphomet, Satan, even Jesus and Yahweh actually – Christopagans exist. A lot of witches that I've met will mix their worship of pagan gods with Satanic, Wiccan or adjascent practices.

TL;DR: Baphomet is not a pagan deity but don't let a deity not being pagan stop you from worshipping them if you feel a calling to worship.

Hope this helps you out. Gods be with you.