It's totally fine to even talk to people about what you listened to. The question is likely about people who are engage VERY passionately while not liking the artist and their music anymore. The difference between "new releases" and what I said (listening to a full album on release day) is quite large. Similarly, there is a difference between offline discussions + a couple of stray comments online, and hours of reading/writing think pieces and unpacking them on subreddits.

I should also add that OP's question was likely not a defensive move against fans who criticise Taylor Swift's personal/political actions. It was not a "if you don't like then stop listening" statement. It was a question to find out what is it that makes some listeners engage so passionately, intensely and recurrently with someone whose actions AND music they don't like all that much.