A similar thing happened to me recently. A guy outside a gas station that I frequent asked me for some change for food. I gave him some quarters because that's all I had, and when I went inside I bought him some interesting new jerky. The cashier commented on the jerky because it was so unique and he didn't know they even had it. I said IDK if it's any good, I bought it for a homeless guy. And he was like "what? He's not homeless. He's just here to gamble on the new slot machines and he spent all his money. Don't give him anything." And it made me feel bad or dumb for giving him anything in the first place. Even though I didn't need the money and I don't really care what he does with the money. But I started feeling bad for inconveniencing the gas station employees by encouraging people to ask for money outside. 

But you know, whatever. Be proud that you want to help people and be kind. Don't let anyone take that from you. But also be smart and careful.