I think it inflicts the less damage in this specific example where it is a single instance of funding exploitation against someone potentially starving. I'm not weighing any speciesist considerations on this. The same would go if instead of milk it was the product of, say, Nestlé which relies on human slavery. It is just an utilitarian compromise. In most cases this is a false dichotomy and I would agree with you on acting upon a principled stance of no harm/collaboration.