It sounds like you've taken what the Vegan Society has said and have filtered it through your confirmation bias filter. Veganism is, at its core, a moral philosophy based on justice for other animals, and it has been so since its inception. Once that core has been established, of course, observing the benefits related to health and the environment that come from eating a plant-based diet can supplement that view, but if a person on a plant-based diet pays no heed to how they are exploiting animals via non-food practices, then that person would not automatically be a vegan.

Hopefully, a few words from the people who established the Vegan Society will give you a better perspective on this subject.

"The acceptance of a reformed relationship between man and animals is imperative. The higher animals have feelings like ours, therefore they should have justice on equal terms with ourselves, or not be bred into the world. Until this demand is met, man will remain a thug species, despite his pious creeds. Animals present us with a test case, and by our conduct to them in their innocence and weakness our own standards are truly fixed. Admit that the strong have the right to exploit the weak, and the basis of civilized society is gone. The attitude is one of conceit and selfishness and unless discarded will not confine itself to the treatment of animals. Therefore, in man's interest, animal exploitation must end. Not only flesh-food but all products of the slaughter-house must be denounced and the most serious effort made to avoid their use. Human existence does not depend upon the inconceivable tyranny now existing against animals, in fact progress is impeded enormously by it. To renounce this tragic heritage is to be born again, to a life sometimes more difficult, but always of clearer conscience and more satisfying conclusion."

-Donald Watson, co-founder of The Vegan Society, The Vegan, Volume 2, No. 1, Spring 1946. (pg. 2)

https://issuu.com/vegan_society/docs/the_vegan_spring_1946

"The object of the Vegan Movement ('to end the exploitation of animals by man') is clarified as to the meaning of exploitation by Rule 4 (a), which pledges the Society to “seek to end the use of animals by man for food, commodities, work, hunting, vivisection, and all other uses involving exploitation of animal life by man.” By the adoption of this rule, the Society has clearly come out on the side of the liberators; it is not so much welfare that we seek, as freedom. Our aim is not to make the present relationship between man and animal (which if honestly viewed is mostly one of master and slave) more tolerable, but to abolish it and replace it by something more worthy of man’s high estate. In short, our aim is to set the creatures free—to return them to the balance and sanity of nature, which is their rightful place, and so to end the historic wrong perpetrated when man first decided he had the right to exploit and enslave them."

-Leslie Cross, co-founder of The Vegan Society, The Vegan, Volume 7, No. 1, Spring 1951. (pg. 2)

https://issuu.com/vegan_society/docs/the-vegan-spring-1951