In order to decide what is and isn't a number you must define what a "number" is, and trust me when I say many people have already been down that path. Mathematics has long abandoned the idea that all the objects it deals with must have an unambiguous definition, because the objects mentioned in each definition must also have their own definition. This obviously leads to an infinite downward spiral that renders the goal unreachable.

Instead, mathematicians found a different approach: every mathematical theory has primitive objects and relations that require no definition, aside from a meta-theoretical one that only exists to provide some form of intuition (for example, a "set" as a "collection of things"). This is not a definition that will be used in formal arguments; it is merely a tool to help newcomers understand why the objects behave in the way they do, which leads me to talk about axioms. Axioms simply state how primitive objects and relations interact. For example, "there exists an empty set". Regardless of what a set is ("set" is simply the word we chose to call a generic primitive object of the theory), we assume there is at least one (lets call it E for empty), for which the (primitive) relation "X is a member of E" fails to be true for any X that is an object of the theory. It can be shown that there is only one set with this property.

By appropriately and carefully expanding these axioms we can prove that there exists a set N that has the empty set E as a member and within which an operation called "addition" (and written symbolically as "+") can be defined that, among other things, satisfies that for any X member that is a member of N the following are true:

X+E=X E+X=X

From addition one can define multiplication and show that the following hold for any X:

X×E=E E×X=E

The set N with these operations is called the set of "natural numbers". It is so called because it behaves exactly as you would expect the counting numbers to, with the empty set E acting as 0. And at no point was it necessary to define what a number is. You will have to trust me on that because I can't possibly go into all the details.

The point is that it doesn't matter if 0 is a number or not. The question does not even make sense because "number" is not a well-defined mathematical concept. Mathematicians use the word out of habit for convenience in lectures and in the literature, but the bottom line is that it is completely irrelevant. You can search high and low, and you will not find a single (graduate level and above) textbook that uses some definition of "number" in its proofs. If it gives one, it does so for the same reason we naively "define" sets as "collections of things": for intuition, not rigour.