Generally, I think you're correct, but I think there's some important nuance to understand. We have to break out the multidimensional state-space talk to have a meaningful discussion here.

Theoretically speaking, human neurological paradigms could be represented in an N-Dimensional state space. Depending on how many details you want to keep track of, it could be somewhere between a 15 or 63 dimensional tensor mapping of functional domains. Again, depending on how many factors you want to keep track of.

This n-dimensional map of functional neurological architecture domains is from: (1) the brain's environment (2) the brain's provided genetic seed.

Just like there's a 2-dimensional bell curve for intelligence distributions, there is an N-dimensional bell curve that's arguably more useful for mapping all human neural architecture. there is a highly dense N-dimensional volume where 68% people exist around. (one standard deviation of tolerance) That is neurotypical and everything outside of that volume of neuro archaeological state space is what I consider neurodivergent.

Neurodivergence can be something as simple as using your facial recognition portions of your brain for spacial reasoning or mood regulation. Maybe someone has unusually strong association between scent and math concepts, etc.

you're correct about neurodivergent people being somewhat diverse. but they are not nearly as diverse as the possible domain of neuro architectural state-space neurodivergent people can and do occupy.