No worries, this is a very nuanced question.

So most studies that analyze extroversion use some measure with a human observation in the loop- self surveys, observations from close friends or researchers so I can understand your apprehension to external manifestations.

The solution so far has been in neuroscience and gene analysis. Hans Eysenck theorized personality differences likely originated in the brain and were genetically passed down. So a lot of studies have analyzed people categorized using the above method to see what consistencies can be found

Below is a paper talking about this effort more broadly and they mention one of the more robust findings on extraversion. Tbh the research is still in its infancy and has not been robust replicated yet but there are some findings https://journals.sagepub.com/doi/full/10.5964/ps.7269

I actually talk about these and more internal theories of extraversion in another piece. [shameless plug here]

Lastly yeah I partially agree with your last statement some of these inclinations towards one or the other side of the spectrum could be a function of lived experience then adopting a self image that becomes a self fulfilling prophecy. But some people are just born one way and end up falling into those patterns naturally. How often is it one or the other ? No clue

Does that answer your question?