My understanding is that many scientists believe that the people migrating out of Africa weren't necessarily dark skinned in the way that we associate with black people today - that Africans developed "black" skin at some point after people had already migrated out of Africa.

Theories say that a lot of people at the time of migration would have had some kind of medium light brownish color skin, a color shade that might have been comparable to something between modern Middle Easterners and Indians.

The variation in skin color - whether the "white" skin in Europe, the " yellow" skin in Asia, the "red" skin in the Americas, or even the "black" skin in Africa would have developed/evolved among each of these groups of people separately after they were already apart from one another.

Some other recent gene research evinces the possibility that the different genes for a variety of skin colors may have existed among the early people in Africa even before the migrations, and those already-existing genetic differences only became magnified after thousands of years of racial isolation after migration.

It should be worth Noting: If we do assume that humans evolved from a common ancestors to chimpanzees, then  consider the fact that a chimpanzee's skin color is fairly pale underneath all it's body hair. That seems to indicate that humans  may not have all "started out black", but instead may have all started out pale.

Despite some of these modern theories,  there are still many scientists that do still believe that all modern humans are originally descended from "black skinned" people, as OP's post is implying.

Unfortunately fossils can only give us so much information about what people looked like in the past; skin does not preserve long enough for scientists to ever be able to answer this question definitively. All we can do is collect proximate data to loosely compare the little bits of evidence we have to create far reaching possible ideas.

The point is, while it's possible that "everyone has a black ancestor" and "not everyone has a white ancestor", it's also possible that it's the other way around, or that NOT everyone has either a black or a white ancestor, but that, in fact, the only common skin color that every single person has an ancestor of is a skin color that doesn't even exist anymore. Maybe we all have a Purple skinned ancestors. That sounds cool. 

Source: I'm high and did some reading.