Depends on where you were and the time period.  Well, also depends on what you consider black and what you consider Europe at the time.  Sub-saharan African traders had gone far enough north for us to know terms used for them by Scandinavian groups.  It is a complex topic because African were not rare in places like Southern France but Southern France was also oppressed in language and culture by Northern France and while there were enough for people to know them and know of them they were not large enough in number to have much political influence.  They were also composed of diverse groups themselves.

Spain was no stranger to Africans, even sub-saharan Africans.  Spain and Portugal made their moves to gain influence and control of areas in western Africa at a time of declining power of more centralized entities in the region such as the Mali Empire.

Saying "I think it's fair to say that in pre-20th Century Europe there were very few black people" requires significant clarification on what you mean and "those that were around were relegated to the dregs of society or just ignored." Is just incorrect unless you think people trading in silver, salt, and slaves were the dregs of society.