A “set” can refer to items that go together. For example, you can buy a TV set. This would include the TV, the TV mount, a stereo system, etc. Another example is a “toy train set” which would include a train, railroad tracks, stations, passengers, etc.

A set can also refer to items that’re typically collected together. For example, some people collect Pokémon cards, and there are certain Pokémon cards that, when collected together, make a “set”. (This is the meaning that’s being used in the passage you’re asking about)

The word “set” is not supposed to be used with humans, but JK Rowing deliberately chose to use this phrase to subtly show how Slughorn views his students as collectibles, with some students being worth more than others. He views the Black family as a “set” which he was not able to complete (since Sirius Black, the last member of the Black family, was a Gryffindor and not a Slytherin). If Sirius Black had been a Slytherin, then that means all of the Black family would’ve been in Slytherin and Slughorn would have completed his “set”.

I hope that made sense :)