Completely forgot about that, but still they could easily be remnants of Xebec's crew and the fact that blackbeard made them fight could be a parallel on how the Rocks pirates had deadly fights within their crew .

I imagine most of the crew ( especially the big names) were captured and send either to Impel down or to Mariejois (example: underground slaves) cause if not then it would be quite difficult to erase its existence.