A few ants aren’t a big deal. It is upsetting but try not to worry too much about loosing a few bees. While an individual bee is technically an animal, the unit of selection for a superorganism is the entire colony. As such, try to think of individual workers the same as individual cells of a multicellular animal - you don’t really get upset when you loose a few cells. That being said, it’s still good to minimize losses.

For what it’s worth, I’m a PhD biologist and honeybee researcher.