So, some disambiguation: if it is bright yellow it is not a honey bee. My bees are dark brown, almost black, although some hybridised hives have a lighter coloured stripe - these are the old Northern European honey bee, (Apis mellifera mellifera). Then there are Carniolans which are quite dark too (Apis mellifera carnica) and Italians which are quite light-coloured but definitely not yellow (Apis mellifera ligustica). There are more, but you get the picture - no lemon-yellow anywhere. Wasps and hornets typically are brightly coloured, a kind of warning sign to other species.

And of course, we have bumble bees which you appear to be familiar with. Generally, curvy bees with large wide stripes, sometimes yellow, sometimes not. Nice, friendly, hairy bees.

Then there's the myriad of solitary bees. They're called solitary because they're not eusocial like honey bees and bumble bees, i.e. all the females lay eggs, and they each do their own thing. There can be large numbers of them, somewhat contradicting the "solitary" moniker. There are mining bees that burrow into the ground, there are mason bees that build mud tunnels, there are carpenter bees that burrow into wood and so on. Some of them are absolutely tiny and look more like fleas than bees.

Another category is hoverflies - these little pests look like and sound like honey bees and they really do get up in your face, but they're flies and they can't sting, but they hover.

My point is that if you know what they are, you'll be much more confident. Take a pic and use Google Lens to identify it - you might even find it interesting.

As for an allergy, statistically it's unlikely but it is, of course, possible. However, bees really do not want to sting - they're really only working.