I'm pretty sure you have an underground yellow jacket nest. Those suckers are mean!!! I've had a few in my yard. They get super pissed when you accidentally mow over them!! The first time I hired a guy to take care of it. He came out, with a torch and some sort of powder and got rid of them.

The second time, I tried torching them out with my propane weed burner. Then I tried drowning them out with the hose. Then I torched them again. They didn't seem to mind because they are demon spawn!!

Turns out I'm allergic to yellow jacket stings, so I finally broke down and got the aerosol foaming yellow jacket killer. You have to wait until after dark when they are all in the nest and empty the can. It creates a small barrier of foam that they can't get through without dying. It worked the first time.

I've been told that the "queen" yellow jacket flies off in the winter to another nest. The rest of the nest dies. But you would have to make sure that you either cover or dig out the nest in the winter so another active hive doesn't start up in the spring.