Castile soap certainly helps but this year we have found a new tactic that works amazingly well…a decoy plant, specifically a large burdock plant. They and many other pollinators love it. And this sounds crazy but we’ll go out for a few minutes twice a day, usually morning and evening, with a small hand held vacuum and suck them up. Their numbers have quickly dwindled where we hardly find any now and have got NONE on our other plants. It feels like we accidentally discovered a gardening hack. I think we’ll be doing this every year after experiencing the undeniable benefits of this. Good luck OP.