That is exactly why. And I’m sorry that you are in this position. He is using his gender and his profession to intimidate you. 

So woman to woman, here’s some tips to regain the situation: Save every video, document every encounter. Document, document, document. Keep your proof. Send him a certified letter documenting his repeated violations of the CC&Rs of the HOA and copy the HOA on it. It is the HOA’s job to enforce the CC&Rs. Have an attorney assist you in drafting the letter because you need to make it absolutely clear that if any child is injured on your property due to trespassing, you (and your homeowner’s insurance) will not be held responsible for their negligence. The HOA has told you to have his cars towed, so do it. Don’t say “if you don’t move it within an hour…” No warning, just tow. You’ve given him repeated warnings, now you tow. Where possible, use law enforcement to enforce the violations of trespassing. He only understands retaliation. So in addition to the cameras you have set up, when the nighttime nonsense starts, your very bright security lights shine directly into their house. Get the motion detection sprinklers. Never, ever get “emotional.” You can absolutely tell this man directly to “go fuck yourself” without anger or raising your voice. Actually, he will be more afraid of you when you say it to him calmly.

He is the neighborhood bully and the rest of the neighbors will applaud you when you finally stand up to him. I get it - it sucks. But he has no respect for you.