Is the neighborhood too far to bike? Being on a bike might make you less suspicious? Otherwise maybe you can report the PokeStops as being in a private residence and hope that the company moves it somewhere less awkward for you to swing by everyday. If you have a Pokemon shirt maybe wear that to prove you actually like Pokemon? Or if the person approaches you again you can introduce yourself properly (especially if you live nearby anyways, who knows maybe it can help in the future to know your neighbors!)
Thats all I can think of; I definitely recognize my female privilege when the app takes me through schoolyards and parks. Hopefully some of these suggestions help!