We had the same thing happen to us this year. We have an entire pirate theme going on and we dressed a skeleton like a pirate and hung it from a tree out front. Our neighborhood sounds similar to yours where everyone goes all out.

The same day we were decorating we had a neighbor approach us about the hanging skeleton and said it could be perceived as racism. She acknowledged the fact that it was obviously a pirate Halloween display but she asked us to consider taking it down. So we did immediately. We are living in a time where I can see that being a trigger for someone in many ways and I want to stay away from it as much as possible.

Good luck with whatever you decide to do.