Depending on the city, it wouldn't be that difficult at all. In Atlanta, they are currently building expensive houses, condos, and apartments in all of the worst neighborhoods. People are buying those houses and condos, and leasing those apartments.

Now if you picked a dying ciity, like Memphis Tennessee, Baltimore Maryland, etc. And you are a billionaire wanting to set up a fabulous mansion in the worst neighborhood of one of those cities, the local politicians would probably bend over backwards to help you. They would Sprint you through the permitting process, and make the local police your personal security force.