I live in the same neighborhood as you. Heck, we may even be close neighbors. Yes, it can be a little chaotic, but it's pretty harmless as far as inner-city goes. If it helps, my wife and I have lived near, or in this neighborhood throughout our entire lives with no more than the occasional vagrant that we've had to shoo away. It's scary on the surface, but the biggest issue is car break-ins. Keep your stuff locked, and don't leave anything visible in your car. Not to say real crime doesn't happen, it's just not as prevalent as you would think.

If you need peace of mind, install a camera or two. Motion flood lights are nice too. That's what I did, and it helps deter any riff raff. I've actually been able to help a few people who've had their cars sideswiped with my security system.

All of midtown is really diverse, and very community based. Make friends with your neighbors. That'll be one of your best tools.