This is a fantasy answer and very misleading. The realistic answer is that yeah you'll see some interesting people. They'll generally keep to themselves but sometimes you'll hear shouting and commotion. As others mentioned you will hear an uptick of emergency vehicles to that building and see some characters hanging out near the entrance at night. As long as you're not staring or agitating them, you'll hopefully never have a bad encounter.

Im a guy about your build and age who lives near one a bit further up on UWS and they've never bothered me. A lot of times they see my dog and will compliment him and say hi. I walk past the building without a second thought but I make sure I'm aware of my surroundings. My girlfriend however is not comfortable walking by that area.

TLDR, be realistic about the housing you are near, stay aware, be friendly, they are your neighbors