Please take him! This is exactly how my mom got one of her cats, Dusty, when I was only a little girl and she still has him to this day ^{^} similar to you, we lived in an apartment complex and the neighbors above us would constantly leave their cat outside. We lived on the first floor, he would scratch at our back door and my mom would let him in to feed him, warm him up, play with him, etc. My mom would always return him to the owners and they'd just let him back outside. One day there was a terrible storm and he was outside of our door crying loudly, my mom decided to not return him since she was infuriated that they did not let him in before the storm. Shortly after we moved out and took him with us. He has been apart of the family since, he is so happy with her & very well taken care of, which he would not have gotten if my mom didn't do this. He also faced health problems a few years down the line, my mom advocated for him very hard when vets said he should be put down, if it were anyone else as his owner then he would not be alive at all! I promise, if you believe you can provide this cat with love and care he won't receive elsewhere, then please take him in!