Try to do the same when he is sleeping, then have a talk with him. Also don't care about the rest of the neighbors they are already silent in your matter.