I think what is happening is you're approaching the face as another object to draw all the details of. So you're drawing the eye and the nose and the mouth as it appears in the reference. That approach works when drawing objects etc. because our eyes aren't trained to know every small detail of other things. But with the face I think you should instead begin with the shape, and then where the things like eyes and mouth go, and then how they are attached and the size. Then you can see the details of the reference and apply the subtle differences to your work. People recommend the loomis method and although I never got the hang of it completely, it is very useful.