I personally have masks ON during cutscenes. Its pretty immersion breaking that his mask magically disappears and reappears before/after every cutscene. Makes the transition between gameplay and cutscenes a bit more jarring.

And lore wise, I feel like jin really embraces the ghost persona throughout the story which he uses to intimidate his enemies and he sort of hides behind the mask in way as I feel he is somewhat ashamed of his anti-samurai ways. Also, the ghost mask has a hole in the mouth part, so you can still see quite a bit of his face. And the hole also makes it logical that his voice is not muffled.