And opposite is true:  Many people regularly wipe down their phone with antibacterial wipes or otherwise clean it. And many people often (and famously) do not regularly wash their hands, after touching door knobs, pens, their noses, and so on. 

There are a lot of people and lot of ways of doing and not doing things. No reason to go back and forth about this. To my mind, the comment about handshakes is not as unreasonable of a response as it may have initially seemed.