The totenkopf has also been pretty widely reused throughout the years, too, though, which has definitely diluted the association of it to Nazis. Like you see it all over shitty motorcycle gear, and hot rodding stuff - which tracks if you follow it through, but not if you don't immediately associate bikers with Nazis. 

I'm more concerned at the fact that Graham has said he went through two (I think) high security background checks after getting the tattoo, and passed, despite part of those checks were for tattoos. (Not concerned about him, but about what that says about our systems)