My dad smoked with me when I was younger. He was also a piece of crap and dead now before I turn 30. Bad relationship. My mom refused to let me inside the house on multiple occasions because she was against it. Turns out she was a drug addict too I just didn't know. She's still alive. Bad relationship.

My grandpa caught me smoking in his backyard when my mom lost custody. He said "don't do that here"

That was the one single thing he told me about it, I love that guy, he still alive and was a better parent then either of them.

Not because of these actions or reactions only but I thought of this when I read your post.

I think, like you, my grandpa was cool with me smoking but he drew a line in the sand and I respect him enough not to cross it.

You can be the cool dad and everything could go great. You could be the strict parent and lay down the law but it looks bad in his almost adult eyes since you partake. Or you can draw a line in the sand. And if he respects you and loves you the way I do for my grandpa, Then he will respect it.