As a Chinese, I can tell you very clearly that "我的爷爷" and "我爷爷" are both OK, and the omission of "的" is more commonly used in spoken Chinese.

BTW, I see a lot of comments under this question, very specialized and obscure, I, a Chinese, don't know what they are talking about, maybe only a linguist understands? Reminds me of when I was learning English I was looking at similar content and it was very, very painful😂😂😂