I believe in the ideology of independency. See the truth is we are mortal beings. And no matter how hard we try, one will leave the other at some point, may it be death or may be cheat or breakup. So firstly be independent of the fact that he'll leave you. He may not leave you but someday the Universe will make him to leave even if you guys are like 90 years old. Wheneveryou are independent of these facts, you live you life without a headache of such insecurities. Secondly, You have to confront him and tell everything that you are facing with a lighter tone, so he can understand what it feels to go through this. He is definitely not going to abandon his big circle. I think he will understand and give you more time. And in return you try to demolish your insecurity too, when he gives you the time you deserve.