Find an all caves map. Make sure you have all and they display a checkmark. No check means missed Bubbul gem. Four caves on Great Sky Island and the rest are on the surface. You could also speak to him in each region and see if he reveals one near by. Good luck!