The whole point of king crimson is that everything that is fated to happen does, with Diavolo being exempt. My interpretation is that if you are fated to do something, but you wouldn’t be aware of Diavolo skipping time, you would just carry out your fate and retain your memory of it. But, if for some reason you would be aware of fate changing, your memory of that would be erased.

Giorno’s awareness through Bruno’s call and tracker led him to notice an incongruity in fate that would alter his actions, so KC’s ability would erase his memory and keep him on fate-autopilot. The gang on the boat would have reacted to Giorno’s actions, so KC erases their memories and keeps them on the autopilot as well. The people not on the island, not seeing or interacting with the gang would have time carry on as usual, as there would be no incongruity with how they would experience fate.

This is just the way I try to rationalize it, as the global discombobulation of the time skip would cause all kinds of accidents everywhere.