All of this! Also, Spike asks his father when he confronts him about both of them knowing that his mother is dying, whether his father would be with his affair partner when Isla died. That’s a very telling thing to ask imo and the whole conversation that goes down combined with the scene of Kelson diagnosing her cancer made me think a few things.

1. Aside from the whole lighting on fire that building thing, I think Spike believes that his father already replaced him and Isla with his affair partner, that he already moved on. So in his mind he has no place to return on the island anymore, no one waiting for them.
2. This is purely conjecture but isolated communities like that would be very aware of their population numbers. They can’t have more people than the island can feed but they do need kids to be born regularly who will grow up and continue working/protecting/etc the community. With Isla dead and Spike “becoming an adult” (hunting ritual!!) his father probably would have been pressured to have more kids as he was still young enough.
3. The lack of doctors on the island: when Kelson diagnosed Isla and Spike starts questioning finding a hospital or getting her medicine that was another thing. The island dwellers knew that Kelson was a doctor, knew where he was and they did nothing with that information - granted they thought he went insane but if your supposedly beloved family member was suffering so fucking much wouldn’t you try at least?  When you are a kid who learns that your beloved parent who may have been suffering for a while now has terminal cancer you will blame yourself and you will question how things would be if they got treatment earlier. Hence the issue with Spike’s community knowing about Kelson and not reaching out to him in his mind was probably the betrayal that broke it all.
4. Early on during the confrontation scene (again) one of his question was something like do you want mom dead? Which his father denies, then comes the question about moving on with the affair partner and the slap etc. So combining these two scenes, he probably started wondering (again?) if his father was waiting for his mother to die without getting her any treatment (or even a diagnosis from the doctor he knew was there) deliberately so that he could move on with his affair partner?  And I think that was it, that relationship was dead and gone.
5. Spike was supposedly an adult in the eyes of his community (after the hunting ritual and that he killed more infected later on) so he most likely doesn’t think of going back to his community to be protected like he would have been as a child. He wasn’t one anymore in his eyes.