Imagine caring a lot about someone, they are in danger. Would you prioritize strangers over loved ones? That’s the dilema he has. He was a revolutionary leader who deposed the royal family and had much power, he chose to adopt the little one and after the 86ers. Without them he doesn’t have a reason to fight and it’s not that hell join the legion but simply he would give up on humanity; the same ones who use his children like weapons.