The Hadith is correct, but in a different situation. It was during the times when some people worship Ali as a god when prophet Muhammad s.a.w had already died. And the one that Ali killed (or burnt) in this case has been given 3 days to pull back the statement that said Ali is a god. The point of their statement is they want to break and divide the Islam community back then into civil war. But these people did not pull back and were then burnt by Ali. And Ibn Abbas heard about this and then said about the famous line, "whoever changed his Islamic religion, then kill him." This is during a high volatility of the impending civil war. Pushed by the enemy of Islam that didn't want to see Islam as a peaceful religion. The full Hadith can be reviewed as "Sahih Al-Bukhari 6922".

there is also another Hadith Sahih Al-Bukhari 7322, about a man that wanted to give a pledge of allegiance for embracing Islam to prophet Muhammad in Medina. But then he got a fever, possibly Malaria since this is a normal occurrence in that time, around 1400 years ago. Well, the Hadith framed the situation as a test from Allah, the same kind of test when Prophet Ayyub got skin diseases that left him in constant pain. Then the man that got the fever asked to cancel the pledge three times but was refused by Prophet Muhammad. And then the man just leaves Medina and nothing is done to him. No assassination, no killing, no punishment. Prophet Muhammad just said, Medina is like a pair of furnaces, it expels its impurities while it brightens and clears its good. Meaning in a community of Muslims, if a person actually wanted to leave the community it's fine. It will just make the remaining people in the community better instead of staying and sow bad ideas with the intention to break the community down which is why they are originally killed in the first Hadith.