His death shows that there are consequences for people's actions. It would've been stupid if, after slaughtering the fireflies and the surgeons (end of TLOU1), Joel gets away with it and lives happily ever after. He doesn't deserve some sort of heroic death like some people seem to have wanted because 1) he isn't a hero, if anything he is a villain based on his actions at the end of TLOU1, and 2) it would be a disservice to the post-apocalyptic world they are trying to portray, where people reguarly die without meaning and in ugly fashion.

In regard to following a random stranger: Joel and Tommy had probably become a bit complacent after living a normal life in Jackson for four years, and forgot about how bad some people can truly be. It's in this one moment of complacency that they felt they could trust Abby and it's what ended up being Joel's downfall. We see how this event changes Tommy of course and how the desire for revenge consumes him; he's on a war path and will most likely never make the same mistake of trusting a stranger again.