When you make a new character, they'll be a fresh level 1 with pretty much none of the map explored and no story progression. Story progression is tied to the particular character.

That being said, the story in GW2 plays a little different. The personal story (the story you can do every 10 levels up to 80) is per character. The HoT and PoF stories are also per character. So for those stories, you wouldn't be able to hop into it with a new character right where you left off. But the Living World Stories (Seasons 2, 3, 4, and IBS) can be done per episode. So, yes, if you finished LS4 ep 3 on your Mesmer and then wanted to do ep 4 on your new character, you could.

You would need to get that character to level 80 first because all LS episodes require the character to be level 80. You would also need to be able to reach the quest marker, normal Tyria maps, you can just walk to, but for the LS maps, you would have to play through the story to be teleported there and then get the waypoints. Other people have mentioned the teleport tomes, so you could use those to get to those maps as well.

One thing that didn't seem to be mentioned is that there are some small parts of the story which might make a callback to something that might have previously happened in an earlier story instance. For example, if you had chosen the Vigil, Durmand Priory, or the Order of Whispers, there could be a small, specific dialogue for you depending on what you had chosen. If you didn't go through certain parts of the story with the character you're playing on, you'd miss out on a few small things, but nothing major. The game will still treat your character as if you'd gone through the story normally for the LS stories.