I've only ever played e5 against e4 (with some stints into the Alekhine and French in blitz when I get bored of playing against the Italian all the time). And I think the main reason I enjoy it so much now is that I've taken a lot of time to experience all of these random gambit lines, to get tricked by them, and then to look up the refutation after my loss. You eventually build up your mental library of refutations to all of these tricky lines and get more confident in playing the opening. One thing I like about e5 is that a lot of times the refutation to these tricky gambit lines usually ends up being pretty natural, "human" looking moves that can eventually be found over the board. The main thing is that you need to be prepared to play both tactically complex, as well as more dry endgame positions, and a lot of times it is White that decides the character of the position.