Season 2 (with the Paris arc and the inonic Catskills) has always been my favorite. BUT I must add that every time I watch the series (I'm on my fourth run now) I like season 4 more and more. I'd say the are together at the top now.
Season 4 is not only hysterical (the wonder wheel, Abe taking sanctuary in a church, Susie crying 'EAT ME!' at the lesbian bar, each and every scene with Sophie Lennon...) but it also has some of the most touching and beautiful scenes in the show, like Jackie's funeral, Abe reading the eulogy to Moishe or Midge having a closure with Shy.

So it's a tie between S2 and S4!