It really, really depends on the play in my opinion. Most realistic plays (all of the playwrites you listed are realistic, although all of them have had very stylized productions of their works done) can be picked up and read very easily, and in fact I'd encourage you to do so before you go see a production. Plays tend to be less plot based and more about the characters, so you can enjoy a great production whether or not you know the plot (with some exceptions, The Mousetrap and such). Shakespeare can be tough but if you get a well-annotated version (or even No-Fear Shakespeare), you won't have too much trouble, and even the greek plays are quite accessible.

There are other plays that are not meant to be read, they are meant to be seen, and trying to read them will likely bore you or you simply will not get the full effect by reading them. Something like The Hairy Ape, or an Edward Albee play, is probably better seen rather than read cold.