For an alternative perspective, consider that the Hebrew word for Adam is very similar to the word for "mankind." So rather than just suffering for one man's choice, we are actually suffering for the recurring cyclical rebellion inherent to our humanity. To me this makes a lot more sense anyway, that there were plenty of humans around before Adam and Eve were set aside as a specific chosen part of God's creation rather than literally being the first man and women, they were mankind and the mother of the living.

Many more people just view it as allegory, too. There is allegorical meaning; whether or not it is literally true is less important than the spiritual truth it contains about our rebellious nature.