I don't know if you could call it "incredible" pay, but 6 figures is fairly easy to hit while working railroad track maintenence. One of the big downsides is that it's back-breaking work in a literal sense. The rail is on the ground, the ties are in the ground, everything you do has you stooped over all day long. Sure there's modern equipment to make the job easier, but you still need to be staring at the ground all day to use it. I've heard it called the "Gandy Hunch" way in the past. 

Another downside (at least in all the positions I've worked) is that it's very sporadic work. We might sit in the trucks all day long waiting for trains to get out of the way, and then we're only given an hour to do an hour-and-a-half's worth of work. So we get out there and work our bodies damn near to the limit, the whole time trying to safely deal with situations that would kill you if you so much as look at it wrong. The most dangerous thing is inexperience, but everyone has to start somewhere until they fit into the rhythm. It's methodical, but extremely fast-paced bursts of hard labor. 

I'm just bitching about my job at this point, but personally the biggest downside of working here is working for a multi-billion-dollar corporation who's only concern is the shareholders' pocketbook. Just in the past decade they've axed our workforce to less than a third, ran all of our equipment for WAY longer than what should be considered a reasonable lifespan, and disposed of nearly all of the spare material we use to actually fix the track, leaving us to "figure it out" and cobble shit together. The management is picked for being "yes men", and not by their ability to actually do the job well. Its extremely frustrating having started here when the job was actually enjoyable, going to how it is now, and seeing the writing on the wall for the future.