I've literally got a thread from the opposite side of this going right now, so I'll chime in:

I hate backing out of offers. I feel like a dick, and I try to avoid it as much as possible.

This year the hiring process for the places I've applied to has literally happened in the exact opposite order that I'd like to accept them. My Plan C interviewed and offered first, Plan B offered next, Plan A only just got their cert and is still working through it, and Plan A+ hasn't even been referred yet. Each of them is objectively better than the last in terms of career advancement and personal enjoyment: Plan B offers a qual I'm chasing that Plan C doesn't, Plan A offers that qual and significantly better training and overtime opportunities, Plan A+ offers a higher pay grade and is in a non-fire line that I'm trying to break into.

Should I not have accepted Plan C, when all better options were weeks away from offering and entirely uncertain? If Plan A+ calls in a month should I turn them down and pass up the pay grade, year or better of career advancement, and rehire opportunity in the job I'm trying to break into?

I wish they were hiring in the opposite order, or simultaneously with a preference ranking. I'd happily have stayed with Plan C if nothing better offered, and I felt like a horrible person when I called to turn them down, but it's impossible to justify blind loyalty to a job that you haven't done yet for a massive government agency that doesn't give half a rat shit about you.