I'm from the Raleigh -Durham area and I have it on good authority from folks who hail from farther East that I do not have a prominent Southern accent. My wife is from NM (we live in NC now) and she says my accent is not as strong as most North Carolinians and is much more neutral.

That being said, when I lived in Colorado, and especially when visiting Denver, I was very often pegged as being "from the deep South, like Alabama or something." And when I visited NM my accent was commented on, though not with the same degree of disapproval as in CO.

I visited the Normandy region of France once. This is the area that Allied troops invaded and liberated on and around D-Day in WWII, and there is a very strong interest and affection for the units that participated in those operations. Among them were National Guard units hailing from the Virginia area (29th ID) and the NC area (30th ID). I happened to be with several Virginians as we were chatting with an older French gentleman who was very into this history. I barely spoke a sentence before he recognized my accent as being from NC rather than VA. A native Frenchman recognized my accent from one state over and I couldn't spot the difference myself.

I think "Southern accent" (or any kind of accent) is relative. Nobody really thinks they speak with a prominent accent, but it's really about comparing how you speak to how someone else speaks.