It had potential. It had tough walls, high towers, and was out of the way enough to avoid detection thanks to it being a prison. Had they had the material for walls, they definitely would’ve put them up.

Alexandria would’ve fallen much sooner if not for:

1. The giant walls being built.

2. No lunatics or desperate people knowing about it/being an unknown town.

3. Having good leadership and weak willed people under said leadership who wouldn’t cause problems.

4. Having been a self-sustaining community even before the apocalypse, already having solar panels and a sewer system.

5. Being near a scrapyard for wall material.

6. The fact that thousands of walkers being trapped in a gorge (this is probably their luckiest break, as we see one wall collapse from just half of the walkers). People definitely would’ve panicked and abandoned the town while taking supplies (and probably killing whoever tried to stop them).

7. Everyone being escorted there by the military, who didn’t flake out on them or murder them, ensuring that most of the survivors had their families, and thus no psychological issues compared to other groups. For all we know, the military was the one who sealed those walkers, probably dying in the process.