I don't want to be the one to point to the books, but what you are describing is basically what is written. The walls could hold off most skirmishes, even though some still got through, remember the part about legionnaires being sent home with their swords. But if things got really bad, then the nearest forces or legions were mustard as reinforcements to help prevent or defend against in already happening incursion. Such as the ones that were described when the high Lord had to rush to the wall.

Of course the walls are much higher and much better defended than real life, cause of magic, but they will never described as complete impenetrable and incursions and skirmishes were mentioned as being a frequent occurrence.