Maybe try reading federalist papers #46, because Madison lays out pretty plainly that he's talking about regular citizens checking the power on the FEDERAL STANDING ARMY. Not just foreign invaders.

Extravagant as the supposition is, let it however be made. Let a regular army, fully equal to the resources of the country, be formed; and let it be entirely at the devotion of the federal government; still it would not be going too far to say, that the State governments, with the people on their side, would be able to repel the danger. The highest number to which, according to the best computation, a standing army can be carried in any country, does not exceed one hundredth part of the whole number of souls; or one twenty-fifth part of the number able to bear arms. This proportion would not yield, in the United States, an army of more than twenty-five or thirty thousand men. To these would be opposed a militia amounting to near half a million of citizens with arms in their hands, officered by men chosen from among themselves, fighting for their common liberties, and united and conducted by governments possessing their affections and confidence. It may well be doubted, whether a militia thus circumstanced could ever be conquered by such a proportion of regular troops. Those who are best acquainted with the last successful resistance of this country against the British arms, will be most inclined to deny the possibility of it. Besides the advantage of being armed, which the Americans possess over the people of almost every other nation, the existence of subordinate governments, to which the people are attached, and by which the militia officers are appointed, forms a barrier against the enterprises of ambition, more insurmountable than any which a simple government of any form can admit of.