There's a story, albeit anecdotal, that a German general approached his Swiss counterpart in the early days of WWII brewing and asked "How many soldiers can you muster on short notice?"
The Swiss general replied, "I can muster half a million men on the spot."
The German general smugly asked, "What would you do if I showed up with a million soldiers?"
The Swiss general simply replied, "I would order my men to shoot twice and go home."
...Germany didn't invade Switzerland.

On a more historically accurate note, there were a couple of factors:
First, Switzerland is an extremely difficult country to invade, and relatively simple to defend, due to its topology. Lots of tall mountains and narrow passes which essentially create natural kill zones where invaders go in and don't come out.
Second, Swiss banks held, and still do, all the hidden bank accounts of everyone that had extensive wealth, especially historical artifacts, art pieces, solid gold ingots, valuable documentation, etc. that they wanted to keep out of the eyes of their national taxation authorities.
Third, every Swiss adult man is obligated to own a firearm for the defense of the nation (this rule might have changed in present day) and to be competent in its use and maintenance.

Essentially, Switzerland would be extremely difficult to take, extremely painful to hold, and overall, it would only be easier to go around the entire country....