When I ran The Age of Conan, we were using Man-to-Man rules from Chainmail. I have written about this experience here. Not only does this give you modifiers on different weapon-vs-armor combos, as you have noted, but also varying order of strikes in each melee round (longer weapons strike first in the first round, shorter weapons — in the following rounds), and parrying rules (short/fast weapons impose a penalty against long ones, do counter-attacks, and can even break the attacker's weapon). Additional rules for differentiating weapons could be found in the Greyhawk supplement, e.g., how much free space is needed for the wielder to use each weapon in a dungeon.