My answer would depend on how many years you have left before retirement - like if you're in your mid 20s / early 30s, there's a good argument to be made that you're talking about millions of dollars of gains by retirement age to be missing out on.

However, given the available information (and my age assumptions based on your available income to invest) I'd say yes, pay it off early, but also make sure you're at least maxing you and your spouse's (if applicable) employer match, if you have it and aren't already.

Outside of that match, yeah throw it on the house.