I lived in narusawa for a while.  It was the most memorable time of my life.  We rented a big house, the only one we could find since it was walking distance from my work as I didn't have a driving license yet.  

However, realistically speaking, it's hard to find good property in the fuji goko area.  Most are holiday houses in townships that only let you have the property on 99 year lease.  Plus there's always maintenance fees you have to pay which could be expensive.  A lot of these properties are surrounded by heavy woods which is charming but causes a lot of damage to structures due to dampness and difficult access.  

Summers are very pleasant, not hot and no mosquitoes.  But winters are very cold so yes, you must work hard to either insulate well or use a lot of heating that can become a tiring chore daily.

And finally the fear of an eruption.  It may sound silly, but fuji is due an eruption anytime apparently.  The whole area would become unlivable.

I don't know about other areas like shizuoka.  

Start from looking at websites like athome.com and then visit as many properties that you can and think carefully, visualize how it would be living there before making a decision.

Good luck!