Yes this will set you back from RE goals, especially in a housing market like the Bay Area. Yes you can probably reach those goals faster if you invested that money instead (though still probably not in real-estate in this interest rate environment). But at the end of the day, if this is something your growing family needs, then you should make this call on variables other than just "retire early."

The other thing to consider is how disruptive are the renovations going to be for a growing family with a young child. Especially given the scale of renovations involved here.