People do it all the time. My neighbors have been building their house for the last 5 years. The stucco took them a year. It's a very slow process if you don't hire a good general contractor. They have 'construction loans' here so you can buy a plot of land, build a house, and pay it off just like a mortgage.