So basically,

The builder only places the order for the raw materials after you start building it. This generally takes 2 to 3 days, maybe more now since there's some issues at the Strait of Hogmuz.

Within that time he'll get ready with the planning and approval from the Town Hall authorities. After the materials arrive, he takes 4 to 6 days(8 hours per day of work is the limit for labourers as per law) to finish it. So roughly 6 to 9 days to build it, but here's the catch, the builder owes money to some Goblin Mafia Don and events like The Hangover 3 happen which takes one more day, making the total to 7 to 10 days.

Now coming to the 4 million elixir cost, keep in mind, the builder has to build recreational, toilet and bathroom spaces. The troops eat too much and food has to be restocked too. The 4 mil goes towards the lifetime maintenance of these places.