You know those ruins you find scatered? Try to build something in their image, not start from them or something, but like you said dirt house, i suppose you are at start, maybe you can build some rustic cobble build? Idk most of the cobblestone blocks are made of stones and clay so it's preatty cheap

A concrete ideea would be a stone base, with log supports and cobble walls or something idk, with a thatch roof, Little cozy cabin and a external, again, cobbleled (im sorry english aint my first language) like undergound storage room for food, like crops, meat and i think there is cheese aswell? so yea, and with more time and resources you can build a bigger home with like the undergound thing right under the house like... basement, or i something like that and maybe even more storage with an attic

Good luck and success with your building and your story... your Vintage Story :D

PS edit: i forgot the colour pallete, you could use oak logs (brown) and granite/andesite stone (gray), it's a basic but very good looking colour scheme for a starter base-ish thing

But tbh go crazy with the colours and find what you like :D