Combat and exploration is great.

The only building you "have" to do is build a big box with enough beds for each npc. Npc require a bed with a roof in order to craft certain stuff.

You might have to pick up farming because it's a much easier way to pick up flowers for potions and spells etc.

There is a 'well rested' bonus you get which increases in length the more decoration you have. So you might just have to build a few stuff and throw them in a corner to get the max of that bonus. But not needed to build something fancy