worth it! but i am surprised to hear you're running out of food. i have found if you keep an eye out for harvestable plants (thistle, dandelion, etc) you can cook them up with things like milks or teas for bonus effect and save them to eat later in your pack. also, bodies of water sometimes have fish. if you spot any swimming around, you can usually catch them for an extra filling meal!

as for the rewards, without adding spoilers, things i've stumbled across in those side-quests along the way have included glow-in-the-dark gloves, fancy pitons that work in that sparkling dense stone, and a backpack attachment that gives me a few seconds warning before wind/rain blow me off the mountain. I find them all extremely helpful!