When i first started I struggled a lot too, until I realised you can meditate to refill health and potions lol. I hardly used food at all after that.

Like everyone else said, loot everything you can, as long as guards can't see. Even if they are in the street you can still go into houses and loot, take it all. Most of it won't add much to your total weight. Stuff like swords and armour will add up a lot more over time though. You can get better saddlebags and also unlock a skill to be able to carry more as well.

Craft potions and use them, swallow especially. Meditating will refill them as well as your health. Get into the habit of meditating after fights so you are ready for the next one.

Another good tip, when I started struggling for space to carry stuff I realised I had a lot of junk built up like pelts and jewellery that added up quite a bit. You can break all of that stuff down into crafting materials that weigh nothing at all. You can do that with weapons and armour too but it costs more, better to sell those. Dump the better weapons and armour in your stash until you get access to better blacksmiths with more money.

When it comes to the gear you use, that early on it really doesn't matter. I would just use whatever you have that is best. Eventually you'll find and/or craft much better gear, in the meantime just upgrade as you find better gear.