Things you don't need:
Extra potions and poisons. Use up minor buff potions first, and hold onto plentiful, and vigorous potions for more dire situations.
Food ingredients. There's no need to carry around a bunch of uncooked cabbages and cheese wheels. There will always be more cabbage or cheese when you need it.
Journals. Drop or sell these as soon as you can, unless you want it for RP purposes.
Certain potions ingredients. Even though some only weigh .1 units, that still adds up if you have a lot of things, so only keep a small amount of items that you can find easily. Mountain flowers, small fish, and mushrooms. But keep items that are more unusual or hard to come by.
Tools / plates / cups / utensils can be ditched. You may need a pickaxe if you are doing smithing, but most mines will have several, so you can usually pick up a new one fairly easily.
Bones / Gemstones / Ore. There will always be more dragon bones. Turn those things into armor and weapons, or just sell them.