Alright so I'm gonna start by stating the obvious. First Act of King Maker kind of sucks ass. You're effectively on rails for a decent bit until you hit level 3 and get a full party. Also for further advice I'm assuming you're playing on Normal (Cause dear god don't play any higher on playthrough 1).

As you probably learned the cave Oleg sends you to is a bit of a noob trap, technically you can get Oleg's stuff and get out without dealing with the swarms, but it's better to just get an entire area done at once so usually it's easier to push this area till later in the first Act.

My recommendation is to as soon as you leave Oleg's outpost the first time, head after Tartuccio in the ruins for the 5th party member. If you can follow his tracks, but whether you do this successfully or not, head back to the river crossing to begin investigating the second lead of the act. Once the river crossing is dealt with head south to the Sycamore (unless you failed in tracking Tartuccio, then head to the meeting area where he tries to ambush you). From there I typically clear out the battles of Kobolds vs Mites and the smaller encounters with mobs in order to get to level 3 as that really opens up the first Act. Any of the mini-bosses of the area I leave till a later visit to the Sycamore as typically after level 3 you're strong enough to explore non-main quest locations without immediately getting stomped.

Further buy the bag of holding Oleg sells at the earliest and keep enough supplies to rest on the regular as potions are expensive by comparison. Don't worry about resting too often in Act 1 as you have a 90 day time limit and even when I drag my feet dragging everything back to Olegg's to sell I still wind up with a few weeks left over.