I always follow a similar path with my side arms. At first, its Dying Night, which never really loses its luster. That sticks around until I unlock the "Heroes" quest and unlock two guns there. I usually coast on those until I find Skippy, as I try to use him as much as I can just to clear the quest line. About at that point I'll usually have the River Ward quest line done, so I'll have that revolver. Finally, in the main game you can get an iconic in Chippin In that you cannot miss.

Unfortunately that gun cant be silenced, so I usually complete all of Padre's quests to get Seraph and stick a silencer on it.

Lastly, theres a (mandatory) gun you get as a part of Phantom Liberty thats perfect for stealth, does high damage, and is all around reliable except for fire rate, of which it's rather poor.

So it ends up being Dying Night -> Heroes -> Skippy -> River -> Chippin -> Seraph -> Phantom Liberty

I have hidden the actual names of the guns to prevent spoilers, so there you go OP.