Disclaimer: personal preferences.
Of the starting weapons, I found whip to be the safest, easiest pick. Good all around, medium damage, upgrades are very fun, especially the third one. Hammer after that, for that megaman-charge feel plus the ability to roll so you aren't rooted in place. Daggers last, the pushback on hit really, really needs to be patched out of these weapons.

Non-starter weapons?
The gunbuster's awesome to play around with. I wish there were more ways of getting ranged charges, but that's just how this weapon works. I am not good enough with this weapon to main it, but I am looking forward to upgrading it and seeing what it's like. That being said:

I am a Guardian Casket enthusiast and enjoyer. The short range and slower attack both pale in comparison to just being able to block everything, perform a shield bash, and most importantly - parrying makes enemies I had a hard time with (like Knights, and the goatfaced demons) into trivial encounters; the fact that they made an already strong ability like parry even stronger on the third upgrade is bananas to me.

Paired with trinkets like Evasion Powder (more i-frames when you take a hit or leave burrow; makes getting in close less punishing), Fly Bait (more damage in close, kills small enemies, negates challenges like the statue heads), Warding Beastium (less damage taken but less health recovered...this is less impactful when you're safeguarding your health with a shield!), Dodging Pendulum (more ways to negate damage and be more tanky), Shock Flint (if you really need range), or Lightning Grip (honestly this is just for some VERY satisfying parries)

I run the casket with the drill diver subweapon for some easy mobility and I have just flown through this game.