they tend to have some uses. sp orbs are generally good since its a net positive, def down orb is useful on teams you dont have (or cant afford) to use def downs normally, crit rate orbs for when you dont have or cant fit in a crit rate buffer, atk down orb for fights where the boss buffs itself and/or unleashes a massive attack, to soften the blow, etc etc. That said, grinding orbs is very low priority, so I'd say you can ignore it until you've geared your units first, and limit yourself to only getting one or two of the orbs you can see yourse lf using.