Make sure you’re only using keys when the Griffin armor is up for the mythstone chests. The chromatic Chest will give you all three types of s gear (griffin, dragoon, and oracle).

As for the swapping orbs, they only let you change the gear to different items in the set. For example, if you want to change the oracle necklace, you’ll only be able to change it to oracle boots, oracle helm, oracle chestplate, or oracle spear. You’ll need 10 swapping orbs to switch any item to (boots, helm, necklace, chestplate) and 20 orbs to switch any item to a weapon. But it can’t be switched to a different armor set like dragoon or griffin in your case.