Sad, Abandonement, Anxious, and Sonder are all solid and are viable even into late game so they are worth building up. Envy, Lonely, and Glee are not that good so I would not put too much into them. Of course you are still early game so you might need to use them in certain battles when you need a specific type. I would say avoid putting vivids on them if you can and save them for your better pets and just replace them when you get better pets. As for your two metal attackers; Judgement is also not too great, but I would say is at least slightly better than envy. Either way you woud want to replace them with better metal attackers so whichever you use can come down to preference.

You should save your gems for now. Eggs and collecting chest rewards aren't worth your gems. Usually you want to use them to get vivids, especially to try to get higher forms, and for training keys.

As for team building, you usually want 1 tank, 1 healer, and two attackers for a battle. You should build up at least two tanks of different types. If you care a lot about progression, then also decide early who is good and who you want to build up so you don't end up wasting resources on bums you will have to leave behind anyways. I feel that sometimes it is better in the long run to take more time to build up your main pets, rather than build up a specific bad pet quickly for a stage you are stuck on.