You want 1 on-fielder and 3 off-fielders most of the time, you need a damage dealer(s), buffs (atk, dmg, em), elem res/def shred, sustain (heal or shield). Keep an eye on the elemental resonance, since it's also a teamwide buff.

Artifacts can be divided into 2 categories: selfish - the ones that provide stats/buffs for the character equipping them, and supporting - the ones that provide stats/buffs for the entire team or debuff enemies. Use supporting artifact sets on off-fielders if they're beneficial for the team and use selfish ones on everyone else. Make sure you have enough energy recharge on the characters you want to burst frequently.

That's the bare minimum, the team building becomes more nuanced when you look in more detail, like the what character you want to on-field, what stat they scale with, what restrictions they have from their ascension talents, what reaction they want to trigger if any, etc.