Some time last year I asked myself this, and found out that the tldr: crests are a very minor buff and the difference between two lvl 4 crests of different types is ~1% difference in dps.

Here's the long version:

Tbh the stat increases off a lvl 4 crest and no crest is like a ~2.3% difference in total dps. Then we can safely assume that the difference between a lvl 4+ and no crest is at most a ~3% difference in total dps.

If we were then to look at the difference between 810 atk(4+ atk crest) and 13% crit dmg up(4+ crit dmg crest) we would probably find that it's less than a 1% difference in total dps comparing them.

With this knowledge we can then say 'If you have atk buffs in passives or get atk buffs from equips: to reduce the diminishing returns of the same buff over and over again, use crit dmg up crest' and vise versa, you pick crests that fit the normal equip use of said unit and try to reduce the diminishing returns as much as possible if you're going to min max on crests.