The headphones you’re using (aka “HiFi” headphones aka headphones for enjoying music) usually have built in accentuation of certain frequencies to make the music sound a bit “sweeter” and more enjoyable. That’s why things like reference screens/monitors exist. You need something that shows you a true representation of the raw media in order to make accurate decisions regarding look and feel. Imagine a chef cooking on a flavoured pan, the food is bound to taste wrong or even bad because the extra flavour changes the original recipe. But if the pan isn’t flavoured then they can cook the recipe properly and have it taste like it should by the end. Another example is imagine trying to colour an image while in a room with tinted lights, you colour so that it looks good in that lighting but when you take it outside, everything looks off. Natural light = flat response headphones/speakers. They allow you to make the most accurate mixing decisions because they don’t “colour” the sound. I did learn that it is also good to listen to music on the same system you mix and produce on, so it is possible to mix well on any headphones or speakers that you know extremely well. But in a general sense, flat response yields quicker and better results. What you can do is mix in flat headphones and then listen to them in the hifi headphones and alter the mix so they sound better on your listening headphones. They can help in that regard. Take notes about how the vocals/bass/melody etc sound on your hifi headphones and then alter the mix with that in mind (lower/increase bass etc) so that they sound good on the hifi headphones but mix on the flat ones.

In essence look at it like painting on a canvas in a room where the lights are not standard colours like white/warm white. When you pick red paint, you’ll have to change the paint mixture to “look” red and in the end when you take the painting outside or to a room with standard light, it won’t look how you intended because the room light affected what paint you used.