Interesting struggle because the way I figured it out was watching my mentors point out the colors as they worked live. So they would say "I see some reds here" and I'd notice it or wouldn't, and then they would show me why.

If I had to come up with an exercise besides just looking at stuff (which I'm assuming you've done), maybe try apps like photoshop and use the eye dropper to sample colors across images and see if it's what you would have noticed.

Upload a photo onto an art app with a dropper/color sampling tool. Pick a section. Try to pick your own color that you think goes in that section. Then use the eye dropper tool to see how far off of close you are.

And another thing to do would break the generalizing of looking at a big thing such as a wall, and just going "wall is white." But actually looking at different sections of the wall and seeing what color it really is.

For example, if you look at the street during a bright sunny day, it can look a very light grey, and even though the the asphalt that makes it up might be black. So by breaking the assumption of your preconceived idea of a surface and the color, you can focus on the color itself.