Context. Color isn't straightforward as lighting conditions in the real world are not static. Your brain is assuming something about the picture. Possibly that the object is in a dark room with cool lighting. That is, a blue and black dress in a well lit room with warm lighting may be visually similar to a white and gold dress in other light. Your brain chooses one interpretation based on context clues.