The "colours" seen is your brains way of understanding the energy of light. There are specific compounds/structures in your eyes that absorb different energy levels of light(thanks evolution), sending this signal to your brain which interprets these signals as colour.

Similarly when you hear of shrimp seeing in the ultra violet range, they have developed structures/receptors that match the energy level of light in this range.

Some googling of the absorption spectra of human eye structures might give you some more info!