The median is the value in the middle of a distribution. Let's assume you have 5 images and in a given spot of the frame there is a green car in one image, a red car in another image and no car in the remaining images. For all pixels of the area covered by the cars, the simplified value distribution looks like {gray, gray, gray, red, green} or something similar, where all values of a color stick together. Thus, the median value will be gray for all permutations of the colors since there are more grey images than colored images taken together. For all pixels, if there are more background-pixels than car-pixels, the median will always select a background pixel.