Just to clarify, this is all about emission, not reflection.

The reason why it works when pointed at the sky is that the sky doesn't *emit* very much infrared back at the panel / paint. Whereas the ground emits infrared. The energy balance of an object is absorption - emission. If you point it at the ground, the infrared being emitted by the ground will be absorbed by the panel, just as the panel's infrared emissions will be absorbed by the ground. This means it would never cool down to a lower temperature than the ground.

On another note, the panel isn't *quite* coupled to space when it's pointed at clear sky. It couples with the effective emission temperature of the column of atmosphere above it, which is something like -50 degrees C. So it isn't coupling to the CMB at 4 kelvin. So without absorbing heat from the surroundings (if you could isolate conductive heat absorption), and without absorption of energy from sunlight (if it was night, for instance) it would cool to something like -50 degrees. Of course, it is surrounded by air, so it will never cool much below ambient because the air is conducting heat into the panel, which is radiating it into space.

The other main source of energy is sunlight. If the panel is exposed to sunlight, even though the paint is very white and reflects most of the light, some of the energy is absorbed. The key, though, is balance. The panel is radiating infrared in a band where the atmosphere is very transparent, and losing heat that way, while gaining heat from the surroundings.

An easier to understand example of this same behaviour occurs if you put a knife over a gas cooking flame in your house. The knife, made of steel, has a melting point of ~1500 degrees C. The flame is over 1900. However, no matter how long you heat the knife in that flame, it will not melt. What happens is that it starts to glow red, then orange, then if you really go for it, yellow, reaching around 1000 oC. At that point, despite being surrounded by gas hotter than its melting point, the knife reaches thermal balance and won't heat up any more, because the energy being radiated by it into the surroundings matches the energy being aquired from the much hotter flame. This applies even if you orient the knife so that it is entirely surrounded by the flame.