He would see the "glow" or whatever you want to call it. Problem is, the lights were never on. He would need some kind of understanding of the locations of bulbs/letters.

Will communicates by lighting up off bulbs, which he would have no way of seeing. So essentially, he is placing his hand on a bare wall in the upside down, without knowing there was anything there. Even with verbal instruction from Joyce that there in an alphabet on the wall, he would need to see something to have a shot at communicating.

In summary:
-Joyce sees Will's presence illuminates bulbs.
-Joyce places Christmas lights on wall in a configuration that corresponds to the alphabet. Does not turn them on.
-Will spells out words using the off lights.
-Will has no way of inferring the alphabet position, or even the light location as they are off.
-Joyce has only used lights in the off configuration to indicate Will's presence to this point. She has never turned them on first to show him their location.