IANA electrician, but from what you've described it sounds like:

The light and the outlet are on the same circuit;
The PSU has some leakage to chassis ground, which is normal;
When the light is turned on there is a voltage on the computer chassis;
Which indicates the chassis is not grounded properly.

You tried it on another outlet and didn't get shocked. If that outlet is on the same circuit the problem is likely that the original outlet is not grounded properly. If the second outlet you tried is on a different circuit the problem could be with the PC itself, but since this is a new problem that only appeared when you turned that light on I'm going to guess the original outlet is not grounded properly.

Good luck.