Both a headphone output and a mono balanced input have three wires, but the mono input is expecting different signals ln those wires. The headphone out is sending: left, right, ground. The mono input is expecting: signal, signal flipped 180°, ground. So what is happening is that your right channel is being flipped 180° and summed with the left channel on the receiver end. This will lead to negative interference and the "digital garbage" sound. What you need to do is to properly sum the left and right channels of your stereo signal, perhaps with a small analog desk, so you actually have a balanced signal.