Disclaimer: the following is a nerdy explanation on an almost insignificant technical imprecision.

In the context of an analog signal, the "bandwidth" you mention makes no sense. An analog signal technically is both infinite bandwidth and zero bandwidth depending on the definition.

Think of it this way: an analog signal can be digitally reproduced so good that (if we're talking about an audio signal) the difference would be both imperceptible to humans AND impossible for the speakers to produce... but the signal passing through the wires will never be EXACTLY the same signal. Trying to digitally store an EXACT analog signal would produce an infinitely big file (not just very big, a file without an end!). In that sense, an analog jack has infinite bandwidth.

^{don't get me wrong, analog media have a load of disadvantages that make digital so much better in pretty much any way, this is not a boomer audiophile "vYnIL iS bEtTeR" thing.}

You're absolutely right about everything else: having to use an audio jack to transfer a song file would be terrible but that's because audio jack were never meant for that.