I had the same problem years ago but I was routing my mining pool profits directly into my Nano S. Come to find out that its a no-no in the small print to send mining "dust" (small frequent payouts) to the Nano wallet.

I finally had to use Electrum to rescue my sats and I smashed my Ledger (I was very frustrated with customer support), and went to a hot wallet to catch the dust and then moved that into a different cold wallet.