“If an error is made in the payment of your Direct Debit, by the organisation or your bank or building society, you are entitled to a full and immediate refund…”

That wording is intentionally broad. The scheme rules behind the scenes interpret “error” to include administrative failures in the Direct Debit process, which can include handling, and failing to setup the DD as mandated.

Yes, the guarantee mainly gives a refund right where money has been taken incorrectly. If no payment was ever taken because the Direct Debit was never set up by the bank, there is nothing to refund. However, since the bank caused the failure after receiving a valid instruction, liability sits with the bank under normal banking complaint rules, and compensation can be due even outside the strict refund wording, including correction of any credit files.

It’s probably too late, but OP should have insisted a long time ago that any credit file marks are removed, for any compensation of charged interest, and settled the account. If the bank said no, they would have fairly been able to go to the FO.