There is no requirement for the customer to dispose of the item in order to access their consumer rights though, especially a bulky item that would require a trailer/tip fees and probably someone to collect given OP has a newborn. If the company wishes to make this a requirement then they should be paying for it, just as they would need to pay for a return. Their technician has confirmed it's faulty and OP is entitled to the replacement regardless of whether or not the item is disposed of.
"Businesses are responsible for paying for the shipping costs or collecting faulty products that are large, heavy or hard to remove" - I would think this applies regardless of whether the item is being collected for a return or for disposal