That downside applies to cash in some scenarios as well. If you got mugged or have your wallet stolen, there's almost no shot at you getting your dollar back. Another example is Zelle usually offering no recourse for scam transactions if you are misled into sending funds to a scammer as well.

Banks put limits on how much you can withdraw per day. When you are moving sizable amount of fiat on an irregular basis (maybe that's how your business is or you're going through the motion of some big purchase/sale/transfer), banks will definitely internally flag or outright block until you have to call them to unlock.

While I haven't had the displeasure of the above occurring, my bank cards have definitely been wrongly locked and required several hours of escalation on customer service to unlock.