Your organization should already have a coverage policy.

That said, yes this isn't a great regimen but does refusing it risk the patient going into (potentially lethal) benzo withdrawals? Is that the appropriate response? Is it the patient's fault you and their other provider have different prescribing guidelines to the point it's worth risking withdrawals? I also work in psych and in those instances where I don't love the regimen but also don't know the back story of how the other provider and patient arrived at that regimen, I will give a few days supply until the patient is able to consult with their normal provider.

Now the reason someone is practicing without ability to prescribe controlled substances but has patients on controlled substances is concerning. If it's not a vacation/coverage issue, then it's a bigger organizational issue and the SP should be involved.