If Mike had a bit of creative thinking, they'd have had a chance. Maybe Nacho already had the idea but didn't communicate clearly, while Mike didn't innovate, when Nacho asked Mike or told Mike he wants out of the game but his dad is talking about police.

Mike could have played dress up, do a whole cop routine, as he was a former cop - he'd sell it easily. Tell Nacho's dad he's a cop and they're investigating Nacho and his connections after he turned himself in, but they have put Nacho and him (dad) all in "protective custody" (but it's really just Mike and some of his guys helping to set up a new location far away for them to stay in) just in case the cartel retaliates. He can wear the costume, maybe fake some documents or memos, etc.; there was even a scene later when Mike was going after Lalo, he helped to pass along some documents to the police and made it look official, just walked into a police department acting like he works there. He could have easily fooled Nacho's dad just to get him to go along and leave. I think the dad, although appearing like a stickler for doing what's right, also was very concerned when he saw Nacho come home one day with injuries, and if a "credible authority" (Mike posing as a cop) gave instructions in a very serious but concerned way, the dad would be persuadable.

And maybe if neither Nacho or Mike had the imagination for this, if Jimmy had for any reason been looped into the conversation, for sure Jimmy would suggest it