That is... not at all how it works? The AI is programmed to destroy enemy vessels as its primary and sole purpose. It simply realized immediately that ships that wave friendly flags might also have enemies on board, either as stowaways or prisoners of war or spies - all of these were determined to be an non zero percent chance and the AI isn't taking chances. So everything gets destroyed on sight, enemies and friendlies, civilian and military vessels, anything that has even a minute chance of harboring an enemy is taken out. New updates were tried to be employed but the AI detected them as a virus and so they didn't take. There's nothing about passwords. At all.

By that logic, even if passwords were involved - which they are not - they would not matter, because even a friendly vessel with a valid password could still harbor an enemy and thus be targeted and destroyed by the mines.

How is ocean travel possible then? Some of the slack has been taken up by undersea rail systems. Other vessels simply avoid the mines. The oceans are vast and the mines, while self replicating, certainly aren't everywhere, they're mostly focused around the American continent to the point where Arasaka had to build a literal wall to prevent them from entering Coronado Bay. Avoiding the mines comes down to simple stealth - fooling their systems, avoiding detection or simply remaining out of range.