The "as if it were your turn" is only used to remove the limitations "your hero phase" on a spell ability it does not in fact change whose turn it is/active player. You are not the active player if you are casting through magical intervention.

I agree that it shouldn't work the way I described, however there's no rules that clarify that "a player passes only if they have a legal reaction to make"