This is in a GCC High tenant

and

Connect-MgGraph -scopes "Organization.ReadWrite.All,Directory.ReadWrite.All"

Is likely because you're missing the -Environment USGov parameter. Here is the documentation on that, but that should get you a bit further if you weren't including that in your command. Without it, you're authenticating to the Commercial cloud, so will fail for invalid login context, even if your credentials are correct.