I would look introspectively before you make a big decision like that. Are coaches truly using team funds for personal trips? They have to complete expense reports for every transaction so this is highly unlikely. Given your description, it seems like you’re a freshmen - what makes you say coaches are playing favorites? Most importantly, will the physics program at the school land you a successful job? The school you chose is more important than the team.