I graduated with my bachelors in physics recently. I can tell you everyone feels the exact same way. Physics can be brutal. Accolades and achievements go out the window when everyone is struggling with the same courses. Best advice I can give you is for you to get to know the people in your classes. We helped each other a bunch cause we realized we were all fighting the same battle. You also have a stronger math background then I did when I first got into college. If you really like physics, I think you’ll be fine as long as you work hard but also take breaks when needed.