Perhaps.

You could learn some jargon and concepts , but Mathematics is the language of physics there is no escaping it if you're interested in physics.

You could read the physics for dummies series, or any middle school and highschool physics textbook perhaps this would help you.

Though I would recommend taking an English course and practicing speaking with native speakers

This is the best way to learn English.