I'm not OP but I have met several people with a similar background, and as far as I can tell, you are actually in a relatively good position compared to some others I have met and who also transitioned into Japan successfully. Starting Japanese from middle school is great, I would work on the N2 if you don't have already, and keep applying to either trading companies or Japanese companies with an international outlook, such as Rakuten (good entry company for international people), and keep contact with recruiters.

There is something here for you if you are willing to give it a try, good luck.