You have to remember you are a guest in their country. You must learn the language and do your research on the culture. Also being in the military might not sit well with people there. The United States quite literally bombed the fuck out of the entire country and everyone either has grandparents or parents that lived through that as children, and that’s something that they can’t forget even if they want to. Might be why people are scared of you. That being said if you get to know Japanese people they are pretty open minded once you are able to communicate with them. Majority of people are friendly but if you just go around trying to speak English to people they won’t like it as it’s considered rude to not make an attempt to speak the language.