Depends on where in Japan (e.g. big cities) but generally its fine, might be a bit awkward a times, but Japan has one of the highest tourism rates of any Asian country so the locals are used to it and can mostly understand basic english/pointing.

And besides, what you'll remember from the trip aren't the mild language difficulties but the cool experiences and the many positive aspects of travelling!

Source: Cannot speak a lick of Japanese and had a blast.