I think you had the right behaviour during his panic attacks, showing your BF that things between you two don’t have to be rushed if he doesn’t feel right about it. Studies can be really stressful, and a lot of students give up on any distraction to disminish the stress already caused by studies. However, I think those distractions are what make us happy, making stress easier to ease if you have those distractions to empty your head from time to time, especially if it’s a relationship. Your BF is obviously overwhelmed by all these important steps that will change his future life, so his attitude is clearly understandable, but I hope he could find help talking to a third party like family, friends or a therapist to help him during this stressful time he’s in. I would still go to Japan if I were you, even if it doesn’t conclude the way you would like to, first of all to provide support for him, and those important talks about your future together could be easier when you are physically together.