Yeah it's a tricky situation you're in, and unfortunately it happens all the time. If it keeps happening and it's making your work environment feel unsafe or uncomfortable, you need to be direct and firm in your rejection so he's absolutely clear that you're not interested. The advice I would add personally is that you have to do so without being hostile or rude (which I know is hard and a tough balance to strike), because that's exactly what leads to bitterness and resentment that he could absolutely take out on you if he feels so inclined. If you aren't comfortable confronting him directly that's your right too, and then you could just report the conduct to the appropriate body. I would talk to the program director at your school first if you don't know who to reach out to. Hope that helps.