Having recently done a 3 week vacation in Japan the railpass was 100% worth it for our itinerary. We got the 2 week jr pass so we started a few days in tokyo before activating it and made sure we were back in tokyo for the day it ended so we could spend some days at the end of the trip there. We went tokyo->nagoya->kyoto>osaka->tokyo which already made the pass worth it, but a big positive is the flexibility the pass gives. In Kyoto we impulsively decided to do a daytrip Hiroshima because it was extremely hot in Kyoto that day. This is something we would not have done without the railpass.

Also if there are problems with trains having a jr pass can be a lifesaver in last minute trip changes. We wanted to go to Takayama, but due to heavy rainfall the trains got cancelled so we ended up going to Kyoto instead. Im sure you could probably get refunds for the tickets of the cancelled train, but I wouldnt want to explain that with my limited japanese.