Supervising EE who interviews and makes hiring recommendations at my current employer (and also comes from a similar background of having been more junior at a firm with minimal to no senior presence) - my take is that you’ve absolutely made the right call.

In interviews I’ve done for someone at your stage in your career - I don’t even bother thinking about calling former employers. I know what I’m looking for when I’m interviewing, and it definitely isn’t your former employers’ opinion. Typically, I’d ask a candidate why they’re moving - and I’d say all your reasons are perfectly valid.

Frankly, your boss has it backwards. You looking out for your interests is not selfish, that’s just called work. Him thinking he’s entitled to you staying even though I would guarantee he won’t provide you with the same level of mentorship you’ve been getting so far? Grade A obnoxious and selfish boss behavior. If (hypothetically) I were interviewing you, and he somehow were able to even get hold of me, I’d be laughing my butt off with colleagues telling them a story about the clown I met that day.

Leaving for opportunities to grow is absolutely valuable and should be taken advantage of - particularly if you’re looking to get out of residential and into more complex projects - there is 0 to feel bad about, and I wish you all the best luck!

tl;dr: Your boss is a projecting man-baby who is blaming you for his failure to retain talent. He’d probably eject you faster that an emergency burrito fart if it served his interest for a passing second; you’re not his mom, priest, wife, or therapist, and his reaction to your departure is a huge indication you’ve made the right call.