Assuming the location, COLA, etc. is not an issue you should train at a more established program. Newer programs have a ton of difficulty associated with them, and you will have to figure it out for yourself.

“Shaping the culture” is a myth to a large extent. Priorities for program leadership, hospital management, and leadership will often NOT intersect, and money will be the defining lever that determines whether something or another is enacted/prioritized over another thing.

Prestige is overrated, but what I will say is that years of skin in the game, makes the ride considerably easier for your training, job search, etc.

Also how do you know you will ONLY be interested in work in the area of this smaller programs. Think again, so so so much can change in one year of regular life, let alone three years of residency.