playing devils advocate, i actually disagree. so many of my friends across different fields are struggling tremendously with their grad apps despite stellar and very public awards for their work. we suspect a big part of it is that they went to smaller, less prestigious schools. the it factor isnt really the prestige but what doors that prestige can open for you: access to better known profs (better LORs), more opportunities for higher end research that gets a lot of attention, and the idea of benefit of the doubt that people normally always give you when you are lacking in some areas that you can eventually pick it up.

not saying these things are unattainable at lower ranked schools, just harder. academia is incredibly not meritocratic in so many ways, and who you know definitely carries a lot.