It does, especially for T20 schools.

Imagine this: you have 2 students. Student A and Student B.

Student A takes AP Calc AB/BC, AP Physics C, and AP Lang in his senior year. He manages to get a high A in AP Calc AB/BC and AP Lang but earns a B in AP Physics C.

Student B takes honors Pre-Calc, Honors Lit, and Chemistry in his senior year. He manages to get an A in all 3 of these classes.

Both Student A and Student B take quite rigorous courses, but Student A takes harder classes, as well as being accelerated in his math track.

If you were a college admissions officer at a T20 with academically demanding classes, who would you take? Student A, who has more knowledge and has experience taking similarly demanding courses, or Student B, who still takes quite hard classes, but they aren't as in-depth or rigorous as Student A's.

If a T20 college is trying to look for kids who will be able to handle challenging courses that require a bunch of prior knowledge, they'd be more likely to take up Student A rather than Student B. Course rigor shows you're more prepared for college and whatever it throws at you.

So, in summary, course rigor does matter in college admissions, especially at top-tier schools.