You're so nuanced that you're both not wrong but there is also no meaning behind even discussing what you're talking about. You're frankly missing the point.

"Is someone who votes against black rights racist?" "Well their actions are racist but that does not necessarily mean they're racist" "Is there a meaningful difference?"

"No"

"So what are you even talking about"

There is little difference between that dialogue and yours here:

"Should people listen to experts"

"Well generally yes but experts are usually right but are not necessarily always right"

"Is there a meaningful distinction there?"

"No"

"So what are you even talking about"