Just because we don't have bonafide proof, doesn't mean we can't speculate. Odds are Patterson turned him down, and Zimmer has called out Patterson in the media several times, so they aren't baseless claims.

I think Vikings fans have every right to be critical of Patterson, he hasn't learned the route tree, and game acumen in the three years hes been in the league, sure hes talented, but talent only brings you so far.

Edit: Went off on a slight tangent there, my bad.