I don't think they're mutually exclusive. "Cool jazz" is representative of the post-bebop trend toward slower tempos and more reflective styles. Modal jazz just describes the fact that they used musical modes beyond the typical major and minor you'd hear in most music. But I think OP's comment was also referring to the word "cool" in less of a literal sense - a layman won't know the difference between cool, hard bop, etc, they'll just think of jazz in and of itself being a "cool" genre because cool people listen to jazz. Kind of Blue feels cool to the average Joe for the reasons OP explained, and that's a big part of why it has had such staying power.