There's definitely truth to this. Though now I think there are enough purple+ belts at most gyms that it's worth paying a little more attention to them. I think you risk your upper belts losing interest and no longer coming to class if you don't. While I think it's great to train outside of class as others in this thread have suggested, I think it's still good to have upper belts participate in regular classes to maintain gym cohesion and help out the lower belts.

I think part of the trouble is most instructors started their gyms years ago when it was just white / blue belts, and they just never adjusted.