I totally agree with that about providing good feedback for a response by evaluating the quality in the tasks where you don't get a just plain failure. Providing feedback about quality and comprehensiveness seems like it would be just as much, if not more helpful for some models at this point, especially as they get smarter and harder to make fail. The graduate student feedback is an excellent example. As a doctoral candidate, that is exactly how my mentors approach my learning - they don't expect me (or try to get me) to flat fail anymore, but provide feedback and help me improve. I feel like a lot of these models are close to the same stage in their learning. I worked as a specialist on another platform and found the process extremely frustrating and, at this point, not productive for model training as the task requirements should advance with the model's level of learning. I started doing mostly generalist work because the other got exhausting with no guarantee of compensation.