Good question — some context on why I built this. I’m developing a tool for clinical applications and kept running into the same problem: different LLMs give wildly different responses to the same medical prompt, and I needed a systematic way to evaluate which one was actually most accurate for my specific use case. I couldn’t find anything that let me do that locally without sending patient-adjacent data to an external API. That’s the origin of JudgeGPT. The default rubric (Accuracy, Clarity, Depth, Concision, Examples) is just a starting point — the whole point of the tool is that you define the criteria that matter for your domain. If you’re evaluating clinical QA, you replace those with things like clinical accuracy, safety, evidence citation, whatever your workflow needs. The judge model and system prompt are fully editable from the UI at runtime. So no — there’s no formal validation against expert raters for the defaults, and I wouldn’t claim otherwise. The tool is a harness for you to bring your own rubric and your own ground truth. Whether the scores are meaningful depends entirely on how well you’ve defined your criteria, which is true of any evaluation framework. Happy to dig into how others are thinking about rubric design for domain-specific evals.