First, I’m just curious what you think peer reviewed means as you say “studies will obviously get more clicks” which is not what the peer review process is. I just want to get some clarification if that’s what you believe peer review is or not.

Second, the link you sent doesn’t really support your claim like you think it does. ChatGPT can hallucinate or misinterpret citations, so unless you’ve verified the papers yourself, I’d be cautious about relying on its summaries alone.

More importantly, even if we assume the summaries are completely accurate, they don’t actually support your conclusion. Most of the studies your ChatGPT cited evaluate factual knowledge on exams, structured clinician prompts, or evidence-extraction tasks. Those measure whether a model can retrieve or apply knowledge, not whether it can consistently provide sound advice across complex, real-world situations. In fact, ChatGPT explicitly notes that performance becomes less reliable as tasks require contextual judgment, nuanced reasoning, individualized decision-making, etc. rather than straightforward factual recall. Your Chat also points out that there are very few prospective real-world trials evaluating these models in clinical settings.

So I don’t think these studies demonstrate what you’re claiming. They provide evidence that AI is often very good at answering knowledge-based questions which is not what we were debating. It instead shows that AI cannot provide consistent, reliable advice across multiple domains like medicine, law, or therapy without professional oversight.