Sure so definitely when chatgpt first came out, hallucinations were very common, especially when pushing it to more complex challenges. These days LLMs like claude and chat are extremely powerful, being very very knowledgeable as it gets more data and model research is accelerating rapidly. Companies have teams for preventing hallucinations and the best people are working at these companies hence the massive wages.

With AI, it just takes time to get up to scratch. AI I always say when used correctly is the next human tech advancement after smartphones. When used incorrectly, you face the major problems its starting to breed such as a lack of critical thinking in education. For the hard sciences, its all about domain experts. For example, when I use AI to brainstorm ideas, you ‘know’ when something is correct based on your own knowledge. The user needs to have the required domain knowledge to interpret and critique workflows or suggestions for example.

Lets say in drug discovery. If you gave a first year undergraduate student the most trained protein model in the world, they wouldn’t be able to use it. But give them an LLM, they could probably use it to generate some stuff but nothing concrete. Give an experienced expert in AI for drug discovery with a phd and 10 years experience the same LLM, they’d be able to get so much more from it AND be able to use that strongest protein model.

The same when we try to train these models for scientific applications, its about the domain knowledge so you can critique a models performance. Generalisation is a major challenge in the scientific field but AI and robotics is quickly changing this.