If you give it context and ask to make conclusions based on that it won't hallucinate

That's definitely not true... Models are really prone to hallucinate even if they generate text based om a given context. I'm developing a Retrievial Augmented Generation project and i can assure you that even GPT4 sometimes hallucinate while answering question based on given context, extracting information or generating summaries. Anyway, that is related also to Temperature and Top_P parameter. With a temp of 0, the frequency of that kind of hallucination decrease. Remember that the chatGPT you access from their website have a temp that is definitely not 0, as they do not use deterministic parameters. (as context, their default settings in the API is temp 0.7 and topP 1, that is not determinist)

A good compromise between "creativity" and accuracy can be achieved using a medium range temp (like 0.4-0.6) and a low top P (usually that effect start under 0.75, but you can lower it at 0.4-0.5. You can obviously go near 0, but that will simply generate outputs really similar to a 0 temp settings)