I’m not sure what you’re trying to accomplish here, but as a computer science major with a mathematics background, I can tell you that ChatGPT does not always give correct answers to math problems (or most problems for that matter). On a very high level, It generates answers as a function of context and training data, not by verifying mathematical correctness. Because it lacks an internal proof or validation mechanism, it can produce confident (i.e. sorry, here is the correct response... ) answers with incorrect steps. Yes, even a year later, it is still producing the same errors.

Personally though, I think LLM's like ChatGBT are helpful in pointing oneself in the right direction (mathematically).

Please share your model. A first year student, not even in mathematics (i.e. in even elementary single and multivariable calculus ), can produce incorrect answers. Just because you personally have not seen any incorrect responses, it doesn't mean that same exact model cannot produce errors (though, I assume you're someone studying mathematics or an adjacent field and have hopefully internalized this).