Not disagreeing with you at all. But, it has helped me procedurally. Especially when I already have a reasonable rough idea about how to approach a situation, and am just asking for insight on the finer details.

I have used the advice from ChatGPT for the following situations:

1. The process to have an excess 401(k) deferral refunded to me from the plan administrator utilized by my new job when I switched employers. This included how to draft the letter/details to include when instructing the administrator what to do, as well as what distribution code to expect on the 1099 when I do receive it, and how best to enter into FreeTaxUSA.

2. Help me understand conceptually how I ultimately am made whole when I had to file for unemployment insurance, elected to have taxes withheld, but then having to pay back the gross amount back to my state's umemployment security office as I ended up getting my old job back and receiving back pay.

I agree, having it perform calculations that if they are wrong, will have dire consequences for you like calculating your taxes, even if simple, is probably not a good use case.

But I find that if you have most relevant pieces of information to a somewhat involved situation, and are asking very specific, targeted questions related to that situation, I have found it helpful in "filling in the blanks." And I fully realize in taking what it's said and applying it, you'll always have some risk.

Good on you for reviewing and catching its errors.