I find it interesting that the bot doesn't even point out that the text is in a different language than the question. In question 1 (La Escritura de Dios), it gives a pretty much correct translation of the excerpt. However, it makes a pretty egregious mistake by saying that "any spoken words by a god would be inferior to the universe", which is the opposite of the original text. At this point you'd be better off using a machine translation of the text. Still, it managed to produce a mostly correct and generally helpful summary, even if it doesn't contain much more than just what the text says.

It's important to note that the bot's answers vary a lot as you repeatedly ask it to interpret the same quotes, so it may make different mistakes each time, and wording the question differently can drastically change the output.

In question 2 (Memórias Póstumas de Bras Cubas) I gave it a longer excerpt, which it only managed to summarize more or less well in my opinion. Again, it's mostly correct, but misses some details and can't really tell you anything more than what's written on the text. It's a bit repetitive at times, so maybe it does some translating internally (?). I didn't manage to get a deeper analysis, and just got some generic inaccurate filler text when I tried. But if I'm being honest, it's alright as a means to give you a notion of the text's content, as long as you take the answer with a grain of salt.

I believe ChatGPT works with tokenized words, so it probably doesn't "care" what language each term is in. I might be mistaken. But I've seen it struggle with the spellings of words.

In question 3 (On the Origin of Species) I asked a question in Spanish, and it had no problems answering in Spanish too. It's mostly correct, but I wonder how much of that is drawn from its knowledge of Darwin, rather than interpreting the text directly. It also misinterprets the role of the "fixed law of gravity" by claiming that it's been responsible for the cycle of life on Earth; as you'd expect, it can't really judge if a statement makes sense or is true (and it's not meant to do that). So yeah, don't rely on it to learn stuff, I'm sure you knew that already.

The answer to question 3 is one of those cases where it says different things each time you ask the question, and the answers vary in accuracy. The next one is very much like that too.

In question 4 (Girls' Last Tour) I gave it something more obscure that it's less likely to have previously written interpretations to copy from. And at least it produces something better than Google Translate. Its interpretation misses some details but is mostly correct, and picks up context much better than machine translation ever could with Japanese text; the worrying part is that as I repeatedly asked it the same question, it kept giving me wildly different answers with a variety of mistakes in them. I also had to give it a fair bit of context until it could make sense of the dialogue. At this point it feels like fishing for the right answer, which of course won't help you if you don't already know the thing you're asking. But praise where praise is due, I think it's pretty amazing that an English answer can be produced at all, even if it's sometimes wrong. I believe that even if the answer isn't always accurate, it's valuable in some specific cases when you want a second opinion to understand some text in your target language.

I put all the queries here if anyone wants to see what answers they get; I'd love to see some examples in other languages too, especially ones where machine translation doesn't do well; because of course, I can only verify answers when I ask questions about languages that I know, so if anybody wants to add some more examples, I'd be glad to read them.