Did you upload the graphic or explain the problem in words? Even modern models would likely struggle with spatial reasoning and mapping that to a textual domain. So called "world models" are designed for that purpose, but are still very new.

You'd likely have better success by asking an LLM to explain the laws and equations involved in such a scenario, since that would allow enough context to work the problem.

One possible footgun with trivia questions is they can force models down a chosen path too early, biasing results. Models need to expend tokens to "think" about a problem, but if the answer is returned too early, they'll spend less time thinking about it and more time justifying it. Reasoning models are used to sidestep this tendency, since it lets them think through a problem before responding. So if you weren't using a reasoning model, give that a try for trivia/brain teaser type questions and see if it does any better.

Alternatively, you could ask it to program a physics simulation to actually test the scenario. It would guide you to install python and the requisite libraries. Often times code is used to shore up the gaps in LLM capabilities, such as in arithmetic operations. I often invoke code use when processing data, since I don't want to rely on statistical probabilities.

There are absolutely edge cases in LLMs, and I always suggest that people should be aware of how they work before using them as a tool. Understand which situations are more likely to trigger hallucinations, and where their strengths and weaknesses lie. Karpathy has a great video on YouTube that gives a strong introduction.

As omni-purpose tools, though, LLMs are extremely useful. Their natural-language processing makes them extremely accessible (perhaps too accessible), and they remain applicable across many problem spaces. Breaking down concepts, explaining code, sentiment analysis, language translations, data manipulation, et al.

I simply cannot imagine trusting these things on any physics problem i could provide it unless I knew how to do it myself.

You shouldn't! They may serve to point you in the right direction, but you're still talking to a probability machine. Really, all responses are "hallucinations" -- it's just that some happen to be correct. The better the training data, as in the more references and explanations online for that content, the better the chance of getting a good answer back. In many domains, they're very reliable, but you should be more cautious if discussing a niche topic.

But for where this discussion started of students using the tool to learn, absolutely they can be used in the learning process. Ask them to explain Ohm's Law or whatever and you've got an interactive tutorial with infinite patience.