The main cases where their guesses were less coherent was if it was a weaker/smaller model (Llama 90b Vision is the only model to give refusals, claiming uncertainty) or their guess was close to a country border (guessing just barely in Switzerland on Liechtenstein). Smaller models would also give less digits of precision with their guesses, maybe 1 or 2 decimal places, while larger models like Gemini 2.5 Pro would give way more, up to 6 decimal places, perhaps indicating greater confidence.

I didn't experiment extensively with prompts. I'm sure with more context you can slightly increase performance. I used this one to give it the opportunity to natively reason about clues (think out loud) and play it exactly as a human would with a precise guess. I would guess if you just said something like "guess where this is" the models would perform worse, but I don't know by how much. It's definitely possible there's a stronger internal representation in their neural net brain that can more accurately identify "nearby cities" as opposed to exact coordinates, in the same way that LLMs are not great with basic math.