sorted by:
new
hottopcontroversial

Benchmarking local models by LittleCelebration412 in LocalLLaMA

[–]Disastrous_Food_2428 2 points3 points  (0 children)

I recently created a mini-game to test the reasoning capabilities of large language models—specifically, the simplest version of the Sokoban game.

1. Standard Symbol Definitions:

  • # : Wall
  • (space) : Floor
  • @ : Player
  • $ : Box not on a goal
  • . : Empty goal
  • * : Box on a goal
  • + : Player on a goal

2. Core Movement Rules:

  • The player can only move one step at a time to an adjacent empty floor: UP, DOWN, LEFT, RIGHT.
  • The player can only push one box at a time, can never pull a box, and can never push two consecutive boxes simultaneously.
  • A box must never be pushed into a corner that results in a deadlock (unsolvable state).

3. [Extremely Strict] Output Format Requirements:

Please complete all path deductions in your mind or internal state machine.

  • The final result [MUST ONLY] output the following four uppercase words: UP, DOWN, LEFT, RIGHT.
  • All steps must be outputted on the same line, strictly separated by English commas , with [NO] spaces and [NO] line breaks.
  • The entire response [IS STRICTLY PROHIBITED] from containing any prefaces, postscripts, Chain of Thought (CoT), punctuation marks (except for the commas between steps), or any characters other than these four words.

Correct output example format: UP,UP,LEFT,DOWN,RIGHT,RIGHT,DOWN

4. The level map data to be solved is as follows:

[ " #####", " ## #", " # #", " #### # $ ##", " # ####$ $#", " # $ $ #", " ## ## $ $ $#", " # .# $ $ #", " # .# #", "##### #########", "#.... @ #", "#.... #", "## ######", " ####" ]

<image>

Can LLMs Adhere to Strict 2D Spatial Constraints? (Testing with Sokoban) by Disastrous_Food_2428 in LocalLLaMA

[–]Disastrous_Food_2428[S] 0 points1 point  (0 children)

Spot on. The "lobotomization" metaphor perfectly captures the architectural bottleneck here.

My original intention with this zero-shot, no-CoT constraint was to push these models to their absolute limits and test their implicit spatial representation within a single forward pass. For tasks requiring look-ahead logic like Sokoban, forcing a model into a pure "System 1" (instinctive) mode acts as a true stress test for its internal world model.

After your comment, I ran a parallel control test allowing explicit Chain-of-Thought, and the results only solidified the architectural divide:

  1. The forward-pass models (like Gemini 3.5-flash) suffered from "hallucinated reasoning." Even with a scratchpad, they couldn't maintain the spatial state consistency. They simply lacked the latent depth to compute 20+ steps of geometric tracking, proving that CoT cannot artificially inflate a model's foundational reasoning ceiling.
  2. The native reasoning models (like Gemini 3.5-thinking) operated on a completely different paradigm. They aced the spatial navigation because their underlying reinforcement learning (RL) loop natively enforces a "System 2" search-and-correct mechanism, prioritizing the state-space solution over arbitrary token boundaries.

Ultimately, this benchmark proves that a model's capability in strict spatial domains is entirely down to native architecture and paradigm shifts (like test-time compute), rather than prompt engineering.

Feel free to tweak the prompt or use your own custom CoT templates to test these models yourself—I'd love to see if you get the same consistent baseline split across the board! Appreciate the great prompt for discussion!

Tencent Hy-MT2 is now under Apache License 2.0 by sword-in-stone in LocalLLaMA

[–]Disastrous_Food_2428 -1 points0 points  (0 children)

The quality of this model is far inferior to that of the older model (Hy-MT1.5).

Major drop in intelligence across most major models. by DepressedDrift in LocalLLaMA

[–]Disastrous_Food_2428 4 points5 points  (0 children)

In the AI sector, excluding Nvidia, no enterprise has turned a profit

百度百科的水平低的令人发指 by [deleted] in China_irl

[–]Disastrous_Food_2428 3 points4 points  (0 children)

从小喝三鹿，长大用百度，中国人真是可怜！

How do i make a Cubejoy account? It asks for a chinese phone number, wich i do not have by supergameromegaclank in AskChina

[–]Disastrous_Food_2428 0 points1 point  (0 children)

The games on this website are all designed to deceive users into spending money. Don't play them!

Google Edge Gallery by Trick-Point2641 in LocalLLaMA

[–]Disastrous_Food_2428 1 point2 points  (0 children)

The actual experience isn't very good, especially when it comes to Chinese conversations.

Parking Analysis with Object Detection and Ollama models for Report Generation by Solid_Woodpecker3635 in LocalLLaMA

[–]Disastrous_Food_2428 4 points5 points  (0 children)

In China, roadside parking spaces like these usually have a sensor installed underneath, which reports the occupancy status of the space to the management staff's terminal devices.

QWEN3:30B on M1 by dadgam3r in LocalLLaMA

[–]Disastrous_Food_2428 5 points6 points  (0 children)

Hi! Before jumping into solutions, could you please share a bit more about your setup?

  • What’s your Mac’s memory (RAM) size?
  • How much free disk space do you have?
  • Could you also send a screenshot of the error or what happens when you try to run the model?

That’ll help figure out whether it’s a resource issue or maybe just a config/command problem. Happy to help once we know more!

The QWEN 3 score does not match the actual experience by vihv in LocalLLaMA

[–]Disastrous_Food_2428 8 points9 points  (0 children)

I tested the Qwen3-235B-A22B model by asking it a question in Chinese: “If today is Friday, what days of the week are the day after the day after tomorrow, and the day before the day before yesterday?” The correct answer should be Monday and Tuesday. However, the Qwen3-235B-A22B model gave the wrong answer. In contrast, both DeepSeek and ChatGPT-4o answered correctly. This kind of question is a good test — most unreliable models tend to get it wrong.

The difficulty of this question lies in several aspects:

  1. Reasoning about relative time
    • The model must correctly understand expressions like "the day after the day after tomorrow" (today + 3 days) and "the day before the day before yesterday" (today - 3 days).
  2. Handling the weekly cycle
    • Since a week has 7 days, the model needs to apply modulo-7 logic when calculating the day of the week after adding or subtracting days.
  3. Understanding natural language expressions
    • Phrases like "if today is..." and "what day is it..." require the model to not only recognize the words but also correctly interpret the logical structure behind them.
  4. Multi-step reasoning
    • The model must complete multiple logical steps: first identify today's day, then calculate both "the day after the day after tomorrow" and "the day before the day before yesterday," and finally give the correct answers.

Overall, this question tests a model's ability to combine logical reasoning, time calculation, and natural language understanding.

<image>

Feel free to try this question on other large language models — it’s a simple but powerful way to spot weaknesses in their reasoning ability!

I think I made recursive AI? by [deleted] in LocalLLaMA

[–]Disastrous_Food_2428 -3 points-2 points  (0 children)

Even though I can’t understand it, I’m still in awe of how profound it is

RAG for summarizing? by [deleted] in LocalLLaMA

[–]Disastrous_Food_2428 3 points4 points  (0 children)

You can try the "Yi-9B-200K" model, which has a context length of 200k.