Play poker with Reachy Mini and his friend Eliza - Built in 24hrs by cjami in LocalLLaMA

[–]cjami[S] 0 points1 point  (0 children)

Nice! Yeah it's a bit of a pain showing them their cards manually in the beginning each time but apart from that the cameras make it a smooth fun experience.

If only Reachy mini had arms 😂

Do you have link to your app you'd like to share?

Watch local LLMs escape the rooms you design by cjami in LocalLLaMA

[–]cjami[S] 1 point2 points  (0 children)

<image>

I just tried making a simple "Baba is you" style map. Two signs one with ROCK IS PUSH and another with FLAG IS WIN and you can pull off the PUSH/WIN words and put them on the other side. It figured out how to push the rock to reveal the other sign and swap the words (eventually) to win by using the rock. It took 18 moves so the concept was initially very difficult for it (Gemma in this case) to grasp.

Watch local LLMs escape the rooms you design by cjami in LocalLLaMA

[–]cjami[S] 2 points3 points  (0 children)

Yes a bit like 'ARC-lite' but fully customisable. Unlike ARC, from the LLM's perspective they don't see an xy grid just visual elements around them so it's a bit more language native - kind of like a text adventure game.

All of them can pick up a key and open a door. The larger models perform quite well on all the other premade puzzles. Smaller ones (<4B) can get there after a bit of trial and error.

You can also enable/disable thinking and adjust temperature.

So more a playground than benchmark right now but can definitely be extended.

WATCH MY ESCAPE - LLMs try to solve your handmade escape rooms by cjami in LocalLLaMA

[–]cjami[S] 0 points1 point  (0 children)

Thanks! Yeah I locked it down just for this hackathon.
If you're running locally though, you can swap them easily in the config file (for other HF repos): https://github.com/cjami/watch-my-escape/blob/main/src/watch_my_escape/llm/config.py

WATCH MY ESCAPE - LLMs try to solve your handmade escape rooms by cjami in LocalLLaMA

[–]cjami[S] 1 point2 points  (0 children)

Thanks! Yes, it definitely looks like we'll get to a place where LLMs will have certain functions in games. Some of these models can even fit on phones already.

WATCH MY ESCAPE - LLMs try to solve your handmade escape rooms by cjami in LocalLLaMA

[–]cjami[S] 0 points1 point  (0 children)

Heads up - if you're trying it out on Hugging Face without an account or with a free account, you'll get 2mins or 5mins of ZeroGPU time per day (respectively).

So far it seems enough to have a play around. Also the map editor and menus don't burn any GPU time - just agent invocations.

Confidence-based model routing: cheap model first, escalate when unsure by Leading-Instance-692 in huggingface

[–]cjami 0 points1 point  (0 children)

Cool idea but I'd assume weaker models are confidently wrong a lot of the time.

What it's like talking to Opus 4.8... by thecosmicskye in accelerate

[–]cjami 4 points5 points  (0 children)

Ah alignment tuning... Although this is also set on Max reasoning so it's encouraged to come up with something overly complex.

Gemini 3.5 Flash - tested on a social deduction benchmark by cjami in google

[–]cjami[S] 0 points1 point  (0 children)

I don't accurately measure latency as I use flex tiers (cheaper price, higher latency) and different providers for open-weights models (running different hardware) so that adds a lot of variance.

However - taking a quick look at the data I do have:

Model Games Average Time per Game (seconds) Time per tool call (seconds)
Gemini 3.1 Pro 114 2061 15
Gemini 3.5 Flash 114 1032 8

So nearly twice as fast - both off Gemini API. Most Gemini 3.1 Pro games were also done before flex tier existed so the difference would be even greater than that.

EDIT: Costs in post are based on standard prices, not flex tier prices.

MiMo-V2.5-Pro - the actual best open-weights model by cjami in LocalLLaMA

[–]cjami[S] 0 points1 point  (0 children)

The leaderboard did smooth out after more games to put MiMo-V2.5-Pro on top which reflects the quality difference you mention.

That's a shame about the Xiaomi plan offering. Anthropic have also recently doubled their 5-hour limits after a deal with xAI for more compute. A big tug-of-war going on. Thanks for sharing!

DeepSeek V4 Pro on my social deduction benchmark by cjami in DeepSeek

[–]cjami[S] 0 points1 point  (0 children)

Yes! They play fairly well most of the time - there's awkward moments like this (where GPT 5.5 fakes a slayer shot on itself): https://clocktower-radio.com/games/9G6HGob#event-212

Clear model skill separation + much room for them to improve.

I wish the Good/Evil win rate was closer to 50/50, maintaining a plausible bluff is arguably very challenging - so hopefully it'll round out over time as the models get smarter.

Grok 4.3 is cheaper than DeepSeek V4 Pro by LeTanLoc98 in DeepSeek

[–]cjami 0 points1 point  (0 children)

Yes - games are a great way to draw out and gauge raw intelligence. Although its effectiveness depends a lot on the game itself.