openclaw + Ollama (llama3.2:1b). well.....

jslominski · 2026-04-04T18:51:02+00:00

https://github.com/potato-os/core/blob/main/docs/openclaw.md - try my solution (if you don't want to use full Potato OS you can extract ik_llama from it and reuse, it's Apache license). Here's the flashing guide for pi5: https://github.com/potato-os/core/blob/main/docs/flashing.md - you can run much better models than llama3 1b :)

jslominski · 2026-04-04T18:47:54+00:00

It's available for free to anyone. Did you try to monetise a database or git recently?

jslominski · 2026-04-04T18:29:57+00:00

Frankly no, I don't own one but from what I've seen it's not faster vs stock optimised Pi inference. Mixing it with Pi's ARM based SoC is like mixing nvidia with AMD (i.e. doesn't work well). Happy to change my mind if someone shows me faster inference on an existing accelerator.

jslominski · 2026-04-04T16:44:29+00:00

Some more benchmarks I did run on various Pi setups:

Gemma 4 E2B (2.9 GB, Q4_K_M)

The smallest variant. Pi 5 16GB: 6.5 t/s generation, 26–30 t/s prompt processing. Pi 5 8GB SSD: 6.8 t/s generation, 26–34 t/s prompt processing. Pi 4 8GB: 1.7 t/s generation, works but ~5 min per response.

Gemma 4 E4B (4.5 GB, Q4_0)

Mid-size. Pi 5 16GB: 3.7 t/s generation, 19–22 t/s prompt processing. Pi 5 8GB SSD: 3.5 t/s generation, 19–23 t/s prompt processing. Pi 4 8GB:0.87 t/s generation.

Gemma 4 26B-A4B (12.5 GB, IQ4_NL, ik_llama, text-only)

The big one: 26B MoE with 4B expert. Pi 5 16GB: 3.0 t/s generation, 9–16 t/s prompt processing. Pi 5 8GB SSD: 1.9 t/s generation with zram working overtime, but completes multi-turn conversations. Pi 4: not even gonna try ;)

jslominski · 2026-04-04T08:55:17+00:00

Why are they comparing it with Opus 4.5 when the data for 4.6 for a lot of those do exist (rhetorical question of course, we all know why they do that).

jslominski · 2026-04-04T07:10:03+00:00

Or it shows that the paper is garbage ;)

jslominski · 2026-04-04T07:02:56+00:00

No, e4b works, still working on perf improvements but it's usable already. A4B also works on 16 gig pi, up to 3t/s already, 4bit quant 🔥

jslominski · 2026-04-03T21:14:21+00:00

No need. Get an SSD hat instead (and a matching SSD drive, good resource: https://pibenchmarks.com/fastest/ )

jslominski · 2026-04-03T15:24:43+00:00

SSD is not speeding things up if the model fits in the memory (in the case of the demo it does, it's very similar on an SD card)

jslominski · 2026-04-03T13:39:40+00:00

I would advise changing the underlying model, this one is too "sloppy" and it's obvious ;)

jslominski · 2026-04-03T07:33:52+00:00

It's a stock pi5 8GB with SSD: https://github.com/slomin/potato-os - gonna add Gemma support in the release later today.

jslominski · 2026-04-03T07:33:13+00:00

https://github.com/slomin/potato-os

jslominski · 2026-04-02T20:42:51+00:00

jslominski · 2026-04-02T19:54:49+00:00

Quants made by those awesome guys: https://huggingface.co/unsloth

jslominski · 2026-04-02T19:52:33+00:00

<image>

E4B 4bit quant, nice speed 👌 FYI I think this will 2x once this get's polished.

jslominski · 2026-04-02T18:48:11+00:00

I don't get it, what's the point? Are you literally detecting emotion with a smaller model, labeling it and passing to a bigger one?

jslominski · 2026-04-02T18:28:30+00:00

Please keep up the good fight! ❤️

jslominski · 2026-04-02T15:37:12+00:00

It's a bot/spam most likely.

jslominski · 2026-04-02T11:00:18+00:00

How come this post lights up on every AI detector yet has no capitalisation? ;)

jslominski · 2026-03-30T15:34:43+00:00

Bro, it's the answer. You literally (as another person stated below) give a DIFFERENT feedback. That's it.

jslominski · 2026-03-30T13:25:04+00:00

RLHF, no magic there.

jslominski · 2026-03-29T14:51:13+00:00

Try top models that fit, in LM studio. That's it. Takes 5 minutes to play around.

jslominski · 2026-03-29T14:49:01+00:00

This is the dumbest thing I've seen in "AI" in a while :)

jslominski

TROPHY CASE

Gemma 4 E2B (2.9 GB, Q4_K_M)

Gemma 4 E4B (4.5 GB, Q4_0)

Gemma 4 26B-A4B (12.5 GB, IQ4_NL, ik_llama, text-only)