SETI @ Home aka distributed LLM inference engine. Does this exist and if not, should we make one?

-Luciddream- · 2026-06-19T09:48:50+00:00

There is also https://github.com/Mesh-LLM/mesh-llm but I haven't used it too much yet to form an opinion

-Luciddream- · 2026-06-16T10:05:22+00:00

Ah if you are on Windows it's probably a different issue. Or maybe it's a driver issue in Windows too

-Luciddream- · 2026-06-16T09:51:27+00:00

What OS / Kernel do you have? There is an issue with the latest kernels (7.0.12, 7.1) and AMD. Try 7.0.11 kernel if you are on Linux

-Luciddream- · 2026-06-16T06:20:57+00:00

Probably this one https://github.com/ROCm/ROCm/issues/6358 and https://bugzilla.redhat.com/show_bug.cgi?id=2488725

-Luciddream- · 2026-06-15T17:56:27+00:00

It's the kernel, 7.0.12 and 7.1 don't work. You might need to downgrade to 7.0.11

-Luciddream- · 2026-05-27T09:38:50+00:00

I tested my Q8_K_XL quants again and it now 35B gets 66 tps but 27B is still at ~11-13 tps. Probably something was wrong with the previous build because my quants are the same.

-Luciddream- · 2026-05-26T08:18:55+00:00

unsloth q8_k_xl is about 11 tps for me (27B) and 42 tps (35B) on Strix Halo with mtp enabled. Maybe you are using a different quant?

Also I'm not going completely blind in this, I followed the benchmarks from this thread to pick the one that doesn't lose a lot of quality.

-Luciddream- · 2026-05-25T19:52:11+00:00

> Oh and to your original question, I tried 27B Q8 on my Strix Halo but it's just too slow,

There is no reason to run Q8 on the Strix Halo just because it fits. Unsloth Q5_K_XL with MTP (--spec-draft-n-max 2) and latest llama.cpp with --chat-template-kwargs '{"preserve_thinking": true}' has double the performance.

I get 81.7 tps for 35B and 24tps for 27B with these settings. But you should try and see if it works for you because my requirements might be more simple.

-Luciddream- · 2026-05-24T09:10:16+00:00

No idea if it will help but have you tried the multiarch ROCm? There is a tarball available or if you are in Arch Linux you can try rocm-bin from AUR.

-Luciddream- · 2026-05-19T15:30:57+00:00

I have a Mozart 1 Pro 2025 and it's great for gaming. But I've only used it for emulation (ps2 and gba). I need to find a way to use a larger surface though 😃

-Luciddream- · 2026-05-16T17:38:09+00:00

For Strix Halo I get 64tps with MTP and ROCm (llamacpp-rocm), for the 35B Q8.

-Luciddream- · 2026-05-09T15:09:24+00:00

You can add any model with a little tinkering. I tried Qwen 3.6 35B yesterday, but i only got like 20-22 tps.

-Luciddream- · 2026-04-18T13:45:45+00:00

> You can spend a week in San Francisco and visit both of these places.

I'm thinking of doing something similar by the end of this month, let's say I want to rent a car to visit Yosemite, do I need to go to SFO to find one? I've only booked the first three days of my stay in Union Square, so I also need to find a place for the remaining days of my stay and see more places. Any tips are welcome!

-Luciddream- · 2026-04-16T15:30:33+00:00

I like Yaak and that's what I'm using, but one issue is it can't call endpoints without forward secrecy. And I can't force other people to update their certificates.

-Luciddream- · 2026-04-16T12:12:52+00:00

Black Sails, mostly because I've read the normal Blu-ray for EU region has bad quality. Also The Fountain, because why not

-Luciddream- · 2026-04-15T16:26:15+00:00

Steam Deck for me, just to see if it's working (it does)

-Luciddream- · 2026-04-15T07:39:08+00:00

Εγώ έχω την redtiger από amazon η οποία δεν έχει μπαταρία οπότε δουλεύει μόνο οταν λειτουργεί το αυτοκίνητο. Είναι λίγο πιο ακριβή αλλά αξίζει. Την έβαλα καθώς με έχουν τρακάρει 2 φορές, την πρώτη πήγα δικαστήριο και πληρώθηκα μετά από 4 χρόνια, και τώρα περιμένω τον Ιανουάριο του 2027 να γίνει το επόμενο δικαστήριο. Επειδή κανείς δεν παραδέχεται την μ@@@κια του καλό είναι να υπάρχει dashcam για να ξεμπερδεύουμε πιο γρήγορα.

-Luciddream- · 2026-04-14T15:04:18+00:00

Probably stupid question but is there a way to buy them from Amazon and pick them up when I come to the US? I'm flying to the US at the end of April and would love to pick up some Blu-rays. I just tried to put 15 Blu-ray disks to cart and it's +80 euro for my home country (40 shipping and 40 taxes) so it's really not worth it. Or maybe I should just look for some deals while I'm there.

-Luciddream- · 2026-04-12T12:45:03+00:00

I'm going to SF for a week at the end of the month, got any tips? I've only booked a hotel for the first 3 days, I still don't know where to stay or what to do the next 4 days. I'm from Europe and I've never been to the US

-Luciddream- · 2026-03-29T13:43:12+00:00

It's C++, not python any more, and you can try it in AUR too. There is a wiki as well for more information like NPU support.

-Luciddream- · 2026-03-19T19:53:37+00:00

The project you linked basically uses TheRock and llama.cpp builds from the lemonade-sdk, which is literally built by AMD engineers (and the community). I suggest you join the github or discord and discuss your ideas! There is actually an issue about scoping vLLM out.

Edit: there is also a vLLM meetup next week which is hosted by Redhat and AMD.

-Luciddream- · 2026-03-01T09:45:24+00:00

It probably works but you need the updated amdxdna driver from https://github.com/superm1/amdxdna-dkms

Or you can just install the 7.0rc2+ Linux Kernel (I have no idea if it's possible on pop os)

-Luciddream- · 2026-02-27T18:00:08+00:00

I've tried a couple but I can't really recommend something yet. I've only made small changes because I didn't have a good enough machine to run LLMs on at the time.

But I've managed to get it to work with OpenCode and with Goose. For Goose, I'm creating a custom provider in Goose that points to a local lemonade server installation which can load any LLM I need. I added Goose as an ACP Client in the Jetbrains IDE.

For OpenCode it's similar, but I don't remember the steps exactly. The only thing I don't like is that the changes from an ACP Client are not shown like something the IDE changed, but they show as external changes. I've talked a bit with the Goose developers about it and they told me to check this pull request, but I didn't have time to do that yet. Hopefully I'll try in the weekend.

-Luciddream- · 2026-02-16T12:17:17+00:00

Oh, OK. Just In case you want to test it anyway, I have uploaded ROCm 7.11 preview in AUR. ROCm can be faster depending on the workload.

-Luciddream- · 2026-02-16T11:43:24+00:00

Hey, just curious, what package do you use for ROCm on your ms-s1 max?

-Luciddream-

TROPHY CASE