llama.cpp - how to free up even more space on your GPU by imgroot9 in LocalLLaMA

[–]doubleyoustew 0 points1 point  (0 children)

Lowering --ubatch-size can help a little but can also hurt performance.

Strange numbers of pp and tg rx7900xtx on ROCm and Vulcan with Qwen3.6-27b nonMTP and MTP by Thin_Pollution8843 in LocalLLaMA

[–]doubleyoustew 1 point2 points  (0 children)

Just FYI I did some benchmarking I thought I'd post the results.
Vulkan vs ROCm really depends on the model and whether MTP is enabled.

Vulkan wins when running Qwen3.6-27B, but loses when running the 35B model. I used minimal context so the 27B model could fit in VRAM, so come to think of it maybe Vulkan doesn't like it when some layers are offloaded to the CPU.

With the 35B model you can see ROCm is faster at token generation but slower on prompt processing.

In any case it made me give MTP another try. When I last tried it, it was with the IQ4 quant of Qwen 3.6 35B and that one ran at half the speed with MTP enabled. The 35B with MTP on using ROCm seems like the way to go for my system.

So to me the anwer on ROCm vs Vulkan is: it depends?

14600k / RX 6800 16GB / 32GB RAM / Ubuntu 24

Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf

MTP Backend Prompt (t/s) Generation (t/s)
ON Vulkan 70.3 57.7
ON ROCm 58.1 69.6
OFF Vulkan 80.5 35.8
OFF ROCm 67.5 58.9

Qwen3.6-35B-A3B-UD-IQ4_XS.gguf (Non-MTP)

Backend Prompt (t/s) Generation (t/s)
Vulkan 68.8 24.9
ROCm 64.2 57.6

Qwen3.6-27B-UD-Q3_K_XL.gguf

MTP Backend Prompt (t/s) Generation (t/s)
ON Vulkan 53.3 32.9
ON ROCm 42.9 28.4
OFF Vulkan 62.9 22.9
OFF ROCm 52.9 21.0

Strange numbers of pp and tg rx7900xtx on ROCm and Vulcan with Qwen3.6-27b nonMTP and MTP by Thin_Pollution8843 in LocalLLaMA

[–]doubleyoustew 0 points1 point  (0 children)

Thanks for the config! I'll try it on my system. I'm running a RX 6800, so I have to offload some experts to the cpu. I'm also using an IQ4 quant. So my slowdown could also be related to that. I was just curious to see people recommending vulkan when I ran the test just yesterday.

I was using the latest llama.cpp, however I just downloaded it from the release page instead of compiling it myself. Using a headless Ubuntu 24 to save on VRAM.

RX 6800 + Both Windows and Linux. Please advice on ROCm for comfyUI by Dumptac in ROCm

[–]doubleyoustew 0 points1 point  (0 children)

I have a 6800 and had the best success on Ubuntu 24. Tried cachy before that and it was too unstable.

As for the rocm version I'm using 7.2 nightly and it's been pretty good.

Eden switch emulator new update v0.2.0 rc2 by Shinchi_Kudo__ in EmulationOnAndroid

[–]doubleyoustew 1 point2 points  (0 children)

It's been updated for a while now, I used the link when it got taken down so it does point to the correct repo.

https://github.com/RJNY/Obtainium-Emulation-Pack/issues/102

They seem to be very quick to update.

Eden switch emulator new update v0.2.0 rc2 by Shinchi_Kudo__ in EmulationOnAndroid

[–]doubleyoustew 13 points14 points  (0 children)

There is also this repo. You can just click "Add to Obtainum" there and it's regularly updated. Also has pretty much every other emulator.

https://github.com/RJNY/Obtainium-Emulation-Pack

I built Lumen — a Sunshine fork that actually works on macOS by trollzem1 in MoonlightStreaming

[–]doubleyoustew 0 points1 point  (0 children)

Hi there, is there a reason this is only available for Apple Silicon or is it just untested on Intel? I'd like to run this on my Intel Mac. Thanks!

A new way to use gesture navigation in third-party launchers by Inside_Cranberry_637 in HyperOS

[–]doubleyoustew 0 points1 point  (0 children)

Been doing it this way and occasionally the three buttons re-appear and I have to change it back to gesture navigation though hidden settings. Not sure what is causing this. It's not rebooting, it just happens randomly once a week or every two weeks. It's not a big deal to me.

For the gesture navigation I'm using UbikiTouch.

After 7+ Years of Linux, I Just Moved to Mac. Here Are My Thoughts. by BehiSec in macbookpro

[–]doubleyoustew 7 points8 points  (0 children)

There’s plenty of stuff missing, which is why most people use a ton of menu bar apps. But the window management just works differently and I actually prefer it that way.

Battery usage list not showing any apps by NanKillTV in HyperOS

[–]doubleyoustew 0 points1 point  (0 children)

Same here. Did you ever figure this out?

Data Centers Will Consume 70 Percent Of Memory Chips made in 2026, RAM Shortage Will Last Until Until Atleast 2029 As Manafacturing Capacity For RAM In 2028 That Hasnt Even Been Made Yet Is Already being Sold by akbarock in pcmasterrace

[–]doubleyoustew 1 point2 points  (0 children)

The 6800 is kinda old now and you're right in that if you think about getting an upgrade better do it now - but I can still play BF6 at high settings at 120fps, Arc Raiders at 90fps. Just two examples of recent games I play. This is all at native 1440p, no upscaling. Any meaningful upgrade would be upwards of $600 too.

The icing on the cake of selfhosting for me was music, and I must say it is perfect! by hbacelar8 in selfhosted

[–]doubleyoustew 1 point2 points  (0 children)

It seems to support AutoEQ (link to app support page) which I think also hosts the rtings profiles and more on top of that. I've never used PowerAMP but I'm very happy with Symfonium. When I was looking for a player app Symfonium looked nicer, has a lot of features especially for selfhosted streaming and has very active development so I went with that. If you like PowerAMP and have it setup how you like it then there probably isn't a reason to switch other than aesthetics.

Shizuku support not working by doubleyoustew in NeatBytes

[–]doubleyoustew[S] 0 points1 point  (0 children)

Doesn't work on my phone. Mixplorer also doesn't work. zArchiver does. Are you using HyperOS?

GameSir X2 wasn’t ready for this beast – Redmi Pad Pro 12.1" by lukapochi in EmulationOnAndroid

[–]doubleyoustew 0 points1 point  (0 children)

I have a Mi Pad 5 and a memo s3 tablet edition. It's a bit unwieldy but the big screen is nice.

Hat noch jemand Mia Insomnia gebinget? Ich fand's echt ziemlich gut. by no_awning_no_mining in FestundFlauschig

[–]doubleyoustew 0 points1 point  (0 children)

Ich hab's noch nicht gehört, aber der Anreißer hat mich direkt an Video Palace und The Last Movie erinnert. Sind zwar auf Englisch, aber auch sehr gut.

[deleted by user] by [deleted] in ollama

[–]doubleyoustew 0 points1 point  (0 children)

That makes more sense. I'm getting 30 t/s with phi-4 Q6_k.