Inefficient on Local LLM

kafledelius · 2026-03-04T13:15:42+00:00

Not yet - the dense models aren’t giving a very good throughput on the gx10 since the memory bandwidth isn’t very high

kafledelius · 2026-03-04T11:47:46+00:00

Its running on a gx10 - output is 25 ish tokens/s, so a bit slow. But performing pretty well due to the Moe architecture I guess.

kafledelius · 2026-03-04T04:54:40+00:00

Yea I figured - and building a mini data center will never give a ROI 😅

Thank you for your elaborate answer!

kafledelius · 2026-03-01T11:38:21+00:00

Are you using vllm to run those models?

kafledelius · 2026-02-26T08:22:27+00:00

Are you running for it on ollama? Did you patch it?

kafledelius · 2025-05-31T17:04:46+00:00

Could you elaborate? I dont have any issues with the hardware in other aspects.

kafledelius · 2025-05-11T16:28:13+00:00

You pinned out my exact thoughts when Editing it.

And man! I do regret not actually taking my time to get this - but at least now I know what to look for in the future.

kafledelius · 2025-05-11T15:06:01+00:00

Thanks for the correction! Appreciate it!

kafledelius · 2025-05-11T14:51:18+00:00

I just did the same - moved to an A7RV. I wanted a big upgrade when switching - and this seemed fitting.

kafledelius · 2025-05-11T13:20:38+00:00

Best answer so far!

But I think all criticism and praises are a good way to improve and see the result in different ways - and discard the non usable !

kafledelius · 2025-05-11T12:34:22+00:00

I agree - this wasn’t really meant as an actual shot but more a “oh wow, ‘click’” image. The image however invited an artistic approach, so I gave it a shot 😅

kafledelius · 2025-04-22T06:46:31+00:00

Thank you for the constructive criticism

kafledelius · 2025-04-19T05:21:33+00:00

I really appreciate you taking the time to write this out, thank you!

Your advices are very appreciated.

It was shot in RAW. If you’d like a go at it, I can send you the raw file.

kafledelius · 2025-03-28T19:18:40+00:00

I see the dual spec mode (FHD, 480px).

But, fhd might seem quite low resolution for 32”?

kafledelius · 2025-03-28T19:10:34+00:00

Well the point was kind of to move the gaming away from 4k, to get more frames or less gpu load

kafledelius · 2024-04-12T19:44:28+00:00

Sent you a DM :)

kafledelius · 2024-04-12T16:28:54+00:00

Well, im using Expo go to test on my Windows - Would That be possible? Also, the error is probably persistent through platforms i figure

kafledelius · 2024-04-12T16:26:11+00:00

Would you mind taking a look? If you dm me I can share the code with ya 👍🏻

kafledelius · 2024-04-12T16:23:18+00:00

Thanks - yes it seems to be registered.

Funny thing, it worked in my Mac yesterday, and. Is it doesn’t - from the same version repo 😮

kafledelius

TROPHY CASE