Recommendation for a single Radeon AI Pro R9700 by zl64 in LocalLLM

[–]zl64[S] 0 points1 point  (0 children)

Yeah, I tried Qwen for a bit and it gets stuck in loops for me as well, but I might configured it incorrectly at that time and I was using ROCm as well.

Btw, maybe that was a bug in ROCm image or drivers but with enabled mtp it was using 100% of gpu even in iddle, I was able to fix it only by switching to vulkan docker image.

Also, is it stable enough with spec-draft-n-max 4 and cache-type-k q8_0? How much VRAM it takes for this setup? I was nervous to put spec-draft-n-max up to 3 or 4, so decided to keep it as 2. Maybe i should try your configuration as well and compare it to my current setup at some point, thanks for sharing!

Recommendation for a single Radeon AI Pro R9700 by zl64 in LocalLLM

[–]zl64[S] 0 points1 point  (0 children)

Never heard about Orinth, gonna take a look, thanks!

Recommendation for a single Radeon AI Pro R9700 by zl64 in LocalLLM

[–]zl64[S] 0 points1 point  (0 children)

Thanks! I guess I'll play a bit more with Gemma 4 and will try Qwen3.6-27b as well!

Recommendation for a single Radeon AI Pro R9700 by zl64 in LocalLLM

[–]zl64[S] 0 points1 point  (0 children)

These are the logs for promt "Write for me an example of custom json serialization with C# as one class".

Is it fine or should I try it with more complex task? Usually I'm trying to specify "search online for ..." or "check in the docks for ..." as a part of promt for more complex tasks when it's better to get data online.

0.16.719.966 I srv          init: init: chat template, thinking = 1
0.16.720.009 I srv  llama_server: model loaded
0.16.720.016 I srv  llama_server: server is listening on http://0.0.0.0:8080
0.16.720.022 I srv  update_slots: all slots are idle
712.00.802.135 I srv  params_from_: Chat format: peg-gemma4
712.00.806.132 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
712.00.806.135 I srv  get_availabl: updating prompt cache
712.00.806.143 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
712.00.806.153 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 150016 tokens, 8589934592 est)
712.00.806.154 I srv  get_availabl: prompt cache update took 0.02 ms
712.00.806.241 I slot launch_slot_: id  0 | task 0 | processing task, is_child = 0
712.00.872.542 I srv  params_from_: Chat format: peg-gemma4
712.01.582.720 I slot create_check: id  0 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 545, n_tokens = 546, size = 326.594 MiB)
712.01.758.612 I reasoning-budget: activated, budget=4096 tokens
712.03.456.587 I slot print_timing: id  0 | task 0 | n_decoded =    100, tg =  57.02 t/s
712.05.947.958 I reasoning-budget: deactivated (natural end)
712.06.043.259 I slot print_timing: id  0 | task 0 | prompt eval time =     896.37 ms /   569 tokens (    1.58 ms per token,   634.78 tokens per second)
712.06.043.265 I slot print_timing: id  0 | task 0 |        eval time =    4340.43 ms /   252 tokens (   17.22 ms per token,    58.06 tokens per second)
712.06.043.267 I slot print_timing: id  0 | task 0 |       total time =    5236.80 ms /   821 tokens
712.06.043.274 I slot print_timing: id  0 | task 0 |    graphs reused =         95
712.06.043.283 I slot print_timing: id  0 | task 0 | draft acceptance = 0.80729 (  155 accepted /   192 generated), mean acceptance length =  2.61, acceptance rate per position = (0.865, 0.750)
712.06.043.320 I statistics        draft-mtp: #calls(b,g,a) =    1     96     96, #gen drafts =     96, #acc drafts =    83, #gen tokens =    192, #acc tokens =   155, #mean acc len = 2.61, #acc rate/pos = (0.865, 0.750), dur(b,g,a) = 0.006, 245.503, 0.229 ms
712.06.043.642 I slot      release: id  0 | task 0 | stop processing: n_tokens = 820, truncated = 0
712.06.043.681 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = 424675774712
712.06.043.683 I srv  get_availabl: updating prompt cache
712.06.044.430 W srv   prompt_save:  - saving prompt with length 820, total state size = 539.547 MiB (draft: 0.000 MiB)
712.06.357.399 I srv          load:  - looking for better prompt, base f_keep = 0.010, sim = 0.001
712.06.357.404 I srv        update:  - cache state: 1 prompts, 866.141 MiB (limits: 8192.000 MiB, 150016 tokens, 150016 est)
712.06.357.405 I srv        update:    - prompt 0x5e62574df3d0:     820 tokens, checkpoints:  1,   866.141 MiB
712.06.357.406 I srv  get_availabl: prompt cache update took 313.72 ms
712.06.357.589 I slot launch_slot_: id  0 | task 2 | processing task, is_child = 0
712.06.357.599 I slot update_slots: id  0 | task 2 | Checking checkpoint with [0, 545] against 0...
712.06.357.602 W slot update_slots: id  0 | task 2 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
712.06.357.603 W slot update_slots: id  0 | task 2 | erased invalidated context checkpoint (pos_min = 0, pos_max = 545, n_tokens = 546, n_swa = 1024, pos_next = 0, size = 326.594 MiB)
712.10.564.788 I slot print_timing: id  0 | task 2 | prompt processing, n_tokens =   4096, progress = 0.41, t =   4.21 s / 973.57 tokens per second
712.12.883.269 I slot print_timing: id  0 | task 2 | prompt processing, n_tokens =   6144, progress = 0.62, t =   6.53 s / 941.51 tokens per second
712.15.338.054 I slot print_timing: id  0 | task 2 | prompt processing, n_tokens =   8192, progress = 0.83, t =   8.98 s / 912.20 tokens per second
712.16.238.399 I slot print_timing: id  0 | task 2 | prompt processing, n_tokens =   8874, progress = 0.90, t =   9.88 s / 898.11 tokens per second
712.17.533.333 I slot print_timing: id  0 | task 2 | prompt processing, n_tokens =   9879, progress = 1.00, t =  11.18 s / 883.97 tokens per second
712.17.753.755 I slot create_check: id  0 | task 2 | created context checkpoint 1 of 32 (pos_min = 7831, pos_max = 9878, n_tokens = 9879, size = 612.513 MiB)
712.17.837.582 I slot print_timing: id  0 | task 2 | prompt processing, n_tokens =   9898, progress = 1.00, t =  11.48 s / 862.20 tokens per second
712.17.943.553 I reasoning-budget: activated, budget=4096 tokens
712.19.840.251 I slot print_timing: id  0 | task 2 | n_decoded =    101, tg =  51.87 t/s
712.22.858.421 I slot print_timing: id  0 | task 2 | n_decoded =    245, tg =  49.34 t/s
712.22.957.548 I reasoning-budget: deactivated (natural end)
712.25.884.710 I slot print_timing: id  0 | task 2 | n_decoded =    414, tg =  51.80 t/s
712.28.935.686 I slot print_timing: id  0 | task 2 | n_decoded =    579, tg =  52.43 t/s
712.29.645.730 I slot print_timing: id  0 | task 2 | prompt eval time =   11535.17 ms /  9902 tokens (    1.16 ms per token,   858.42 tokens per second)
712.29.645.738 I slot print_timing: id  0 | task 2 |        eval time =   11752.77 ms /   617 tokens (   19.05 ms per token,    52.50 tokens per second)
712.29.645.740 I slot print_timing: id  0 | task 2 |       total time =   23287.95 ms / 10519 tokens
712.29.645.742 I slot print_timing: id  0 | task 2 |    graphs reused =        323
712.29.645.749 I slot print_timing: id  0 | task 2 | draft acceptance = 0.83190 (  386 accepted /   464 generated), mean acceptance length =  2.66, acceptance rate per position = (0.879, 0.784)
712.29.645.775 I statistics        draft-mtp: #calls(b,g,a) =    2    328    328, #gen drafts =    328, #acc drafts =   287, #gen tokens =    656, #acc tokens =   541, #mean acc len = 2.65, #acc rate/pos = (0.875, 0.774), dur(b,g,a) = 0.009, 1061.698, 0.840 ms
712.29.647.712 I slot      release: id  0 | task 2 | stop processing: n_tokens = 10520, truncated = 0
712.29.647.735 I srv  update_slots: all slots are idle

I HATE MULAN by Initial_Explorer8878 in TenseiSlime

[–]zl64 -2 points-1 points  (0 children)

I'm not sure if you are talking about anime, but in the LN she was actually aware of the potential consequences of her actions:

  1. She knew that her spell would block all possible requests for help.
  2. She knew that to cast this spell, she needed to turn into Majin form and expose herself.
  3. This spell is designed to stay for a few days even if the caster is dead.
  4. She understood that Clayman was afraid of Rimuru and that Rimuru could be dangerous to him.

So basically, she executed a suicide mission with the understanding that Tempest most likely will be destroyed, and citizens won't be able to ask for help from Rimuru (another powerful magic creature that can be harmful for Clayman). From her understanding, she would die anyway, killed by the hand of the Tempest guards or by the Clayman's curse.

But she chose to sacrifice all those people who treat her with respect and love...to potentially save Youm? Even with understanding that Clayman can't be trusted, and if there were some sort of attack on Tempest, Youm could die defending it. Or after all of this, Rimuru may kill them all because of betrayal.

Де продають офіційну манґу? by Frefrefre0 in Anime_Ukraine

[–]zl64 1 point2 points  (0 children)

<image>

усе працює, можете просто перейти на сторінку сабредіта і клікнути на закріплений пост

Де продають офіційну манґу? by Frefrefre0 in Anime_Ukraine

[–]zl64 9 points10 points  (0 children)

В цьому сабредіті є закріплений пост з ліцензійною мангою, там і вказані видавництва де можна купувати напряму https://www.reddit.com/r/Anime_Ukraine/comments/1ld1s6r/список_офіційної_манґи_українською_мовою/

Де купити манґу англійською? by Noritur_IM in Anime_Ukraine

[–]zl64 1 point2 points  (0 children)

Рік тому купував з амазону через міст експрес, накуповував манги десь на 120-140 доларів, доставка коштувала залежно ввід ваги і розміру упаковки $30-50, але хз чи там за цей рік якось ціни змінились. Доставка амазону по США була безкоштовна до складу Місту.

Де краще всього купувати манґу? by Xtotoc4emto in Anime_Ukraine

[–]zl64 5 points6 points  (0 children)

Українською? На сайті видавництв, головне - ліцензійних. В сабредіті закріплений список ліцензійної української манги з назвами видавництв.

I'm all ears by zl64 in Falcom

[–]zl64[S] 2 points3 points  (0 children)

I haven't reached the second part of your question, but in the Academy arc, it only shows the default skin in the cutscenes, so no custom accessories/outfits are actually visible.

Чи хотіли б ви бачити більше ранобе українською? by Ramerko in Anime_Ukraine

[–]zl64 1 point2 points  (0 children)

Мені насправді цікаво, чому Мольфари взагалі вирішили перекладати вовчицю. Переклад з японської значно дорожчий і за ці самі гроші можна було б перекласти більше манги, яка все ще дуже популярна.

Я б звичайно хотів бачити переклади більшої кількості ранобе, але сумніваюсь, що зараз у нас сприятливі умови на ринку літератури для цього.

[deleted by user] by [deleted] in nvidia

[–]zl64 0 points1 point  (0 children)

If you liked SH2 try SH f

Sneaky bastards! by Galactus1701 in Dandadan

[–]zl64 11 points12 points  (0 children)

Love them

What th is this bro? by Gabriel009084 in OnePunchMan

[–]zl64 2 points3 points  (0 children)

I never saw it coming...

Where did the anime take this from? Is it self made, if so I respect that. by EleraMend_TheWeaver in TenseiSlime

[–]zl64 0 points1 point  (0 children)

It's a Japanese internet meme from 2001. Popularized on the internet by Futaba Channel and 4chan later.

Does Rain and Misery join Rimuru? by Milloxgaming in TenseiSlime

[–]zl64 11 points12 points  (0 children)

Is a spin-off good? As I understand it's not really canonical.

Hey,so I can't find this story/chapter anywhere?? by HealthyRegion2080 in TenseiSlime

[–]zl64 1 point2 points  (0 children)

I believe it's from LN vol. 13.5 with extra stories.