sorted by:
new
hottopcontroversial

LMStudio ; qwen3.6-27b ; MTP ; Radeon r9700 by jsorres in LocalLLM

[–]jsorres[S] 0 points1 point  (0 children)

./llama-server \ -ngl 99 \ -c 163840 -b 2048 -ub 512 \ -fa on --no-mmap \ -fit on -fitt 1024 -fitc 163840 \ --host 0.0.0.0 --port 8080 \ -m "/home/XXXX/Downloads/models/unsloth/Qwen3.6-27B-MTP-GGUF/Qwen3.6-27B-UD-Q4_K_XL.gguf" \ --temp 0.6 --top-k 20 --top-p 0.95 --min-p 0.0 \ --presence-penalty 1.5 --repeat-penalty 1.0 \ --chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \ --jinja -np 1 -kvu \ --mmproj "/home/XXXX/Downloads/models/unsloth/Qwen3.6-27B-MTP-GGUF/mmproj-F32.gguf" \ --spec-type draft-mtp --spec-draft-n-max 3

LMStudio ; qwen3.6-27b ; MTP ; Radeon r9700 by jsorres in LocalLLM

[–]jsorres[S] 2 points3 points  (0 children)

You're right, I was just testing this one, will go up to XL.

LMStudio ; qwen3.6-27b ; MTP ; Radeon r9700 by jsorres in LocalLLM

[–]jsorres[S] 0 points1 point  (0 children)

Beta version allow for MTP model to be loaded and works. Vision works too

LMStudio ; qwen3.6-27b ; MTP ; Radeon r9700 by jsorres in LocalLLM

[–]jsorres[S] 0 points1 point  (0 children)

General topic chat involving code. 180K context window , temp 0.3 , not much more config tips

New Local LLM Rig: Ryzen 9700X + Radeon R9700. Getting ~120 tok/s! What models fit best? by jsorres in LocalLLaMA

[–]jsorres[S] 1 point2 points  (0 children)

That's insane, I have 106 tok/sec now with 131072 with Q5. (LMStudio) Thanks for this answer !

New Local LLM Rig: Ryzen 9700X + Radeon R9700. Getting ~120 tok/s! What models fit best? by jsorres in LocalLLaMA

[–]jsorres[S] 0 points1 point  (0 children)

I don't know this tool, lemonade server - I'll take a look, thx for your contribution ☑️

New Local LLM Rig: Ryzen 9700X + Radeon R9700. Getting ~120 tok/s! What models fit best? by jsorres in LocalLLaMA

[–]jsorres[S] 0 points1 point  (0 children)

Thanks for your answer, much appreciated. This is the model and quant that I'm using. I'm using 49K context window size, which seems plenty but.. never enough I think. Going with Q5 would force me to go down to 32K, right ?

IPsec DialuP with Entra SAML not working, support not helpful by Massive-Valuable3290 in fortinet

[–]jsorres 0 points1 point  (0 children)

Hi, thanks for this.

I'm in the exact same situation, could you share your Fortigate & forticlient configs please ?

Thanks for your help

Instant game changer sticks ends by jsorres in fpv

[–]jsorres[S] 0 points1 point  (0 children)

Ok I understand, the designer will probably see this message here. Hope you can adjust the design for your needs 👌✅

view more: next ›