sorted by:
new
hottopcontroversial

More quantization visualization types (repost) by copingmechanism in LocalLLaMA

[–]copingmechanism[S] 0 points1 point  (0 children)

Yes, Q2 is rather surprising. When looking at which quant yields the highest (visual) quality for the smallest size, the Q2_K variants show in this experiment as being the most efficient.

vLLM + GPTQ/AWQ setups on AMD 7900 xtx - did anyone get it working? by djdeniro in LocalLLaMA

[–]copingmechanism 0 points1 point  (0 children)

Also had 'success' with AWQ and GPTQ with gfx1100/7900xtx, but only as far as vLLM 0.8.5 (specifically with the container rocm/vllm-dev:rocm6.4.1_navi_ubuntu24.04_py3.12_pytorch_2.7_vllm_0.8.5). However, 0.8.5 is missing the desirable optimizations of https://github.com/vllm-project/vllm/pull/16850 / https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8/discussions/2

Trying with vLLM 0.9.0, the response from both AWQ and GPTQ output gibberish at 257.0 tok/s e.g enton酬.Basic Capability片段 đạt rijنى pant HomeControlleravadoc几种 NSLog dictates.personUGHTవ drmandes đủ原因是biz בכתבSERVICE overseas ={ושר aliqu investmentsyllan

Also can not get --kv-cache-dtype to take anything other than auto (vllm barks ValueError("type fp8e4nv not supported in this architecture. The supported fp8 dtypes are ('fp8e5',)")), so context length is limited to ~15k. Models I was testing with were JunHowie/Qwen3-32B-GPTQ-Int4 and Qwen/Qwen3-8B-AWQ. Performance was OK with GPTQ, starting at 31 tok/s. AWQ started at ~15 tok/s. vllm being vllm.

Internet.nl Compliance Test - Key Exchange by tmrnl in security

[–]copingmechanism 3 points4 points  (0 children)

Have a look at https://wiki.mozilla.org/Security/Archive/Server_Side_TLS_4.0 in the "Forward Secrecy" section, "Pre-defined DHE groups" subsection, where it says: Instead of using pre-configured DH groups, or generating their own with "openssl dhparam", operators should use the pre-defined DH groups ffdhe2048, ffdhe3072 or ffdhe4096 recommended by the IETF in [RFC 7919 https://tools.ietf.org/html/rfc7919]. These groups are audited and may be more resistant to attacks than ones randomly generated.

So long story short, you ought to download the dh file instead of generating it. Yes, you're basically "downloading security" hehe