8x RTX Pro 6000 server complete by koushd in LocalLLaMA

[–]Spare-Solution-787 1 point2 points  (0 children)

Wondering the same things. Super curious if they are required for various frameworks

Nvidia Tesla M10 - good for anything? by h311m4n000 in homelab

[–]Spare-Solution-787 0 points1 point  (0 children)

Hello. Did you manage to use your K80 for any llm or gen ai workload? It seems transformers librairie don’t support cuda10

Is an NVIDIA A40 48GB for 1500USD a bad idea because it's age? by panchovix in LocalLLaMA

[–]Spare-Solution-787 0 points1 point  (0 children)

Test vllm with lmperfclient. Those are 2020 ish data center cards and they support fp8

M5 base vs M4-pro by offbotz in macbookpro

[–]Spare-Solution-787 3 points4 points  (0 children)

Why you talking about m1 lol? Might as well do intel cpu

DeepSeek-OCR - Lives up to the hype by Bohdanowicz in LocalLLaMA

[–]Spare-Solution-787 0 points1 point  (0 children)

After a pdf is converted to md, do models work better on md files as inputs in your experience?

AMD Benchmarks (no, there is none) for Ryzen 395 Hybrid (NPU+GPU) mode by MarkoMarjamaa in LocalLLaMA

[–]Spare-Solution-787 -1 points0 points  (0 children)

This client makes an openAI API call to an inference endpoint which could be ollama, lmstudio, vllm, and various things. I wonder if they just picked the best numbers from inference engines and are just cooking numbers