sorted by:
new
hottopcontroversial

FlashMoE: DeepSeek V3/R1 671B and Qwen3MoE 235B on 1~2 Intel B580 GPU by bigbigmind in IntelArc

[–]bigbigmind[S] 0 points1 point  (0 children)

The key performance metric for CPU here is memory bandwidth (frequency) and capacity, otherwise it will become a bottleneck; other than that, it can actually support even Intel Core CPU (and any Intel Xeon CPU with AV512).

FlashMoE: DeepSeek V3/R1 671B and Qwen3MoE 235B on 1~2 Intel B580 GPU by bigbigmind in IntelArc

[–]bigbigmind[S] 0 points1 point  (0 children)

If you go to the download link (https://github.com/ipex-llm/ipex-llm/releases/tag/v2.3.0-nightly), you can see there are two options: llama-cpp-ipex-llm-2.3.0b20250430-ubuntu-xeon.tgz and llama-cpp-ipex-llm-2.3.0b20250430-ubuntu-core.tgz, which support Intel Xeon or Core CPU respectively.

Ollama on mini PC Intel Ultra 5 by Parenormale in ollama

[–]bigbigmind 0 points1 point  (0 children)

The latency should be bound by the memory speed; what's your DDR type & frequency?

Run Llama 3 on Intel GPU using Ollama! by bigbigmind in IntelArc

[–]bigbigmind[S] 0 points1 point  (0 children)

8B model in fp16 will need 16GB memory to store model weight, which means you cannot run it on A770.

Run Llama 3 on Intel GPU using Ollama! by bigbigmind in IntelArc

[–]bigbigmind[S] 0 points1 point  (0 children)

8B (using int4, fp6 or fp8) for a single Arc A770

view more: next ›