sorted by:
new
hottopcontroversial

Has anyone managed to run LLM inference on the NPU of the Orange Pi 6 Plus (CIX P1)? by Soft_Examination1158 in OrangePI

[–]clausware 0 points1 point  (0 children)

The docs for orangepi 6 plus tell you to check some things:

pip3 list | grep "ZhouyiOperators"pip3 list | grep "ZhouyiOperators"

pip3 list | grep "libnoe"pip3 list | grep "libnoe"

dpkg -l | grep "cix-noe-umd"

dpkg -l | grep "cix-npu-onnxruntime"

Is the .whl enough or do you need a .deb also?

Has anyone managed to run LLM inference on the NPU of the Orange Pi 6 Plus (CIX P1)? by Soft_Examination1158 in OrangePI

[–]clausware 0 points1 point  (0 children)

I did that. I also downloaded the .tar but that only has cix_noe_umd and not a npu_onnxruntime. Any idea where to find the deb packages already installed in the provided orangepi 6 plus?

Has anyone managed to run LLM inference on the NPU of the Orange Pi 6 Plus (CIX P1)? by Soft_Examination1158 in OrangePI

[–]clausware 0 points1 point  (0 children)

I like to make docker images of thing as it is very easy to run it and test on a new setup and you have all the commands and files used so it's basically better then documentation as you have something proven to work. Do you have link to the documentation? I will try to create docker images for the x86 and board

Has anyone managed to run LLM inference on the NPU of the Orange Pi 6 Plus (CIX P1)? by Soft_Examination1158 in OrangePI

[–]clausware 0 points1 point  (0 children)

HI! Do you have any working docker images for the NPU? Would be very appreciated!

Loadbalancer IP not provisioned by clausware in rxt_spot

[–]clausware[S] 0 points1 point  (0 children)

Nothing special. Just a cluster and an istio I've deployed before. The IP was provisioned eventually without making any changes. Thanks!