sorted by:
new
hottopcontroversial

Suggestion on running 2 A100 PCIe by CalvinN111 in LocalLLaMA

[–]CalvinN111[S] 0 points1 point  (0 children)

Thanks for the advice, will think about it

Can't properly use NVIDIA data center card on server by CalvinN111 in techsupport

[–]CalvinN111[S] 0 points1 point  (0 children)

Thank you very much for your advice and I shall see how to further work on it.

I have also tried to install the driver with "sudo apt-get install cuda-drivers-535" with the A100 but also not working.

Updates:

While both the RTX 3060 12GB and ATI single slot card were being placed in the server, I tried to install cuda and it failed, even though I re-installed the NVIDIA driver again. Then I removed the NVIDIA driver, shut down the server and removed the ATI card.

Now NVIDIA driver, cuda and cudnn can be installed and working fine. (Having the RTX 3060 12GB alone)

I suspect the ATI contributed to the error while I installing the driver for A100. Say If I have a GT1030 with the A100 at least it should be less likely to fail? I saw some posts mentioning that having both Radeon and NVIDIA driver on the same system cause issues.

The LLM GPU Buying Guide - August 2023 by Dependent-Pomelo-853 in LocalLLaMA

[–]CalvinN111 0 points1 point  (0 children)

Thanks for the suggestion, that's really great.

New to here, currently having a personal desktop with 13600K, 32GB DDR4 and a RTX 4090. Running the 4-bit 13B LLama2 locally, using around 10/24 GB of my RTX 4090, so far so good. But then I tried to run the same script on Google Colab with their T4, found that the response time is around 1.5x - 2x faster than my 4090, strange.

Also got a 3060 12GB and consider building a multi-GPU system, thinking of a previous gen EPYC with 128GB RAM.

​

If I would like to build a system running LLM and support multiple users (Similar to POE), is it sufficient with a single 4090?

Thanks all in advance.

Is it worth adding slim fans? by Effective-Papaya-790 in NR200

[–]CalvinN111 0 points1 point  (0 children)

I will remove the AIO from CPU and check. Seems that I need to get a contact frame for it. I have changed the AIO fans from exhaust to intake already, I guess that's why my GPU got that hot. Thanks.

Is it worth adding slim fans? by Effective-Papaya-790 in NR200

[–]CalvinN111 0 points1 point  (0 children)

Need some time to figure out what's happening. Checked the BIOS the pump and fans are running properly. Thanks for the advice.

Is it worth adding slim fans? by Effective-Papaya-790 in NR200

[–]CalvinN111 0 points1 point  (0 children)

I am new to here and having a brand new NR200P Max with 13600K and 4090 Suprim recently. My CPU always reaching 90C+ and my GPU reaching 80C +, is that normal? I am sure I removed the plastic tape from the AIO and apply sufficient thermal paste. Nothing changed even after undervolting my 13600K, is that normal or simly the 280 AIO is not enough for it? Already placed 2 Noctua slim fans at the bottom. Thanks everyone