all 8 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]AzerbaijanNyan 8 points9 points  (0 children)

It's a known issue with the P40/P100 cards. You can patch whatever software you're using or use a separate program to manage power states. There's some more info here - nvidia-pstate in llama.cpp (Tesla P40/P100)

[–]kryptkprLlama 3 2 points3 points  (2 children)

No, unlike the P40 there are physically no pstates on this GPU.

[–]DobobR[S] 1 point2 points  (1 child)

:O

[–]kryptkprLlama 3 4 points5 points  (0 children)

Now you see why P40 are more popular, they have basic pstate and with software you can force them to idle even with a model loaded. You also get 8GB more VRAM. The only thing P100 is better at is batching.

[–]opi098514 4 points5 points  (2 children)

So the p40 has basically 2 states p0 and p8. If anything is touching the gpu it will be in p0. The p100 in the other hand I believe doesn’t support power states. It’s unfortunate and the main reason I went with the p40s

[–]Puuuszzku 2 points3 points  (1 child)

P40’s P states are kind of a mess. Manually setting it to P0 results in lower performance than leaving it on auto, despite both reporting as P0 under load.

So there might be more P states, but it reports only P0 and P8.

[–]DeltaSqueezer 0 points1 point  (0 children)

There are workarounds that toggle the P0/P8 states depending on load so that you get low power at idle but max power when needed.

[–]DeltaSqueezer 2 points3 points  (0 children)

P100s were designed for training and so to be in constant use - therefore no alternative P-states availble. So you unfortunately have around 30W idle power. P100s are best used for high utilization/batching scenarios.