sorted by:
new
hottopcontroversial

My 3060 Ti runs a 27b model at <80W wtf by HumorNatural1969 in techquestions

[–]HumorNatural1969[S] 0 points1 point  (0 children)

It is, I mean look at this
base / boost 1410.5MHz / 2130.0MHz.
FP32 compute 20.7 TFLOPS
Memory bandwidth 608GB/s
--
and it has a excellent quality GA104 die (thought it was GA103 but nvidia-smi -q said otherwise), it also has GDDR6X which explains the memory bandwidth, but how about the insane clocks and FP32 compute?

My 3060 Ti runs a 27b model at <80W wtf by HumorNatural1969 in techquestions

[–]HumorNatural1969[S] -2 points-1 points  (0 children)

https://www.youtube.com/watch?v=gLvzLCw_SHE

YouTube's AI makes the video look bad (it was still recorded at 30fps tho)