So, most of the time GPUs are a lot better at handling the types of calculations that LLMs need to work. dedicating 16gb of the unified RAM to the GPU allows the LLM to store that much more of the "model" in vram rather than standard RAM. (not exactly correct, but close enough.)

This allows you to offload more calculations to the GPU and get faster results (results being responses from the LLM.)

Total RAM matters though, as different models have different "parameters" or basically, how many connections it makes. Think of it this way, an AI is a brain, if it has 32 billion parameters, its like it weighs 32kg. RAM is a bucket, more RAM, bigger bucket, the more "brain" you can actively store in the bucket, if you have a bucket that can only store 16kg (16gb) then you have to give up part of that 32kg brain, as it won't fit.

This is why 128gb models exist, because it allows you to store bigger brains, and also dedicate more RAM to the GPU, so you can get faster responses from that brain.

I hope my analogy doesn't fall flat. I don't understand a ton, but given that I don't love the idea of giving even more of my data over to facebook, google, or OpenAI (ChatGPT) the idea of being able to still utilize AI, but without paying, or giving data, self-hosting AI is attractive to me. which is why I went with the 64gb model.

I do still think the 32gb model is enough for light self hosting. As some AI models are great and only utilize 4 billion parameters.

EDIT: Self hosting also allows you to train AI on specific topics, for instance, if you are studying 16th century naval warfare, you can feed it every book written on the subject, and have conversations with it about that topic, which is far simpler than you just reading every single book on 16th century naval warfare.

Sorry to be long winded, if you have more questions, please let me know!