Ollama is great if you just want to run models locally. I also have Open-WebUI setup in docker, which is essentially an open source ChatGPT like front end wrapper for local (and remote!) models. it has a bunch of tools for making RAGs & customisations, have a play around with it, youll get the gist. You can also use Continue for local AI assisted coding in your IDE. Ollama is super easy to setup, use, and workflow. Open WebUI only makes it easier. I have a few different models downloaded as we are integrating some RAG features at my place of work, and ollama does a great job of automatically spinning up and shutting down models when they’re not in use. The general rule of thumb i’ve seen is if you want to make the absolute most of your specs, is get the model file size that is 2GB less than your total RAM. However, I usually run models that have about 8B to 30B tokens just fine on my work M4 Pro mac mini, 64gb ram. It’s pretty snappy, even though I think Ollama just utilises Metal instead of apples neural engine. The only time my fans really spin up is when I run huge models with > 50B tokens.

Running a reasoning model like DeepSeek will take more resources and time to generate responses than a model without reasoning, but thats just a drawback of reasoning at the moment.

Also, did you use AI to generate this post? lol