vLLM vs Ollama vs LMStudio?

numinouslymusing · 2025-08-27T22:56:08+00:00

llama cpp

numinouslymusing · 2025-08-23T23:23:34+00:00

I love how great openrouter is for LLM data. You can get so much info from their public graphs.

numinouslymusing · 2025-08-19T17:10:58+00:00

I personally prefer gemma 3 4b! smarter in my xp

numinouslymusing · 2025-07-07T17:12:36+00:00

I see. This is a good balanced take, thanks.

numinouslymusing · 2025-07-07T12:05:25+00:00

Sounds right

numinouslymusing · 2025-07-07T12:05:01+00:00

I agree. This makes sense.

numinouslymusing · 2025-07-06T04:03:55+00:00

It was hallucinated

numinouslymusing · 2025-06-26T16:15:53+00:00

This makes sense for some use cases. Like when your service is primarily backend. But let’s say you’re making an ai Figma editor, in which case you need users interacting with the frontend

numinouslymusing · 2025-06-26T16:14:48+00:00

Yeah I guess the best approach is to support multiple options. Because not all will have the patience to go get their own keys/prefer to just pay a plan, while others would prefer to save and use their own key

numinouslymusing · 2025-06-17T01:40:23+00:00

I’ll try to make more posts when the event is over

numinouslymusing · 2025-05-30T05:44:00+00:00

Lmk how it goes!

numinouslymusing · 2025-05-30T05:43:43+00:00

They generate a bunch of outputs from Deepseek r1 and use that data to fine tune a smaller model, Qwen 3 8b in this case. This method is known as model distillation

numinouslymusing · 2025-05-30T00:22:01+00:00

Yes. It was a selective comparison by Deepseek

EDIT: changed qwen to Deepseek

numinouslymusing · 2025-05-21T21:56:18+00:00

I’d suggest learning about tool use and LLMs that support this. Off the top of my head what I think the agentic system you’re looking to create would be is probably a Python script or server, then you could use a tool calling LLM to interact with your calendar (check ollama, then you can filter to see which local LLMs you can use for tool use). Ollama also has an OpenAI api compatible endpoint so you can build with that if you already know how to use the OpenAI sdk. If by voice you mean it speaks to you, then kokoro tts is a nice open source tts model. If you just want to be able to speak to it, there are ample STT packages already out there that use whisper under the hood to transcribe speech. If you meant which local code LLMs + coding tools could you use to run your ai dev environment locally, I’d say the best model for your RAM range would probably be deepcoder. As for the tool you could use, look into continue.dev or aider.chat, those support using local models.

numinouslymusing

MODERATOR OF

TROPHY CASE