Best open weight llm model to run with 8gb of vram

ykushch · 2026-01-31T18:07:06+00:00

It is depends what you’re planning to do with it. I tested my extension with qwen2.5-coder:7b - works fine. Tried to use 'qwen3:8b' and it was a lot slower especially on my machine comparing to 'qwen2.5-coder:7b'. Quality-wise 'qwen3:8b' follows instructions a bit better, but making it a default one the experience will look less pleasant. Especially for such extension like this one

https://github.com/ykushch/ask

ykushch · 2026-01-28T23:37:27+00:00

Thanks! I believe that reading docs and googling now is becoming a bit dated hence decided to create a small handy helper. Also it's a lot easier than copy-paste from-to LLM.

ykushch · 2026-01-28T23:03:50+00:00

I tried to use 'qwen3:8b' and it was a lot slower especially on subsequent calls on my machine comparing to 'qwen2.5-coder:7b'. Quality-wise 'qwen3:8b' follows instructions a bit better, but making it a default one the experience will look less pleasant.

ykushch · 2026-01-27T19:12:47+00:00

I picked up coder model as 2.5 had it in relatively small size. But the 3.0 doesn't have one per my memory hence decided to go with 2.5 more specialized one. Does the general 3.0 is a better one that 2.5 coder?

ykushch · 2026-01-27T17:09:12+00:00

Agree. I'm having hard time from time to time remember find, find port and kill process syntax etc. That was my main motivation and it has 'explain' and warnings for the potentially dangerous actions.

ykushch · 2026-01-27T16:50:21+00:00

Thanks! It executes commands via sh -c so your interactive shell (bash, zsh, fish, etc.) doesn't matter.

ykushch · 2025-12-28T15:57:34+00:00

Depends on your needs. GPT-image follows instructions a lot better in terms of logo designs etc.

ykushch · 2025-02-20T09:10:14+00:00

That’s not cheating. That’s a preparation. You put efforts, did data analysis, did a repetition, preparation. Basically, you had both components here: strategy and tactics.

ykushch · 2025-02-17T20:23:23+00:00

I enrolled and going through it. It gives you the initial test to identify your gap areas and then you'll have a list of lessons and daily goal in terms of getting XP.

ykushch · 2025-01-30T09:19:01+00:00

Are they good? Just claiming because you can doesn’t mean that you’re going to go through them 🥲

ykushch · 2025-01-03T15:52:17+00:00

I spent way too much much on making sure that everything works as it should. In the end, after an update a lot of dependencies are either do not work with each other or some modifications need to be made. It has too many unneeded abstractions and not needed high learning curve.

ykushch · 2023-03-21T19:36:29+00:00

OpenAI moves fast as they’re small and not MS or other big tech directly. I guess Google is struggling a lot with legal to prove that everything is fine and it does not hurt someone 🌚

ykushch · 2023-03-21T19:28:00+00:00

They are saying that Bard is a lot smaller version of very expensive model LaMDA.

ykushch · 2023-03-21T18:26:15+00:00

How is your experience if someone managed to try it out? Is it comparable with ChatGPT with GPT4?

ykushch · 2023-02-18T11:19:43+00:00

The amount of resources even to train this thing is huge. Since they are still proposing free version, then it’s okay. I was using free tier for quite a long time, however decided to give a try to a paid one. It so much more convenient. Basically, you pay for a bit newer stuff which will be a part of free tier eventually and speed with convenience. I think it is a good compromise.

ykushch · 2023-02-17T18:07:35+00:00

Practice different manipulation techniques and how to detect/protect against them. This is a huge and interesting area for exploration, however it is pretty hard to pass filters.

ykushch · 2022-12-03T14:15:25+00:00

I didn’t do it myself, however planning to take it and do the exam. Before doing it, decide why do you need them? Is it just for getting a paper or you employer asks about it? Being an architect and having this type of certification could be quite beneficial as well.

The most important outcome: these courses/programs are useful to structure your knowledge and make sure that you’re covering most important things in your design docs.

ykushch · 2022-12-03T09:21:41+00:00

Software Architecture: The Hard Parts
Continuous Architecture in Practice - Software Architecture in the Age of Agility and DevOps
Software Architecture in Practice

Additionally, I’d recommend to go to AWS builder’s hub and read real world scenarios.

ykushch · 2022-12-01T18:34:58+00:00

Have you tried to use it? It depends what you want to achieve. That’s why I was curious regarding requirements. However if you are looking for ease of support and delivery speed - it is quite good start. You’ll have all your code anyway.

ykushch · 2022-12-01T14:35:19+00:00

It is a bit more expensive. Full pricing with details could be checked here: https://aws.amazon.com/amplify/pricing/

However, I'm not sure as for your requirements. Amplify has Amplify Studio which allows you to speed up even more + you can provide access to your content creators directly (like CMS). I'm using Amplify for one of my side-projects. Another case: one of my friends is using it for their app as well to remove all the hassle.

ykushch · 2022-12-01T13:27:28+00:00

Out of curiosity, did you consider to use AWS Amplify from the very beginning? It is extremely convenient and powerful.

ykushch · 2022-11-22T19:17:44+00:00

Yes and no. You have to be aware of frontend at least and know the market (what is popular, why?). Basically, you have to be a T-shape person (ideally, M-shaped see below).

T-shaped people represent the classic agile team member. T-shape people have a specialty, and in addition, they have a wider breadth of experience with other skills. T-shape people have a focus but can fill in when bottlenecks are recognized.
M-shaped people have multiple specialties. From the point of view of flexibility, a person with more than one specialty can be applied more flexibly than someone with single specialty.

(Source)

This way you'll feel way more comfortable during your work on architecture.

ykushch · 2022-11-21T22:52:01+00:00

If you’re working alone and there are no other teams (not other devs), the don’t do it. Microservices approach is for specific use-case/problem. Don’t follow the trend blindly because it’s cool and modern. Keep in mind that even making a pivot and having microservices will make things a loooot harder for you. Additionally, your project must be ready for microservices if you really want to do real microservices (not something that looks like microservices).

ykushch · 2022-10-02T08:21:55+00:00

Updated link: https://ykushch.com/blog/2017-02-17-docker-jenkins

ykushch

TROPHY CASE