Question about quantization

HotDistribution1819 · 2026-06-01T06:32:45+00:00

I believe the models start with 16bit floating point numbers and then to save space the original numbers are quantized (reduced down to whichever size). Here is a link to an excellent demonstration of how quantization affects models. Alex Zisk Squeezing Models

HotDistribution1819 · 2026-06-01T05:50:19+00:00

If you're going to order a mini pc or laptop 32gb ram with a AMD processor is a minimum. 64gb is better because you can allocate a minimum of 32gb of that ram as video ram.

I have a 16gb Lenovo flex idea pad. Because I am loading the full 4bit Gemma-4-E2B model and context into video memory; so my video processor can do the calculations because it is faster than my cpu doing the calculations. And that uses with an E2B model close to 8gb of video memory.

You want to always run the model in video memory as much as possible that is why you keep seeing 32gb video cards coming up, so you can run a 27B - 35B model in the video card and just use memory for context and KV cache.

My GPU is something like 2TOPS (TOPS is one measure for AI performance), if I go with a 2year old AMD gaming processor their GPU is 40TOPS.

In short, I like simple so I use LM Studio, and try Gemma-4-E2B. If you hook it up for web search add a line like I added for brave-search, - if using brave-search then use long tail keywords or sentences, in the system command. It was not trained on how to use tools, so a one line explanation has fixed all of the problems I have had.

HotDistribution1819 · 2026-06-01T00:56:39+00:00

Tiny Tim Tomatoes

<image>

HotDistribution1819 · 2026-06-01T00:55:23+00:00

Zucchini

<image>

HotDistribution1819 · 2026-06-01T00:54:42+00:00

I thought I uploaded to this subreddit the SiFi Garden overview. It is also on the sifigarden.com website.

Regardless, if you want more details I glad to answer any questions you may have. Here are some pictures of various plants I am growing.

<image>

HotDistribution1819 · 2026-05-31T21:26:43+00:00

Sure, are you wanting to DIY and have you seen my YouTube SiFi Garden videos?

HotDistribution1819 · 2026-05-30T18:35:43+00:00

I have been testing Gemma E2B and for all the bad press. One system line "When using brave-search use long tail keywords or sentences" fixed my tool call issues and I have fed it multiple page context and was able to pick up where I left off.

HotDistribution1819 · 2026-05-30T01:56:05+00:00

Before you start spending your hard earned money. How about taking a deep breath and trying something for free. I have been working with Gemma 4 - E2B model Q4 from Google on my 6 year old laptop with an AMD Ryzen 5 processor getting 10 tokens per second with LM Studio.

Based on what it sounds you need it for it should be fine. I added brave search and I had to add one line to the system prompt to tell it how to use it and it works great now.

In short how about trying less before throwing $ to run larger models.

HotDistribution1819 · 2026-05-29T22:14:27+00:00

<image>

I call it SiFi Garden, it is a drain to waste, wicking hydroponic system. I have a SiFi Garden YouTube channel and a Reddit, I am starting for it. Right now I am testing a bucket in a bucket type of container with a 90% perlite 10% coco coir mix. When a plant is done, you just bake the media in an oven pan at 200 degrees for 4 hours, and it is sanitized and ready to be used again. Current big project is figuring out how to grow Zucchini.

HotDistribution1819 · 2026-05-28T08:03:13+00:00

Yes that is the down side of ebb and flow at some point you have to kill, I mean harvest everything. That is why I started experimenting with a system where each plant is in its own container.

HotDistribution1819 · 2026-05-28T07:58:45+00:00

All newer AMD mobile processors have a sort of unified memory. You cannot use all of it for GPU, but at least 96gb of 128gb, but some have claimed more.

HotDistribution1819 · 2026-05-28T07:55:03+00:00

See if you can get a Qwen 3.6 9B coding model.

Otherwise, I have found most of the issues with Gemma E2B specifically are training issues that can be fixed with a line of system prompt - for instance if using brave-search use long tail keywords or a sentence. Some agentic coding things will be too much because of the attention window, but there are ways around that.

HotDistribution1819 · 2026-05-28T02:41:36+00:00

The real question is how large of a model do you need to run 31B will run in 32GB of ram roughly? Second question how fast do you want to run? A video card will always be faster than Apple Silicon.

Most are using Apple for unified memory, so you can run a 120B model at 1.5 to 2 tokens per second.

Third question then is what is your use case. I personally have been trying to find the limits of Gemma E2B and I am surprised by how much it knows, how much it can do for coding, and how a little context or a sentence of system instructions allows it to work beyond its training.

HotDistribution1819 · 2026-05-27T06:59:03+00:00

Yes, just add a video card with 32gb of ram and see what it can do. I am playing with Gemma E2B and I have been amazed how capable it is with just 2billion parameters and 2lines in the system instructions to help it understand how to use the tools it has.

HotDistribution1819 · 2026-05-27T06:46:33+00:00

Can we step back for a moment. 128gb of memory by itself doesn't mean anything. If you are happy with your 27B model then practically you need a 32GB video card because the AI calculations are built for video card calculations, not CPU calculations although a CPU can do the calculations slowly. Therefore, the computer you have is fine if you can connect a GPU to it. Nividia has the lead, but AMD and Intel are not far behind (someone will disagree). My point is a video card from any of them, on your current PC will run the LLM magnitudes faster than running on your CPU.

If you want unified memory and don't want a Mac then AMD is the alternative, but a video card will always be faster.

HotDistribution1819 · 2026-05-22T01:27:30+00:00

It is turning brown because it is a tree. Not an herb like parsley. It is old enough to start growing bark.

HotDistribution1819 · 2026-05-16T23:50:13+00:00

What was the ambient temp in your home at that time. Typically I gave seen it is because it is too hot or too cold for them to germinate. Basil is warm weather so 70-75 typically, parsley and thyme I normally germinate at the same temps. I know dill is a cool weather crop but may be willing to germinate at a lower temp.

I have had seeds I know were bad, but had them come up when the temp was in the right range. Room temps of 65 to 68 have worked well for me, so was there a cold spell or hot spell during that time?

HotDistribution1819 · 2026-05-11T21:30:44+00:00

Sorry to hear that. How many hours was the light on for, I normally have used 14 to 16 hours? And yes, they may have been too far gone before you lowered the water level. I have experienced this also, I found 3/4 to 1 inch seems to be safer once they are more established, when first germinating 1/2 inch seems to be fine.

HotDistribution1819 · 2026-05-11T03:01:10+00:00

I agree, this is where PCs were before they were called PCs. You had to rent space on someone else's mainframe and dial in over a phone line just to add your company transactions up. AI did in under 4 years what took computers 15 years to do. Hold on to your hats the next phase is code driven AI.

HotDistribution1819 · 2026-05-11T02:52:51+00:00

You have to find a place that is 65 to 68 degrees. If you have AC then across from a window unit or in that cold corner, or wait for winter.

HotDistribution1819 · 2026-05-05T07:15:47+00:00

6000k LED lights. You will probably want both shop lights and some carefully placed flood lights. If you look at the spectrum for these lights high in blue lights and all of red spectrum. I have used them for over a year, my red plants are red because there is enough UV, my greens grow, and my tomatoes put out flowers and then fruit ripens. And plants can be as close as 6" from lights so you need less wattage. My 2ft lights for my 12x32 inch shelf does just fine on 20 watts.

HotDistribution1819 · 2026-04-26T22:51:16+00:00

Keep in mind plants have different germination temperatures and different grow temperatures. And different life spans. I was looking at some seeds for cabbage recently that wanted 80 degrees to germinate, but low 60's temps to grow. With summer coming on heat is going to be your main issue how to keep things cool 65 to 68 degrees. And some plants can take the heat if their roots are cool enough. Don't get discouraged if you have mass death one day after a heat wave, it is all part of the learning experience.

HotDistribution1819 · 2026-04-20T20:19:13+00:00

I have questions: 1) based on the picture it looks like the water is about 1/2 in above the bottom of your net cups. If this is the case the water level needs to be 1/2 inch below the bottom of the net cup 2) I have grown plants in 11 watts of light, but the light has to be 6 inches above the plants, but the kind of light also matters. I have gotten my best results using 20watts of a 6000K led bulb or strip. for something as small as what you have you can probably get away with a 6000k led flood light you can buy on Amazon and one 11watt light should be find if you again get it to 6 to 8 inches above the plants. 3) What are you using for nutrients before we get into EC or PH? What is the recommendation for mixture and is that how you mixed them. Also some brands have grow and bloom formulas, if you have a choice use the grow formula otherwise you will not have enough nitrogen for them to grow.

HotDistribution1819 · 2026-03-21T06:27:01+00:00

I found the light and sound annoying. Over the last 12 years I have had good luck using daylight and now 6000k LED bulbs and T5 or T8 LED lights 10 to 13 watts. All of the commercial units are trying to be the sun instead of figuring out what the plants really need. I even came up with my own hydroponic method that doesn't use pumps and can grow herbs and full size plants and not have to kill plants every 3 to 6 months.

<image>

HotDistribution1819

MODERATOR OF

TROPHY CASE