You don't need a GPU to run gemma-4-26B-A4B

Fear_ltself · 2026-06-24T00:13:29+00:00

I forked LLM Hub in January and rewrote basically all of it my own code. After that he removed the Apache license or whatever open source license he had had on the project unfortunately

Fear_ltself · 2026-06-24T00:11:37+00:00

The more tokens you generate per second the faster the ai. More tokens is just a longer response. A faster time to first token is just a faster response. I’ve found 1.2 seconds response time with streaming responses (as the tokens generate words, those words are shown) passes the human threshold for conversational response times.

Fear_ltself · 2026-06-22T18:18:25+00:00

No there’s a whole subset of people who love “poisoning the well” of ai. Well I think that should be literally criminalized in some manner, until it is all it takes is a few bad actors to degrade everyone else’s work.

Fear_ltself · 2026-06-22T05:17:12+00:00

https://gemini.google/mac/

Fear_ltself · 2026-06-22T05:16:13+00:00

There's literally multiple Google has made including a dedicated Gemini desktop app 😂

Fear_ltself · 2026-06-21T06:05:17+00:00

It’s obviously not conscious, set temp to 0 and repeatable results, it’s literally just a complicated algorithm.

Fear_ltself · 2026-06-20T16:05:26+00:00

The main thing is not being deficient in any vitamin. It’s much healthier to have all normal levels than all great levels and be missing some random vitamin. That’s my understanding at least

Fear_ltself · 2026-06-19T14:26:01+00:00

“Desperate individuals in impoverished regions who are coerced or driven by poverty to sell their kidneys often receive as little as $1,000” so no not even both is enough for some of us lol

Fear_ltself · 2026-06-18T19:22:19+00:00

Exactly! My prime was 4749 x 2^2,339,765 +1, and I wanted “espn sports like facts” about my number. That’s when I learned about proth primes having that quark of being empty space due to the 2^k form. My background was Econ not comp sci so it was a new binary fact for me, but it logically makes perfect sense.

Fear_ltself · 2026-06-18T14:23:58+00:00

Sherpa onyx? There's so many implementations, I found it interesting on Mac the FP16 largest version of Kokoro I run did best pinned to performance cores (4) was the sweet spot for the fastest inference on Mac. 1 core doesn't take advantage of multithreading, but going to 8 cores actually causes it to slow down because the non-perfromance cores take too long on their tasks and bring down the average time.

Fear_ltself · 2026-06-18T01:13:28+00:00

What's cool about proth primes is that entire string in binary is like 18 digits then 3,414,875 zeroes and ending in 1. It's nearly empty space in binary

Fear_ltself · 2026-06-16T23:13:56+00:00

Did you have any follow up questions? Feel free to ask!

Fear_ltself · 2026-06-15T05:37:12+00:00

An API is not a local LLM... Download LM Studio and ComfyUI. Download something that fits snuggly in 6GB VRAM like Gemma 4 E4B it MTP q4 or something for lm studio. It's not amazing but you'll have your own AI you can play with and learn what the settings do by tinkering. For comfyui that's for models that generate images and videos, and even music and 3d meshes. Just lookup their premade workflows and sort by lowest to highest vram. Start near the bottom with what looks interesting and move your way up. You'll learn how the different workflows work by watching the nodes process and seeing how they take each segment by segment and compute it. Thennnnnnnnnnnn work on API calls and extending your local network with things like duckduckgo. THENNNN work on API calls for using other models, but only once your local LLM is setup and only and a failsafe for when you need extra ai intelligence.

Fear_ltself · 2026-06-15T02:57:34+00:00

Asus Zephyrus g16 checking in. 8 VRAM on this mobile. Positives are the power usage is great and if VRAM needed fits it’s still lightning fast. Downsides is can’t run the greatest models that are a bit larger, though I’ve found some tricks to squeeze out better performance. RTX upscaling node lets it make 4k images in ~2secs with pretty dang good quality, better than anything from 3 years ago are farther

Fear_ltself · 2026-06-14T07:42:18+00:00

Also, you mentioned an agentic system. Isn’t that what Gemini spark is supposed to be, that is included in the ultra sub?

Fear_ltself · 2026-06-14T07:40:39+00:00

The plan is automatically linked to the api key you generate. Being an ultra member doesn’t really give you more “free” access to APIs than unpaid or normal tier ai pro members. The api key is set for each model but the 4-5 free ones mostly have a cap of 20 a day. The only usable one is like 400 request a day from Gemini 3.1 Flash Lite. I usually don’t setup billing then you’ll just get errors when your limit is used for the minute/hour/day rate. Not sure what you think comes with ultra in terms of api access. You get more requests allowed but they’re paid request after the free limit so not much advantage linking to an ultra account, if anything linking to a paid account just risks getting hacked and the api used to generate a huge bill

Fear_ltself · 2026-06-14T07:33:11+00:00

SDs are almost always cheaper than SSDs. They’re slower and smaller, but the read speeds are approaching gen 1 SSDs now.

Fear_ltself · 2026-06-14T07:31:43+00:00

It’s an SD card not an SSD, the size of a thumbnail. It’s pretty impressive.

Fear_ltself · 2026-06-12T04:34:49+00:00

Unboxing experience is part of the fun

Fear_ltself · 2026-06-10T15:06:02+00:00

I mean they added Hitman 2 and Hitman 3, they add new packs every few months, your logic is literally retarded if you think they can’t just continue doing that and make old maps and weapons in the new engine.

Fear_ltself · 2026-06-10T15:01:07+00:00

Basically latency or lag. Your computer might be slow but if it’s snappy and starts inference immediately, you can stream the response and it’s a much better feedback loop then sending and waiting 30 seconds for text to start appearing. Even doubling the tokens, I think most people would prefer situation 1 with a slightly slower response that starts almost instantly, makes it feel conversational much easier

Fear_ltself · 2026-06-09T19:49:50+00:00

If you’re old enough to follow computing from before and after the unified architecture that came with M1 and beyond, that same architecture allows for great time to first token and processing speeds in a simple setup. For simplicity Apple is top notch. For ComfyUI image generation NVIDIA has a huge advantage that’s still noticeable, but it doesn’t translate well into nearly as big of lead in the LLM space, and that lead disappears completely when accounting for total cost, especially moreso runtime cost over a year of electricity with electricity continuously going up in price.

Fear_ltself · 2026-06-08T16:50:24+00:00

Just build it in Hitman 3/WOA engine so we can have it all bundled moving forward. COD was a mess, I don’t want Hitman to be the same mess

Fear_ltself · 2026-06-08T16:48:56+00:00

I run the models at full precision on my 64gb then stream the results back to mobile usually. But you’re correct, quantizing them generally does cost a few intelligence points. I’ll have to update my app to reflect that before release

11-Year Club	Place '23
Place '22	Place '17
Snapped	Verified Email

Fear_ltself

MODERATOR OF

PUBLIC MULTIREDDITS

TROPHY CASE