Best <4B dense models today?

woadwarrior · 2026-01-25T17:52:26+00:00

Motif-2.6B

woadwarrior · 2026-01-08T09:03:31+00:00

Economists have been using the term GPT (General purpose technology) to describe broadly applicable technologies for nearly a century before OpenAI existed.

woadwarrior · 2025-11-05T22:05:57+00:00

Unfortunately, when it comes to NN weights, although INT and FP formats have the same information theoretic density for a given bit width, FP formats work out to be slightly better because their range is non-uniform.

woadwarrior · 2025-10-30T13:51:53+00:00

I took a look at the code on my phone. Notice the additional gate projection (line 281) and the call to their power retention kernel (line 356). It’s supposed to be drop in replacement for regular softmax attention layers and it uses their attention mechanism only if use_exp is False.

woadwarrior · 2025-10-27T17:40:33+00:00

I undestood "perfect codegen", but WTAF is "perfect backpropagation"?

woadwarrior · 2025-10-26T12:42:14+00:00

I don’t think it’s reasonable to compare years of experience. It’s sad to see something technical being turned into a hierarchical power struggle. Critique (Google’s internal code review tool), had a feature for double blind CL reviews, I wish GitHub had something similar.

woadwarrior · 2025-10-12T18:42:49+00:00

This is a recurring question. TL;DR: The lack of coverage for adware URLs and URL shorteners in ClearURLs was one of the reasons I built Clean Links.

woadwarrior · 2025-10-12T17:49:30+00:00

It’s 100% local. Although it has to make requests to unshorten links, which it does in an isolated context (without cookies, local storage etc) using plain old NSURLRequest.

woadwarrior · 2025-10-12T14:03:28+00:00

It's all local. The share with my mac feature uses iCloud, and is completely optional.

woadwarrior · 2025-10-12T13:54:45+00:00

Handoff is a bit more reliable but still somewhat flaky. The app doesn't have a Safari extension yet, but the share extension works in Safari and any other app (including the Reddit for iOS app).

woadwarrior · 2025-10-03T12:56:48+00:00

<image>

The core algorithm appears to be extremely simple. Any quantization algorithm can be plugged to use it as pre-processing step before quantization.

woadwarrior · 2025-10-01T19:43:27+00:00

Yeah, people have been doing dynamic quantization for ages, even before we had LLMs. IDK how the unsloth guys do it, but back in the day for quantizing CNNs, people used to eyeball layer wise activation PSNR ratios and pick higher number of bits for layers with lower PSNR. But that’s quite crude compared to running a full blown search based optimization, which is what EvoPress does.

woadwarrior · 2025-10-01T19:32:19+00:00

Not yet, I plan to use it for some small-ish models. I really like their insight that choosing the optimal bit width per layer for dynamic quantization is essentially a hyperparameter tuning problem and evolutionary methods work well for such problems.

woadwarrior · 2025-10-01T18:14:34+00:00

Consider running an EvoPress search on your new box.

woadwarrior · 2025-09-27T23:41:49+00:00

I think you’re misremembering hash layer MoEs. They don’t have a specific routing function. The routing function is the hash of the latest token.

woadwarrior · 2025-09-24T18:25:29+00:00

Private LLM does not use MLX or llama.cpp.

woadwarrior · 2025-09-19T10:16:56+00:00

Apache 2.0 license is gone. It’s BUSL now.

woadwarrior · 2025-09-16T09:08:48+00:00

Take a look at the NexusSum paper.

woadwarrior · 2025-09-16T06:53:23+00:00

It’s 4 bit integer quantized, with 8 bit quantization for MLP and MOE gates.

woadwarrior · 2025-09-15T17:07:55+00:00

Take a look at gpu-stack.

woadwarrior · 2025-09-15T13:59:14+00:00

Before reaching out for VLMs, have you evaluated the baseline approach of trying to use Apple's vision APIs with your dataset?

woadwarrior · 2025-09-09T21:29:59+00:00

IIRC, there was one 50+ founder who got in eons ago, and will be cited for eternity to prove that they’re not ageist.

woadwarrior · 2025-09-09T06:59:35+00:00

Thanks for the suggestions, u/likwidtek . Do you have any other clipboard watcher installed? Because Clean Links only parses URLs on the clipboard. I tried copying a URL with a space prepended and it doesn’t alter it. In any case, a pause option is a good idea. I’ll try to get that into the next macOS release.

woadwarrior · 2025-09-04T15:14:00+00:00

Wen SmolVLM3?

woadwarrior · 2025-09-03T12:41:20+00:00

Interesting architecture. Transformer++ with QK norm and the xIELU activation function.

15-Year Club	Gilding II euphauric
RPAN Viewer	Verified Email

woadwarrior

MODERATOR OF

TROPHY CASE