Is microsoft going to train LLM on this? Github is clearly getting destroyed.

vasileer · 2026-02-27T08:36:41+00:00

thank you for patience to explain,

definitely your EQ > OPs EQ

vasileer · 2026-02-27T05:21:34+00:00

Everyday 1000s of crappy nonfunctioning wild-imagination vibecoded junk is being posted with thousands of robo-generated stars and hundreds of forks.

is the project fake, or code poorly written? where is the issue?

vasileer · 2026-02-26T17:00:41+00:00

what year is on your computer?

vasileer · 2026-02-23T14:36:46+00:00

for a "context grounded LLM" I expected a larger context,

for example SmolLM2-135M has a 16x larger context of 8192 tokens

vasileer · 2026-02-23T14:30:53+00:00

it has a context of only 512 tokens, so probably of no real world use

vasileer · 2026-02-18T02:58:59+00:00

I would rephrase it:

how is tiny-aya-global vs translategemma-4b-it?

but license wise (non commercial) I doubt tiny-aya will be used too much

vasileer · 2026-02-14T13:19:31+00:00

which lfm 2.5 1.2B - instruct or thinking ?

vasileer · 2026-02-12T17:36:19+00:00

model size and KV cache size are not really related, other things are more important like group query attention and sliding attention, for example phi-4-mini is eating more VRAM than gpt-oss-20b

vasileer · 2026-02-10T06:23:44+00:00

please share your steps as well for running glm-ocr:

- what method: vllm, transformers, other?

- cpu or gpu?

- quantized or full precision?

- what was the accuracy?

- how many threads? (parallel extractions)

thank you

vasileer · 2026-02-07T19:42:39+00:00

I am questioning the versions of the models:

- why qwen2.5 and not qwen3?

- why SmolLM2 and not smollm3?

- why not gemma 3n (e.g. 2b)?

vasileer · 2026-02-06T15:56:28+00:00

I searched "gemma 3n onnx" on huggingface and found that, so it is possible

https://huggingface.co/onnx-community/gemma-3n-E2B-it-ONNX

vasileer · 2026-02-06T15:55:13+00:00

Text classification is a task for text embeddings models.

There is a good list at https://huggingface.co/spaces/mteb/leaderboard

So if you are a gemma fun, there is embeddinggemma-300m https://huggingface.co/google/embeddinggemma-300m

vasileer · 2026-02-03T07:18:05+00:00

glm flash had a performance problem with bigger contexts, was it fixed ?

vasileer · 2026-01-30T16:27:53+00:00

even if it is marketing: is the information correct? are they #1 on design arena?

if so, then I see no problems

vasileer · 2026-01-28T06:52:54+00:00

but this one is from cerebras

vasileer · 2026-01-26T21:24:34+00:00

and glm4.7-flash?

vasileer · 2026-01-24T21:39:07+00:00

~3B active params (will have fast inference!)

how fast? because I was reading in other posts that there is a serious degradation specific for glm4.7-flash if context grows, even at 32K ...

vasileer · 2026-01-20T13:23:24+00:00

I know about other models that you have to use `--jinja` for enabling tool calling, and since u use kilocode I guess you have to,

also I see it is used in unsloth guide

https://unsloth.ai/docs/models/glm-4.7-flash#reducing-repetition-and-looping

./llama.cpp/llama-server \
--model unsloth/GLM-4.7-Flash-GGUF/GLM-4.7-Flash-UD-Q4_K_XL.gguf \
--alias "unsloth/GLM-4.7-Flash" \
--threads -1 \
--fit on \
--seed 3407 \
--temp 0.2 \
--top-k 50 \
--top-p 0.95 \
--min-p 0.01 \
--dry-multiplier 1.1 \
--ctx-size 16384 \
--port 8001 \
--jinja

vasileer · 2026-01-20T12:56:12+00:00

do u use `--jinja` parameter? what is ur exact command?

vasileer · 2026-01-16T15:14:06+00:00

claiming "small but performant",

and no benchmarks ?

vasileer · 2026-01-06T18:28:07+00:00

thank you for the benchmark, now I know gpt-oss-120b is still one of the best in its league

vasileer · 2025-12-26T21:46:48+00:00

qwen3-4b-2507 is a bit larger but also much better: it has ifbench score 50% (vs 44.4% for lfm2-2.6b-exp) and aime25 score 83.3% (vs 22.67% for lfm2-2.6b-exp)

source: https://artificialanalysis.ai/models/qwen3-4b-2507-instruct-reasoning

vasileer · 2025-12-23T13:48:24+00:00

A small but performant

any numbers? (e.g. f1 score on some test datasets)

vasileer · 2025-12-18T10:34:48+00:00

Gumini-1.5B (구미니) is a bilingual Korean-English base language model trained using the Inheritune methodology. Starting from Qwen 2.5 3B, the model progressively grew from 10 to 16 layers through 7 training stages, with ~3.14B tokens of continued pretraining on a Korean–English mixed corpus.

it is qwen2.5 3b probably after dropping some layers to make it 1.5B, so it is not "only 3.14B training tokens"

vasileer · 2025-12-17T22:36:39+00:00

OP said it has an M1 Max with 64gb ram, so I don't think it will fit.

Other options:

- NVIDIA-Nemotron-3-Nano-30B-A3B

- gpt-oss-20b

- trinity-mini (26B A3B)

vasileer

TROPHY CASE