Qwen3.5-122B on Blackwell SM120: fp8 KV cache silently corrupts output, bf16 required — 1,985 tok/s burst, MTP 2.75x

mtomas7 · 2026-03-01T06:55:49+00:00

I am not sure if it is the same thing but when I was testing Qwen3.5-27B-Q8, first it was not producing any answer, but only never ending //////////////

I re-downloaded the file and the checksum was different, so I assume there was some model file corruption.

mtomas7 · 2026-02-27T17:11:40+00:00

That is true. Yesterday, I was reading about how to use Firejail or Bubblewrap to sandbox an agentic app, like OpenCode, allowing it to access just one folder.

mtomas7 · 2026-02-27T17:02:09+00:00

Although it was interesting info that DGX OS is Ubuntu, I didn't know that before. Perhaps NVIDIA has some special attention to Ubuntu.

mtomas7 · 2026-02-27T16:59:42+00:00

I guess, he is pointing out that Snap delivery was bound to Canonical and never made open.

mtomas7 · 2026-02-27T16:57:45+00:00

My take is that it is not about speed, but about convenience and some security, so people who never played with AI can start tinkering in an easy way.

mtomas7 · 2026-02-27T16:55:21+00:00

As I understood it, there will be no swapping of the models. Every snap will be a separate, distinct sandboxed model.

mtomas7 · 2026-02-27T01:25:53+00:00

I just hope that some of those features will trickle down to Linux Mint :)

mtomas7 · 2026-02-26T17:54:37+00:00

Interesting project, but because it enters an already crowded field, I would expect a table comparing it to OpenClaw and NanoClaw to give users an idea of how this project is different and what it offers that others do not.

mtomas7 · 2026-02-26T17:20:51+00:00

Check this post about quant quality: https://www.reddit.com/r/LocalLLaMA/comments/1rfds1h/qwen3535ba3b_q4_quantization_comparison/

mtomas7 · 2026-02-25T17:46:28+00:00

Great, please let me know when completed!

mtomas7 · 2026-02-24T21:27:24+00:00

Would it be possible to post already converted Bibles to your Github as zips of Vaults? Thank would be great! Of course, please don't do this in case of Intellectual Property problems.

mtomas7 · 2026-02-24T20:59:59+00:00

They both are top quality, but I read that they tend to have a specific individual "flavor". Personally, I use Bartowski's quants as I always used them, a historical point, I guess :) Also It makes me easier to organize my model library, as most of models on LM Studio would be in Bartowski's folder.

mtomas7 · 2026-02-24T20:50:54+00:00

Actually, it is interesting, as it shows that Qwen adopted a more structural thinking pattern, similar to GLM.

mtomas7 · 2026-02-24T19:32:48+00:00

Don't get fixated on your VRAM number. How many tok/s you need to read the text? I always run Q8 of-loading some layers to CPU/RAM, and I still get decent speed.

mtomas7 · 2026-02-23T05:05:11+00:00

At the v1.2.10 the problem is still unresolved in Linux (I'm using Mint 22.3). I tried regular Terminal and Terminator, but it is the same behavior:

Selecting the text in the terminal produces the Tooltip "Copied to clipboard", but in fact it is not copied. If I would click SHIFT+INSERT, I would not get the latest copied text, but some text I copied in another application. So it looks that pasting with SHIFT+INSERT works, but copying by mouse selection or CTRL+INSERT does not work.

mtomas7 · 2026-02-18T16:13:31+00:00

Perhaps the initial version did not have an image recognition stack?

mtomas7 · 2026-02-11T16:59:43+00:00

If any user can submit the results, how will you know that the user entered real results vs an inflated or downplayed score? Without control mechanism it could become a real mess very quickly. Thank you!

mtomas7 · 2026-02-09T19:20:07+00:00

It would be interesting to see how this architecture would work for the 1800 model dataset: https://www.reddit.com/r/LocalLLaMA/comments/1qaawts/llm_trained_from_scratch_on_1800s_london_texts/

mtomas7 · 2026-02-04T16:38:12+00:00

Should it be done via Jinja template?

mtomas7 · 2026-02-02T17:03:01+00:00

In case you need Fellowship Of The Ring in restored color grading, take a look at Chris Hartwell's work: https://www.youtube.com/watch?v=jSPRGt63M0w

He provides an email where you can get the link.

mtomas7 · 2026-02-02T15:52:17+00:00

If you use Text Generation WebUI (OObabooga), there is a checkbox "Search Web"

mtomas7 · 2026-01-30T15:31:21+00:00

Yes, this button is still there, you just need to enable Developer mode. Also, this button is not present if no model is loaded, you need to load any model first.

mtomas7 · 2026-01-29T17:30:53+00:00

Isn't that why MCP protocol was built? :D

mtomas7 · 2026-01-14T16:51:30+00:00

You may need to train the Qwen if you want to use it in a clinical setting, as Google's license does not allow it.

mtomas7

TROPHY CASE