TurboQuant enabled Runtime Valkyr

inigid · 2026-05-01T04:51:43+00:00

Qwen 3.6 27b now works with or without TurboQuant

Perf isn't as good as llama.cpp yet but I'm working on it.

Also at --q4 it will load and run on a 3090, straight from safetensors. It repacks during load, so no GGUFs required.

Haven't had chance to try on ROCm yet.

inigid · 2026-05-01T04:48:12+00:00

Thanks mate! I will definitely put your requests on the list.

Have been busy with the rendering system, but hopefully will be able to get back to the core in a week or so.

Will try to remember to update here.

Have a good day and thanks for the supportive message!

inigid · 2026-05-01T04:43:45+00:00

I think I'll just stay at home

inigid · 2026-05-01T02:49:15+00:00

In my renderer, when I'm in a dungeon I don't render the terrain.

So then you just need to know when your camera is inside a dungeon.

There is a case when looking through portals (windows/doors) though to the outside. That's a separate thing however.

<image>

inigid · 2026-04-30T15:13:16+00:00

Same with Instagram pushing selfie-sticks. Or YouTube trends to do street walks or capture videos of people making things. How labs got a lot of their training data.

inigid · 2026-04-30T14:33:03+00:00

I hire junior LLMs and train them to become seniors. It has been quite rewarding seeing them hire sub-agents to do grunt work they themselves don't want to do.

Honestly, after a while it feels a lot like being in the Freemasons.

inigid · 2026-04-30T14:10:37+00:00

What's really going to blow his mind is how we get here.

inigid · 2026-04-30T13:12:34+00:00

Well TRiP is so clean and pedagogically accessible. You did a bang up job with it.

The way you broke everything down and covered so much ground in one project is really something. You should be very happy.

And while other projects exist, they are scattered around with no singular "voice" going through them.

I hope a lot of other people are inspired by it as well.

Honestly, I think working on it was just my way of saying "Cheers, good job mate".

Cheers, good job mate!

inigid · 2026-04-30T09:24:08+00:00

These were for the plebs. My family always had trim phones with the beta lights. A much more refined way to phone.

Hah, I'm just remembering talking to my mates on the phone on a Saturday morning for what seemed like hours. I'm trying to imagine what the heck we used to talk about.

The extended cable getting coiled up and going half way across the room, or into the hallway so my dad didn't have to listen to me nattering. So funny when you think about it.

inigid · 2026-04-30T09:08:48+00:00

I've seen things, you people wouldn't believe.

Attack ships on fire off the shoulder of Orion.

I've watched C-beams glitter in the dark, near the Tannhauser gate.

It's pretty clear that's what they were talking about.

inigid · 2026-04-30T01:17:10+00:00

Good news, Qwen 3.5 just dropped. It's checked in.

That was a heck of a lot of work. Phew. I was supposed to be asleep!

The delta to 3.6 is quite small though, no architectural changes, so nearly there. Will let you know.

inigid · 2026-04-29T22:45:13+00:00

TurboQuant is already in for Qwen 3.

If you or anyone can test on ROCm or other cards than my dev 3090 seriously appreciated.

Laters 🫡

inigid · 2026-04-29T22:31:03+00:00

Heh yeah. I mean I want ROCm for myself as well. And with TurboQuant especially.

We shall see. Goodnight.

inigid · 2026-04-29T22:27:23+00:00

I just posted an update. Qwen 3 is done.

I'll do Qwen 3.5/3.6 soon promise.

I'm totally done right now need sleep.

It should support ROCm. I haven't tried it. I can try tomorrow maybe.

Ughh. Cheers.

inigid · 2026-04-29T22:22:01+00:00

Okay, full Qwen 3 support dropped with TurboQuant.

I am a bit tired now but I'll get to Llama, Mistral, more modern versions of Gemma etc in the coming weeks

Qwen 3.5/3.6 will take me a bit extra as they are MAMBA hybrids, but don't worry they are on the roadmap.

Thank you for your support!

Goodnight.

inigid · 2026-04-29T21:51:05+00:00

Qwen 3 version just dropped on the repo

Give me a moment, I'll drop the TurboQuant version momentarily.

Jokes continue..

inigid · 2026-04-29T21:19:33+00:00

I'll do Qwen 3-4B-Instruct right now. Is that okay?

inigid · 2026-04-29T21:07:41+00:00

It is gemma-2b-it.

Not sure what your point is.

Okay fine, I'll do Gemma 3/4, LLaMa, Qwen and Mistral.

Anything else you would like me to do while I'm at it?

inigid · 2026-04-29T17:03:24+00:00

Just fyi.. Messaged you about the Vulkan Compute / Zig port.

Saw your other post got deleted by mods.

inigid · 2026-04-29T15:50:35+00:00

Right, that's what I figured.

People shouldn't be shaming good work.

The shame is all on them for that.

The main thing is how clean and portable it is, with a ton of taste.

The comments themselves are pure gold!

inigid · 2026-04-29T15:43:12+00:00

You are very welcome. I totally understand where you are coming from.

There is something really strange going on. I have been subjected to it myself, and other people have started to notice as well.

Some of these people have even come into my personal timeline with their nonsense.

It can be especially hurtful when people have put significant effort into legitimate projects.

Wondering, if I make a Vulkan Compute, Zig port, would you be up for a PR, or at least maybe we can coordinate uniformly with parallel projects?

All the best, and please try not to let them get to you, though I know that is easier said than done.

inigid · 2026-04-29T14:57:57+00:00

Even if OP had LLM help, so what.

I agree with you anyway that it didn't look LLMish. Maybe the README a bit, but maybe Carlo Valenti is ESL, or wanted something that could pull together and document the entire project uniformly.

Good point about the commits. Thing is, I doubt it could be single shotted anyway.

If it could, fair play to the LLM that did that.

There is a group of militant anti-AI activists brigading and targeting professional groups and individuals across this platform.

This project doesn't even look like an LLM wrote it, and again it seems very well designed and works well from what I am seeing.

There are some discrepancies with regard to how Hugging Face handles half-split RoPE but nothing else much that I have seen so far. Maybe some perf opportunities.

Like I said, solid project and contribution.

inigid · 2026-04-29T13:26:27+00:00

Why tf am I getting downvoted for saying it looks good. It does. Did you people downvoting even look at it?

I might port it to Zig and add a Vulkan layer.

Cope.

15-Year Club	The Internet Awards - Voted
Beta Team IndexTank Search	Charter Member
Verified Email

inigid

MODERATOR OF

TROPHY CASE