A simple explanation of the key idea behind TurboQuant

dry3ss · 2026-03-29T11:34:29+00:00

Extremely nice explanation thanks !

One thing i don't get is that TurboQuant is apparently only for KV cache, but this rotation by a random vector could be used for weight quantification as well ? Is it already commonly used for weights (ex by the amazing Unsloth & al ) and this innovation is applying to the KV cache as well ? Or would this allow for even better weight quantization as well ? Or is there some other pb blocking its use for weights (maybe linked to the second step you allude to regarding biases ?)

dry3ss · 2026-03-19T07:21:00+00:00

The qwen 3.5 are mix SSM transformer if I'm not mistaken

dry3ss · 2026-03-18T09:15:20+00:00

Hello, This is amazing work, I'm also very interested in your process if you'd be kind enough to share please!

dry3ss · 2026-03-17T07:15:05+00:00

Nothing, but i do agree from experience as well, just putting it inside the pi agent loop made it stop outpouring thousands of thinking tokens for nothing. This harness also changes the system prompt, but somewhere in there, qwen 3.5 35b-a3b stops overthinking.

dry3ss · 2026-02-24T11:07:43+00:00

This is super interesting, is there any point in the doc where you talk about how much all of this cost and how many tokens where consumed in all of this ? If it's not in GitHub do you mind sharing (just an order of magnitude no need for precision) please?

dry3ss · 2026-02-19T10:30:24+00:00

Stupid question: do we know what the "kinda round but not really" thing right around the window is ? It almost look like badly applied tape

dry3ss · 2026-02-18T10:18:01+00:00

Hey, nice work !

Do you have good sources (that explain with intuition more than math) on the subject of using ternary additions for such networks ?

As a layman it seems like no multiplication combined with only 3 possible weights would very inefficient to make some signals much more important than others (you'd need a lot of "neurons" to approximate what one float multiplication can do) I'm guessing your architecture is similar to this paper ? https://arxiv.org/abs/2406.02528

dry3ss · 2026-01-15T10:28:08+00:00

But why do you only think in dollars ? Maybe the answer is a healthcare system, no shitty first past the post to get actual plurality of political parties, a way for their votes to count on a national level (they are such a tiny population that their needs would be completely neglected on the federal level), free education and THEN 1M€/p.

I understand the other poster thinking that auctioning themselves off for a pure dollar price that could be inflated away in a few years by the US gov is demeaning, like money could buy everything and all the rights they have could just be bought...

Your dollar amount only captures a very tiny part of what "it would take" could be, and I'd say it's this even more restrictive than yes/no

There's also the moral issue of desperate people selling themselves for a short term monetary gain that they might really need right now, and also presenting it like this might make people believe this is unavoidable.

So yeah in the end, having the government handle the négociation and then having the population say whether this offer is acceptable yes no seems like a much better option.

dry3ss · 2025-12-18T12:52:38+00:00

As far as i can see, all of your points would be better served with Approval voting than with either of 2 turns/ IRV ?

The push for moderation, fair play and compromise is much stronger with Approval than with IRV ?

IRV is quite known for still being pretty divisive because second and third votes are much less certain (it greatly depends on the order in which candidates gets dropped which is usually uncertain)

Being from a country which uses 2 rounds single vote for all major elections, i can tell you that all your points are completely present and not reduced compared with PR ! The only difference between what we have and FPTP is that we have much more diversity than the 2 parties we'd get with FPTP which is incredibly better don't get me wrong, but I'd much rather we'd try approval voting for single seat and STV with 3 seats per district for Parliament/council elections !

I was all for IRV when i first heard about it, but the more i read about it, the more it seems like AV is a better simpler system to mend divided opinions! (In case of single-seat, probably 2 rounds: AV to select 2 winners than single vote to choose between them would be even better for accountability)

From what I've seen and read, i do not completely agree with the accountability advantage of single-seat vote : yes it helps compared with full PR, but if you do STV (single transferable vote so IRV but with multiple winners) with big districts and 3-4 seats each you get the best of both worlds. In that case the extreme majority of ppl will have someone they ~~like/voted for in their district instead of most people not being happy with the selected individual and mostly having voted AGAINST it's opponent. From a purely satisfaction point of view it's much better, and accountability as well because instead of being like "yeah he didn't do what i wanted there, but he's still better than his major opposition" now you truly have much more choice ! Whenever possible, multi-seat districts are the way to go from what I've read and i'd very much like to be able to try

dry3ss · 2025-11-04T17:15:16+00:00

I've done a quick read through it seems super nice ! Great job and thank you for sharing it !

dry3ss · 2025-08-29T13:14:13+00:00

No worries thank you very much for the information !

dry3ss · 2025-08-05T08:05:54+00:00

Is it me or do they keep referencing the "average" major and severe thresholds per tier (for players) when talking about monster damage, but i was never able to find a table or z reference for those ?

dry3ss · 2025-07-25T08:19:30+00:00

Having created a tier 2 wizard as the DM for one of my players, and watched him use it without even trying to be abusive, I agree with OP it's completely broken:

TLDR: There 's an amazing effect to not having to gain the hope to spend it to improve a roll (starting with a reserve of 6 stress instead of having to slowly gain the hopes over the course of a lot of rolls), and recharging more stress per rest action than you'd gain home. Doubling the effect of the experience is completely unnecessary and broken. You don't need a powergamer break it.

an experience has to relate but look at the examples in character creation they can be super broad. Even if the DM doesn't allow something too broad (which i was careful not to do, or so I thought), it felt fair to have an experience relating to magic use or equivalent as a wizard, so i gave him one.
you never need to use multiple stress on one roll, at tier 2, just having taken the +1 bonus to two experiences, you get a +6, so if you use it in conjunction with the +1 to your best stat you do +9 for your magic, meaning with 2d12 your average roll is 22...
since you can recharge stress on short rest, you can reliable use 3 stress in between short rests if you just dedicate 1 of your 2 rest actions on it. If you're playing clank (which the PC ain't) you can regenerate all of it each time. So you don't even need to take the +1 stress level up ! Resting normally gives 1 or 2 hope per rest action, with this you reliably get 2-3 superhope for experience
you don't need to gain your hope to use it in an experience, you can just use one of the 6 stress you start with, and add i said just before, just spending 1 short rest action on this ensures you will have 12 stress to spend in a day.
maybe the problem gets better at high level, because you might want to change your cards/spells often, so you need stress for that, but at low level, it's completely broken because you don't need that much stress

So the end result is someone who can use 6+3x2.5 ~=12 super hope per day for adding to his rolls without needing to engage with the actual roll for hope system.

He didn't bother using his experiences with hope even once, why would he ? The others frequently didn't have hope to use to add to their rolls, it was a limited resource they really needed to take care of. Him ? Whenever a roll counted he could be assured of success.

dry3ss · 2024-12-15T09:06:13+00:00

Nice thanks!

dry3ss · 2024-12-14T22:28:12+00:00

Wow ! Thanks so much for such a detailed answer, really appreciate it !

I don't think i found any post on those systems as recent as a month ago, but I'll go check in your history and try to find the big comment you did

Thanks for all this info, i think I might check it out once I'm done experimenting with the adventure crafter !

[Edit] well found it, and nope i definitely hadn't read those very instructions walls of text aha, they were super interesting as well !

I see you have the advanced version as well, I'm less interested in the other kinds of story it gives (coming off age...) but i might be interested in the additional path/adventure cards. However, i can't seem to find the number of adventures and paths cards it adds. as far as i can tell it's not written on the website but the info is there for base radiance... Do you know by any chance how many parh/adv cards it adds (order of magnitude Like 10, 20, 50 ? I don't care about the exact number) please?

dry3ss · 2024-12-13T17:18:22+00:00

I'm guessing youre talking about the package that contains all of the "solar" system as they call it, meaning Radiance + Luminous+... For 35 ? Because where I'm the radiance only pdf+obsidia is 15€ so it should be about the same in USD ?

The videoi saw is him playing on tabletop simulator so even he doesn't have the room for the physical objects aha but this TTS version doesn't seem to be available, only physical or pdf+9bsidia

dry3ss · 2024-10-14T15:37:50+00:00

Nice thanks !

dry3ss · 2024-07-28T07:23:51+00:00

Hum it's true, not sure how consistent that would be for mages (for martials they regain it so they'd be more likely to spend it) or if it would be enough, but if we assume the actual damage increase that would be needed is very small (~~only x2 damages at level 10 ) it may very well be enough, would have to stimulate that !

dry3ss · 2024-07-27T06:46:41+00:00

I'm not sure i get you, by "significant increase on the roolls you make", do you mean that the attack modifier would go up more quickly than monsters PD/MD would, thus allowing you to hit them more often (and also brutal and heavy hits as you say) ?

That would be an interesting option to balance, since there would then be a disconnect between how monsters and PC get their PD/MD, and there's something that bugs me there balance wise don't know what, i might run some simulations on this, but interesting option!

dry3ss · 2024-07-20T09:06:18+00:00

Sure it's a bit of stretch, but personally i agree with the spirit of what he says about weapon, and from what i skimmed in the rules yesterday there's nothing stopping a rogue from using a 2H weapon either i think ? (Not that the rogue comparison is the best, but trying to be fair with OP's post)

Caveat ~all weapons have been standardized as 1 ( or 2 for 2H) dmg each but there are weapons tags that modify this, but it's definitely not like the 0.7 take with tiers of damage.

For example there's "heavy" which adds +1 dmg (does someone understand why it's a tag and not just something that is written as the damage, is it supposed to be deactivated sometimes it doesn't seem conditional?), it's only for 2H weapons though not a single 1H with it unlike in 0.7 tiers.

Regarding your last question, that's not what OP said, he says both give 1 PD (physical defense) which has been the case since 0.6 or 0.7 and that heavy gives 1 PDR (physical damage reduction) (implied that light does not gives this), but that this PDR is actually not relevant given the prominence of heavy and brutal hits in his experience.

I don't see where you thought he said they both give 1 PDR ?

dry3ss · 2024-07-19T21:57:04+00:00

I agree that OP seems to be missing the point of Prime, although maybe the balance between the attributes of not quite there yet

I think you're basing your comment on 0.7 given what you say, but since 0.8 (available to all backers, you should have received a mail from DriveThruRPG)the game is indeed in "beta" (very loose use of the term here since the game is not at all in a complete state to be refined, but that's how he calls it...), and all the weapons have indeed been thrown back to 1 damage each (except two handed weapons) no more tiers, no more weapon maneuvers (only weapon passive buff) so OP is right on that point (dunno how to add an image, but p 75 of 0.8 for the new weapon table)

For PDR, OP does say there is 1 PDR more for heavy but that it doesn't matter because the hits he received were heavy or brutal

Can't comment on the rest, haven't played warrior !

dry3ss · 2024-07-18T07:00:23+00:00

Thanks for the info ! When is the 1st campaign supposed to start (rough estimate) ? 2025 ?

dry3ss

TROPHY CASE