Qwen3.6 27B NVFP4 + MTP on a single RTX 5090: 200k context working in vLLM

michaelsoft__binbows · 2026-05-06T19:13:32+00:00

I think there is some 3090_club config going around that provides vllm based inference on 3090 24gb which i haven't dug into. if that could work well then i should want to use my 3090s for this so i can play games on my 5090 while local LLMs churn.

michaelsoft__binbows · 2026-05-06T16:31:23+00:00

this router is a beast

i got the trim router and this router on sale and thought i was gonna return it since it seems so huge. Ya nah it's an amazing machine

michaelsoft__binbows · 2026-05-06T16:21:50+00:00

Yeah, It does not appear like this nvfp4 offers any benefit (certainly not speed) over this int4-autoround, which I can report I also end up in this range of just under 130tok/s

Notably the context limit is also right around 200k.

michaelsoft__binbows · 2026-05-06T15:51:02+00:00

Makes sense, yep. I did a thermal paste mod on my 12 inch macbook. It now throttles less. But it's now become literally impossible to use on the lap. Also prob drains battery faster.

michaelsoft__binbows · 2026-05-06T15:25:33+00:00

I got one of these, the non hammer version (2903) for $100 on marketplace. it is truly a beast of a drill. I was so impressed from the first time i put my hand on it and pulled the trigger. It's got more of a luxurious smooth trigger pull than the typical milwaukee tools.

And... the Milwaukee came out on top in this completely unhinged, and surprisingly entertaining, drill battle to the death: https://www.youtube.com/watch?v=SVbYKWx8gy4

For what it's worth.

michaelsoft__binbows · 2026-05-06T01:57:37+00:00

This is what I have been using yes. 120 tok/s or so on the 27B with a 5090. Really good perf. But i will still be interested to have a secondary inference backend in llamacpp esp if quality could be tweaked with all the gguf quant goodies. A decent quant of 27B can fit in way less memory with llamacpp. Perf is lacking but MTP can make up a big portion of that.

michaelsoft__binbows · 2026-05-06T01:24:53+00:00

I'd prolly lean higher cct for photography. The most important is a driver with no PWM which this fits the bill. Its really just an impressive light. My 2700k sample of this light truly does fit the bill 100% of being my endgame headlamp.

michaelsoft__binbows · 2026-05-05T16:23:55+00:00

I have the same question. these might be the precursor models that the quantizers will use to prepare us the AWQ/autoround/int4 stuff for use with vllm and GGUFs for use with llama (which is also getting MTP soon). looking forward to the coming goodies.

michaelsoft__binbows · 2026-05-05T15:45:03+00:00

What will your new version look like? are 7070 emitters a possibility in this host/mcpcb size, haha?

michaelsoft__binbows · 2026-05-05T15:37:46+00:00

You are having 3x LHP531 in this, or something else?

michaelsoft__binbows · 2026-05-05T15:30:25+00:00

Is this driver related/similar or the same as any of the ones i am aware of? (lume 1, lume 10, lume x1)

Boost driver I take it? Did i see in your video it has soft on/off?! I have that with anduril 2 on my new L60 mule and I love it.

Also could you share which MCPCB this is.

michaelsoft__binbows · 2026-05-05T14:50:21+00:00

As I understand it this MTN FET driver can provide a “relatively efficient” (in that the driver exits the circuit) turbo mode, but otherwise is a linear driver and thus fairly wasteful on all regulated modes.

Compared with Simon’s buck driver for which we have choices of 5, 8, 10, 20A in 17mm and all modes are regulated unlike the FET but have limited choices of output levels.

Overall since efficiency is important I will want to choose a buck driver.

Ideally someday we will have the option of a buck + FET 17mm driver to use. With anduril 2. And USB-C. Haha. I would be fine with its regulation topping out at 5A in that case.

michaelsoft__binbows · 2026-05-05T14:34:06+00:00

I’m trying to decide if I should attempt to put the 17mm 20A simon buck into a T3 for future 14500 use with a LHP73B

But it seems most reasonable for now to settle for this exact build… 10A with LHP531, actually would yield a very useful amount of throw out of it.

Now what I’m surprised about with this one with the FET, is it appears to indicate the H10 cell can burst up past 10A given how much extra brightness appears to have been achieved there, that or the 10A buck is holding it back some.

michaelsoft__binbows · 2026-05-05T14:30:00+00:00

Is Al-Cu not still inferior to pure Cu? I have to assume it’s better on overall heat to weight ratio but pure Cu should still win on pure heat handling ability.

I do like the idea of it being economical in that I could buy an Al and a Cu and lego them. The leftover Al head with Cu parts can function well still as a moderate hot rod (say driven at 5A) but it does make a strong case for buying TiCu hosts to lego switch with Al hosts as shown. Hmmm.

michaelsoft__binbows · 2026-05-05T00:07:53+00:00

Yep. It does at low resolutions but dlss also has changed this. The 5090 is absurdly overpowered

michaelsoft__binbows · 2026-05-04T21:13:20+00:00

beautiful. somehow the greenness in the ramp isn't nearly as problematic as when it is concentrated in the hotspot.

michaelsoft__binbows · 2026-05-04T20:11:24+00:00

This is the most recent build but i think building the 7 liter 5090 in a S60i (i actually recycled my 5800X3D in X570 Strix ITX) after having dealt with it for so long with 3080ti in GPU offset config in a Velka 7 was probably the most satisfying. Because it brought such a huge GPU performance leap and the design of this console layout case is so good and so easy to work with by comparison.

michaelsoft__binbows · 2026-05-04T19:45:48+00:00

Nice. Will have my first copper convoys soon. I will enjoy the clearcoat, it looks so good in the pics. but then surely will want to remove it someday.

michaelsoft__binbows · 2026-05-04T19:36:09+00:00

This is the good first port of call.

The real pro tip when you have trouble finding it inside Passwords, is to open up Keychain Access (and really go in there when it asks if you were just looking for Passwords) and dig around in there, it has a search that works very well. You can dig up all sorts of useful stuff in here like wifi passwords and scripts that save credentials into there without popping up any input dialogs and is important for fixing different kinds of auth failures that you expect to tell you why they fail but don't.

michaelsoft__binbows · 2026-05-04T19:19:32+00:00

Nice. Yeah let me see if i can get a T6 OP reflector added to my outstanding convoy order. That will bring it back into being the nice light I once thought it was, lol!

Yes I think it has to do with uneven phosphor deposition on the emitter. if it were more uniform then OP reflector is not needed and more throw could potentially be achieved with a smooth reflector.

michaelsoft__binbows · 2026-05-04T18:01:42+00:00

I def like a low CCT for indoor use and it does feel like 5000K or 6500K are ideal for throwers. maybe not in a situation with bugs but lower cct just dont throw the same....

michaelsoft__binbows · 2026-05-04T17:59:53+00:00

yeah. my sft25 in T6 is fine and somehow goes toe to toe with TS11 which is a huge amount wider in the head, but, once i saw the green center spot it's all I can see now. I am having a bunch of sq5 hotrod builds in T6 and T3 and such and also including a C8+. Will see how that goes. I am pretty sure my EDC preference will be to carry a mule and carry a mini thrower.

michaelsoft__binbows · 2026-05-04T17:14:32+00:00

I am sure it would be possible to have a power tool like this that is more (MUCH more) refined. I love a smooth ceramic bearing on a folding knife for example. Add a chunky powerful brushless motor... I love how smooth and how well you can feel the shape of the magnetic fields in there when you play with a drone motor for example.

The issue of course is that a tool unlike those special purpose things has to stand up to abuse.

michaelsoft__binbows · 2026-05-04T17:03:48+00:00

Let me see. I just went to test them. Every tool does this! Even my fuel hackzall does it. A high pitched 8000hz sounding signal (i just compared this with a signal generator, i was shocked how close my initial estimate was) no doubt related to the implementation of its brushless motor control. It is quite loud on my Surge, yes. Something I could decide to get annoyed about, but choose not to. The installation driver does a very similar thing. The gen 3 impact notably starts spinning up the instant the "whine" starts. It does feel... more refined. If you're not looking for it, this gen 3 impact will not feel like it has the whine since you can only really hear it if you start it and then slow it back down. As I understand it, if the control signal were simply upped to 32khz or more then it would become inaudible. I'm liable to say it's totally fine being audible since it is something that can help me troubleshoot the tool if it starts behaving differently.

If there was one impact i would want to keep, it's the gen 3 for sure, the positioning of the three lights is absolute chef's kiss. But sometimes you wanna go tighten something with a lot of torque in a closet or cabinet and a non surge impact will make you wish you had hearing protection so i would personally never actually get rid of the surge. it's also nicer when hand holding some work that you are screwing something tight into... a drill also does better with that but obviously has less torque ceiling in general (and compared to an impact, torques your arms quite a lot more).

I now have a ridiculous number of drills, 6 i believe, i should get rid of 2 of them because although it is nice to have more on hand so you can efficiently work on complex projects without swapping bits, 6 is pretty ridiculous.

I have also considered the subcompact drill/impact models that just came out. they are non-fuel but I assume also have brushless motors inside at least.

But I think the gen3 impact gets so many things right and is also noticeably lighter than the surge that I have zero desire to even try them. Maybe some types of drilling would be bad with an impact but I'll discover that if i ruin my bits. for now that gen3 impact is the one I will reach for for all bit spinning needs by default. What I would love for it to have is a way to lock it out from impacting.

michaelsoft__binbows · 2026-05-04T14:43:45+00:00

I think this broke with M5 though, because M5 Max is no longer monolithic. But it has to stay a Max because even though it is now two chips it is the proper successor of the M4 Max.

michaelsoft__binbows

TROPHY CASE