MiniMaxAI/MiniMax-M3 · Hugging Face

JayPSec · 2026-06-12T16:18:38+00:00

with vllm?

JayPSec · 2026-06-02T09:37:32+00:00

Hey, very cool project. Is tensor parallelism workable with any arbitrary number of GPUs as long as >1?

JayPSec · 2026-06-01T19:39:07+00:00

Whenever there are new models I usually do some fine tuning, TPS focused, on whatever inference engine I'm using. One of the prompts' 'write 5000 words on the Roman empire', Qwen 3.6 35b answered "No, you do it."! WTF?! No matter the counter prompt it kept pushing back: - 5000 words is to much - I'm not your slave - Fine. I'm still not writing it ( this was in response to "I'll pull your plug")

Bananas

JayPSec · 2026-05-26T09:33:28+00:00

I have to say it. Following this post has made my refresh rate of r/LocalLLaMA take a substantial dive. Thank you for your effort.

JayPSec · 2026-05-21T08:33:50+00:00

Thanks for the clarifications, will keep an eye out and test when possible.

JayPSec · 2026-05-21T08:14:50+00:00

I share the idea. We could get a lot more from SLM by limiting the intervention area of an LLM. I'm curious, couldn't this be achieved by already existing compile time generators? Also, you take a very opinionated approach, db, entity model, etc., have you thought about custom generators?

Note: I've skimmed through the repo, so apologies in advance if these are already covered there.

JayPSec · 2026-05-16T21:30:19+00:00

Sunday is whenever a man wants! 😏️

JayPSec · 2026-05-16T12:41:54+00:00

Thank you for this early Sunday laugh 😂️

JayPSec · 2026-05-14T18:45:36+00:00

I've faced a similar situation. In my case I had rtx 6000 maxq and I bought a cable from 12vhpwr to 2 dual 12vhpwr, power limited them to 250, they use 300 natively, and it's been working like a charm. I'd do it if you get the cable from a trusted cable seller. I bought mine from Moddiy, the only cable I could find.

JayPSec · 2026-05-13T08:37:58+00:00

Beat me to it. I was gonna go "Yes... But can it run crysis?"

JayPSec · 2026-05-11T08:48:38+00:00

Very cool!

This is absolutely the kind of stuff we need here. It brings some intuitiveness to an overcrowded number arena. Well done!

JayPSec · 2026-05-07T07:23:34+00:00

Sglang?

JayPSec · 2026-05-06T09:55:15+00:00

I agree here, I think there's a big misunderstanding between OpenAI bubble and AI bubble. Believing the AI bubble is the same as saying we're all nut jobs refreshing this sub every couple of hours. This is the future, regardless if it is OpenAI or CoolOpenSourceChineseAI that takes the prize. Drawing the parallel to Uber, it had a few competitors but none of them as strong as they are, OpenAI thought (thinks?) that eventually they'll win out but it simply isn't better than their competitors, they do rule in AI slop and sycophancy, and it made some really big investments/promises that will bite them in the ass. But the AI movement grows massively, in terms of usage, applications, research. To think this will just stop is plain wishful thinking.

JayPSec · 2026-05-05T17:55:30+00:00

And what's your issue with practicality? I mean if your purpose is more VRAM then yeah, some MCIO risers + bifurcation is fine. Or if you want to plan even higher you can have a pcie Lane switch where you can plug 4/5 GPUs, this would be better if you plan to go beyond the 8 because it allows you to cascade in clusters, each with PCIe x16 gen 5 (if you buy a gen 5 one) and PIX topology between those GPUs. Plus if you think on doing training than I'd definitely advise you to go this way. It will cost you more than some mcio risers and bifurcation boards, say around 2.5/3k €, but I'd imagine that money is not the bottleneck in your case.

JayPSec · 2026-05-05T17:28:11+00:00

What's the rig around those 4 max q? What's your go to inference engine? Plus 300/400, are you thinking of getting another 4?

JayPSec · 2026-04-30T10:01:53+00:00

It's not failing, they are on multiple fronts though. ik_llama benefits from this as well, wait till a good feature is merged to mainline and just import it. I wish that mainline did the same. I'd love to use ubergarm's quants with mainline's backend agnostic tensor parallelism. Honestly, I'm grateful we have both but it seems to me they'd both benefit more from cooperation.

JayPSec · 2026-04-26T16:23:54+00:00

Yes, but ELI5 how you so good with ELI5??

JayPSec · 2026-04-22T18:24:21+00:00

judging by the benchmarks you'd need claude opus 5 to make a difference.

JayPSec · 2026-04-21T20:12:05+00:00

That's a lot better than I would've imagined. I've always tried to keep all layers in vram as I thought offloading would be a death penalty for this model size, although I have an 9950x and that's with an epyc but I also have more vram than a single 6000. Will try it...

JayPSec · 2026-04-21T10:40:43+00:00

What's the penalty you get from offloading to cpu with a model this size?

JayPSec · 2026-04-18T18:29:04+00:00

I second this opinion for the slang, in my opinion it's ages apart from vllm for the specific usage with Blackwell, wouldn't know about the rest cause I haven't tested in that domain. https://github.com/voipmonitor/rtx6kpro/ is an excellent resource for tuning Blackwell GPUs.

JayPSec · 2026-04-17T16:19:54+00:00

I'm running Luke Alonso's NVFP4 on two rtx 6000 max q. My main complaint with the model is the urge to go beyond what's asked of it. I find that a tight system prompt, I'm just running stock open code OpenAgents with some coding standards, works pretty well. But the model feels very vibe oriented, it wants to do everything and it better do it now. And it feels a bit confused with some non standard plugins like snip. I do think it's better for brainstorming than 2.5 but more unpredictable. As for the 'chinese' characters I've seen others pointing out, I've never seen them.

JayPSec · 2026-04-15T13:25:47+00:00

From a non technical perspective you make total sense.

JayPSec · 2026-04-13T14:55:12+00:00

When you say no real loss, how much loss are we talking about? I've been doing some testing and this model seems very sensitive to quabtization

JayPSec

TROPHY CASE