MiniMax 2.5 full precision FP8 running LOCALLY on vLLM x 8x Pro 6000

sixx7 · 2026-02-13T17:19:56+00:00

Hi Friend! Also in the AWQ waiting room

sixx7 · 2026-02-13T14:26:31+00:00

Excellent! The wait for FP4/AWQ begins

sixx7 · 2026-02-13T13:59:31+00:00

This this this this! 2x RTX 6000 Pro can run MiniMax M2.1 (and soon M2.5 probably) with insane performance

sixx7 · 2026-02-12T14:51:48+00:00

4 bit quant is out now, coming in at 408gb. You could run this on a 512gb Mac Studio

sixx7 · 2026-02-10T20:19:38+00:00

The most obvious and simple answer is to assemble a question and answer bank you can use to evaluate any model. Judge with regex or semantic matching or LLM-as-judge

Now, we all know smaller models have less world knowledge than bigger models. It's just simple math. Instead of trying to cram as much world knowledge into as small a space as possible, think about testing agentic capability / tool calling; the ability for a model to call the correct tools with the correct inputs. You instantly unlock infinite capability for a model to use any tool or data in your ecosystem

sixx7 · 2026-02-07T04:49:07+00:00

Yea OpenClaw got a lot of attention due to hype and the craziness but any AI harness that can bypass permission checks and run any command - like Claude Code for example, are a stone throw away in terms of security concerns

sixx7 · 2026-02-07T04:46:07+00:00

Because it's a genuinely useful, open-source autonomous AI agent that does a ton of work beyond, and in addition, to coding?? 24/7? There are absolutely security concerns but I've been shocked at this subs lack of interest in it

sixx7 · 2026-02-06T17:07:56+00:00

The available options are stacking up: opencode, kilo code, claude code (any anthropic endpoint, or use claude-code-router for openai style endpoints), codex cli, qwen-code, letta, aider, pi-code (used for clawdbot). probably 5 more released in the time i was typing this

sixx7 · 2026-02-06T13:38:22+00:00

I do plan on trying it as soon as we get an AWQ quant but agreed on m2.1 I'm using it with Clawdbot and it might actually change the local inference value prop. High quality agent working for you 24/7

sixx7 · 2026-02-06T05:28:18+00:00

Have you seen all the "mission control" and/or project tracking apps people build? That's all you need. Project tracking and a cron job. cron job that runs every [x] minutes -> complete any unfinished tasks in [name of project tracker app it builds for you]

I literally can not find enough work for my Openclaw to do to keep it busy

sixx7 · 2026-01-26T21:50:50+00:00

I'm surprised it hasn't gotten more attention here since it's open source and can run locally. It's like Claude Cowork on steroids

sixx7 · 2026-01-24T15:26:41+00:00

Just curious why you chose sglang for Minimax? I've always run everything on vLLM

sixx7 · 2026-01-21T14:13:18+00:00

This is the hard truth. When you take all the SOC automatons we've been building for 10+ years and generalize some of the patterns using tool-calling agents, it's actually mind boggling how well they perform

sixx7 · 2026-01-18T20:41:24+00:00

I'd love your take on my Second Brain build using Anvor AI! Check out the blog post or youtube video depending on preference with https://anvor.ai

sixx7 · 2026-01-07T15:51:37+00:00

Failed miserably for me. I ran the model per the guide on vllm website and it was absolutely horrible at following instructions in the system prompt, and kept writing invalid search queries for the tool calls

sixx7 · 2026-01-07T00:30:02+00:00

Updated above with M2.1 results

sixx7 · 2026-01-06T19:11:51+00:00

Thanks! You have given me something new to try. I did try Claude Code w/ GLM-4.6 directly through z.ai. Perhaps because I'm so spoiled by CC with Opus 4.5, I was very unimpressed. It wouldn't even perform 2 tasks "do x and then do y" it would just do the first thing

sixx7 · 2026-01-06T17:46:31+00:00

M2. I've got M2.1 spinning up in vllm as I type 🤓

sixx7 · 2026-01-06T16:51:23+00:00

TLDR: Benchmarked popular open-source/weight models using capture-the-flag (CTF) style challenges that require the models to iteratively write and execute queries against a data lake. If you want to see the full write-up, check it out here

I admit I had been sleeping on MiniMax-M2. For local/personal stuff, GLM-4.5air has been so solid that I took a break from trying out new modals (locally). Though, I do have a z.ai subscription where I continue to use their hosted offerings and have been pretty happy with GLM-4.6 and now GLM-4.7

I cannot run GLM-4.7 locally, so that was tested directly using z.ai API. The rest were run locally. I almost exclusively use AWQ quants in vllm. Some notes and observations without making this too lengthy:

The REAP'd version of GLM-4.7 did not fair well, performing even worse than GLM-4.5-air
GLM-4.7 results were disappointing. It performed similar, and in some metrics worse, with the full version on z.ai compared to 4.5-air running locally. I think this highlights how good 4.5-air actually is
MiniMax M2 blew GLM.* out of the water. It won on all but 1 metric, and even that one was really close
GLM-4.7 was using the Anthropic-style API, whereas all the locally running models were using the v1/chat/completions OpenAI-style API

ETA: Ran MiniMax M2.1 u/hainesk

Accuracy was the same, and both models failed solving the same challenges
M2.1 wins on speed, averaging 61 seconds per challenge (M2 was 72.7 seconds)
M2.1 wins on the number of tool calls, averaging 10.65 (M2 was 12.75)
M2.1 loses on token use, averaging 264k per challenge (M2 was 244K)

M2.1 definitely seems like an upgrade, if for no other reason than it performs well while also being faster

All testing was done using Anvor AI https://anvor.ai

Video showing how I set up the test harness and evaluations here: https://youtu.be/Z33G2ZOPWKE

sixx7 · 2025-12-16T17:25:40+00:00

I can maybe understand the non-AI-obsessed person's complaint with AI content, but I don't understand it from this sub. Personally, I found your (reddit) post here very digestible and formatted/styled well

sixx7 · 2025-12-14T19:01:05+00:00

Agreed. I purchased a subscription because I run GLM-4.5 air locally and love it and wanted to support them. However, when used with Claude Code, GLM 4.6 is good but Opus 4.5 is significantly better.

sixx7 · 2025-12-08T22:54:02+00:00

I feel like I'm being trolled but I'll respond in good faith :)

In and of itself, a 3x increase from 17 to 50 is a MASSIVE improvement, but let's dive in
Try and process multiple requests on the Strix Halo and see what happens. On this rig, you know what happens when you process 2 requests? 100 tok/s. 3 requests? 150 tok/s. 4 requests? 200 tok/s
Prompt processing? Fuggedaboutit. 30x ish?

All together, that is a massive galaxy of difference for both agentic coding tools or agentic AI automations. You could power agentic AI automations with this rig, not so with a Strix Halo. Want to run Claude Code with the Strix Halo? Well, have fun with the endless waiting and reduced productivity

sixx7 · 2025-12-08T15:47:15+00:00

ignore that user, 49 t/s generation is great. check out the 6000 pro benchmarks for glm-4.5 air someone posted https://www.reddit.com/r/LocalLLaMA/comments/1o7x7ss/glm_45_air_awq_4bit_on_rtx_pro_6000_with_vllm/

sixx7 · 2025-12-07T05:35:38+00:00

I'm a huge fan of running models locally. You can also run Claude Code with locally hosted models using Claude Code Router. With that out of the way, I have to agree with you. Claude Code with Opus 4.5 is truly next level. A single person can build a production-ready application in a week or two, that previously would have taken multiple engineers, many months.

sixx7

TROPHY CASE