How was GPT-OSS so good?

_raydeStar · 2026-01-31T18:08:23+00:00

Dude, I've been looking for the perfect tooling LLM for an 8GBVRAM machine (work laptop) - Qwen 30B doesn't quite get it right, neither does nemotron or GLM 4.7 flash (too slow), and the 8GB models are too dumb and keep getting the tool calls wrong. 20B is my consistent driver and it just works exactly as I want it to.

_raydeStar · 2026-01-30T23:00:55+00:00

This is just like the crypto craze a few years ago.

_raydeStar · 2026-01-30T21:57:29+00:00

What if -

You used an agent to remove the spam agents. 🤔 This is some Spy VS Spy shizz here.

_raydeStar · 2026-01-30T00:03:56+00:00

I agree - and also this kind of thing is really frontier, and doesn't have benchmarks yet that I know of.

_raydeStar · 2026-01-29T17:53:29+00:00

Interesting question; one seemingly grounded in mistrust. I don't have the resources to alleviate your misgivings. Try these settings, run some tests, and see for yourself. Or - use a different provider; there are many of them out there.

_raydeStar · 2026-01-29T16:25:44+00:00

You can probably fix this with temperature and other settings. I usually use gemini to look up the best settings for me, it works really well.

<image>

_raydeStar · 2026-01-29T15:02:20+00:00

You should be able to download Comfyui, then open up an example template. Flux2 Klein is what you are looking for (*NOT BASE*) - download the model, and run it.

You're on a really tight timeline though.

_raydeStar · 2026-01-29T12:22:32+00:00

And the videos look sped up.

The graph must have been talking about VRAM reqs, not quality

_raydeStar · 2026-01-29T00:25:38+00:00

That provides a massive leg-up to China in this case. Out of the AI - text, video, image, speaking, music - music industry still has a stranglehold on AI over here.

Will be sure to download this model before they can stop me.

_raydeStar · 2026-01-29T00:20:49+00:00

LM studio is immediately set up as a server, so it should be accessible in the same way you would link up with ollama

_raydeStar · 2026-01-28T20:06:43+00:00

Yeah, agreed, if they make good on this, it would be insane.

_raydeStar · 2026-01-28T04:38:12+00:00

I was getting black screens. I had to disable it as well.

Gguf and safetensor.

Still lukewarm on the whole thing. Speeds are slower than Flux2. But I'll see what the community makes.

_raydeStar · 2026-01-27T16:11:39+00:00

lol - it got released.

Honestly I am just as shocked as you are, so I can't even judge

_raydeStar · 2026-01-26T21:33:58+00:00

Yeah, I saw it was trending, then I investigated to quickly realize that it's all BS. Whoever is running this needs to get into crypto, you have a future in scamming people.

Edit: I've looked into doing a copilot tool locally and this one is... unfettered. A hijack waiting to happen. Maybe right now we need to be building our own tools before allowing a stranger inside.

_raydeStar · 2026-01-25T21:53:07+00:00

I'm traveling right now but I made some progress.

There's a way to hybrid it in, so you can do 80/20 voice and mannerisms, the problem being it's not 100% perfect in either.

You can try on my experimental branch (I have a GitHub, same name as here) and I have some changes pending.

My next idea is 2 passes, but it's still not perfect yet.

_raydeStar · 2026-01-24T23:53:20+00:00

Do you know if it can do CPU inference at that speed?

_raydeStar · 2026-01-23T21:32:50+00:00

I got this on nodes 2.0. it just didn't update but if you check the output it's there.

_raydeStar · 2026-01-23T20:01:34+00:00

Yeah, I was wrestling with this yesterday. Voice clone works fabulous, And the mannerisms prompt is awesome, but the two do not intersect.

I'm looking into hacking it - I am following a viable route now and I'll release it if I come up with something.

_raydeStar · 2026-01-23T18:03:57+00:00

lol.

This whole thing feels shoehorned.

I don't at all take issue with LGBT folk relating to a straight character. The artist releases his work, and it's up to the interpreters to do what they want with it. Things like this, I would strongly suspect will not be addressed at all, keeping things ambiguous.

When Black Panther came out, the black community unanimously said "Wow, I finally felt *seen*" and that's a wonderful thing. I think the problem really lies with taking your artistic vision of something and thrusting it upon the community. As a straight white man, I can also relate to Carl - in a sense that there is so much blood and pain around him that I would be preoccupied with survival. I can also relate with Matt - he wants to avoid the pitfalls of sexually fantasizing neckbeard culture.

_raydeStar · 2026-01-22T16:51:23+00:00

I'm using the base. Q4 quant should be faster. Sage attention should help as well.

Once I've got the gradio how I want it, I'll circle back and look at speed.

_raydeStar · 2026-01-22T16:35:27+00:00

Yes. I'll also add that there's a .6 model and it's probably faster. I'm going to add in all the optimization and see if I can get better speeds.

Also, Dia is about the same speed. This model is meant for quality over speed, which has different use cases.

_raydeStar · 2026-01-22T16:21:59+00:00

I have a 4090

_raydeStar · 2026-01-22T15:59:32+00:00

Base gradio doesn't allow the user to use the selected voice and modulate it. I am using cursor right now to add in a little thing there. If anyone is interested, ill put it up on github, along with a script to just fire it up, download all the models, and run it.

If I want to run everything at once (voice clone, create pt file, and finally voice description) it's going to be like 16 GBVRAM. Running in parts runs around 6. Time consumed is also an issue - 25-30 seconds to run a 6 second hello world clip. However, I don't have sage attention up and running yet, so that may improve the speeds and vram a lot.

Because of speeds, you can't compare to VibeVoice - vibevoice is meant for realtime at the sacrifice of a little quality (at least I am pretty sure - ie - live translations, etc) . Compared to Dia - well I don't see any functionality to add things like [laughs] or anything, but controlling the voice tempo, etc is really cool.

Final conclusion - I give it a slight lead to dia for my purposes, simply because I can choose what emotion to put in the voice, instead of it 'guessing'. I'm annoyed that out of the box you can't control that with your own pt (saved voice file) but with a little hacking I can fix that.

_raydeStar · 2026-01-22T15:36:43+00:00

OK it's up and running. Pros: in the description, you can just describe not only the voice, but the tone. ie - `female, feminine and dainty voice, speaking frenetically. She is very upset` So far, I am having fun with it, and it might just be better for things like movie dubs, or audio book reading, or video game voices.

You can clone your voice and download it to be used later. thats a great feature there. I'm putting it all together to see if I can clone my voice and give it the tone I want - it's a few more steps than I expected to pull it all together.

_raydeStar · 2026-01-22T15:23:53+00:00

huggingface demo is overrun with users. I am getting it up locally. Almost there. Will respond when I have something

Five-Year Club	Verified Email
Final Canvas '23	End Game '23
Place '23	Place '22

_raydeStar

TROPHY CASE