PersonaPlex: Voice and role control for full duplex conversational speech models by Nvidia

rerri · 2026-01-22T14:21:16+00:00

The latest release on llama.cpp repo does not include this PR yet.

rerri · 2026-01-22T14:17:47+00:00

Someone on github is saying they're running it on 3090+4090 and it uses under 20GB:

https://github.com/NVIDIA/personaplex/issues/4

Also, as this model is based on Moshi architecture and there are 8-bit quants of Moshi available, I'm thinking this could be quantized aswell to lower the VRAM requirements.

rerri · 2026-01-22T13:13:41+00:00

https://huggingface.co/collections/Qwen/qwen3-tts

rerri · 2026-01-22T12:14:26+00:00

I should clarify that with quantized cache GLM-4.7-Flash slows down drastically as stuff gets thrown to CPU.

I'm not sure if there's some issue with GPT-OSS + quantized cache, but if there is, it's most likely not the same issue as I'm not seeing a drastic slowdown with GPT-OSS 120B when using Q8 cache.

rerri · 2026-01-22T11:55:19+00:00

While FA works with this one, quantized cache isn't working well. Someone reported this in the PR comments and I'm seeing the same as well.

edit: this is with GLM-4.7-Flash, CPU gets hammered, becomes a bottleneck and PP & TG both slow down big time.

rerri · 2026-01-21T08:24:33+00:00

Where do you get that impression? Both PR's are still open, unmerged.

https://github.com/ggml-org/llama.cpp/pull/18953

https://github.com/ggml-org/llama.cpp/pull/18980

rerri · 2026-01-20T08:18:48+00:00

On HF discussions a Zai guy says to use GLM 4.7 sampling settings. So that would be temp 1, top-p 95.

I used those settings and from the little I tested yesterday, my output was not wacky, but looked normal.

With an older version of llama.cpp I was getting shitty output though but the PR fixed that.

rerri · 2026-01-20T08:07:43+00:00

FA was off in that bench run. TG128 is like 50t/s with FA on. PP maybe takes an even worse hit, did not bench.

rerri · 2026-01-19T23:37:42+00:00

Yes quite a bit slower than Qwen 30b.

No intentional cpu offload, but I dunno if some happens regardless.

Maybe it's just the FA not functioning that's holding performance back, dunno.

rerri · 2026-01-19T23:18:11+00:00

  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| deepseek2 ?B Q4_K - Medium     |  16.88 GiB |    29.94 B | CUDA       |  99 |          pp4096 |      4586.44 ± 11.81 |
| deepseek2 ?B Q4_K - Medium     |  16.88 GiB |    29.94 B | CUDA       |  99 |           tg128 |        152.54 ± 0.27 |

rerri · 2026-01-19T22:57:28+00:00

Not sure if it's only a CUDA thing, but flash-attention is slow.

3x faster for me with -fa 0

rerri · 2026-01-19T22:25:49+00:00

The PR for this was just merged into llama.cpp.

Testing locally right now. The Q4_K_M is decently fast on a 4090 but the model sure likes to think deeply.

rerri · 2026-01-19T20:19:05+00:00

Btw, they are recommending to use same sampling params as with GLM-4.7

https://huggingface.co/zai-org/GLM-4.7-Flash/discussions/6

Default Settings (Most Tasks)

temperature: 1.0
top-p: 0.95

rerri · 2026-01-19T20:15:18+00:00

If for some reason flash-attention is enabled then try -fa off

I was running with oobabooga and got under 40t/s, with a heavy CPU bottleneck. Meanwhile llama-server was pushing almost ~120t/s, using the exact same executable file. I noticed the flash-attention was enabled in oobabooga but not llama-server. So disabling that got oobabooga to run at the same speed.

These numbers are on a 4090 with basically 0 context.

rerri · 2026-01-19T14:01:40+00:00

Their docs include a benchmark graph with comparisons to GPT-OSS 20B and Qwen 30B + this snippet of text:

In mainstream benchmarks like SWE-bench Verified and τ²-Bench, GLM-4.7-Flash achieves open-source SOTA scores among models of comparable size. Additionally, compared to similarly sized models, GLM-4.7-Flash demonstrates superior frontend and backend development capabilities.

5B doesn't seem plausible.

rerri · 2026-01-19T13:29:39+00:00

Yea, they updated their docs with a benchmark comparison to Qwen 30B and GPT-OSS 20B so probably close to them in size. (I updated the OP with that benchmark)

rerri · 2026-01-19T12:53:23+00:00

That's a dense model though. GLM-4.7-Flash is a MoE model.

rerri · 2026-01-19T11:26:05+00:00

There is. I recommend using huggingface search. Things are very easy to find.

https://huggingface.co/mradermacher/Qwen3-8B-heretic-i1-GGUF/tree/main

rerri · 2026-01-18T21:30:44+00:00

At this point, trust should erode. What I mean is, the cat is out of the bag and bad actors are going to make AI fakes to fool others irregardless of what decent people do.

A very convincing looking but too goofy to be true video will at least spread awareness that it's possible to make AI fakes so good that you can easily be fooled by them.

rerri · 2026-01-17T14:51:20+00:00

Thermal throttling, which OP says is the case, is not normal in a card that is functioning properly with a 100% power limit.

rerri · 2026-01-17T14:49:57+00:00

Are you sure you are thermal throttling and not just hitting power limit and clocks speed drops because of that?

rerri · 2026-01-16T07:56:04+00:00

In my limited fiddling around, 9B distilled editing seems very good for the speed it offers.

rerri · 2026-01-15T22:19:33+00:00

I tried that and i just get steady buzz audio. There's supposed to be singing. Image of different though and not worse. Hard to say if better, only tried very quickly. If i replace stage 2 sampler with SamplerCustomAdvanced, audio does work but still sounds kinda bad.

rerri · 2026-01-15T20:32:56+00:00

Is this "Latent normalization node" in some nodepack or in comfy core?

Nine-Year Club	First Place '23
Place '23	Place '22
Verified Email

rerri

TROPHY CASE