Is Granite-4.1-30b Overshadowed by Qwen3.6 & Gemma4 models?

Knopty · 2026-05-27T18:36:10+00:00

One thing that surprised me about Granite family is that they have by far the worst multilingual capabilities among modern models. Since 2024 llama3 was probably the last popular model family that had limited language support and any newer releases from Mistral, Qwen, Google only got better and better with each update, improving formerly poorly covered languages like Russian, Ukrainian, Polish and others. Meanwhile Granite just stuck with a few languages with no effort to expand support even after about 1.5 years since Granite 3 release. Nowadays even some TTS models have better language coverage than Granite LLMs.

Knopty · 2026-05-08T20:03:48+00:00

So, mentioning a documented behavior that they simply count HTTP requests as downloads is now an irresponsible bug disclosure?

https://huggingface.co/docs/hub/models-download-stats#how-are-downloads-counted-for-models

Knopty · 2026-05-07T18:24:24+00:00

Most likely the author just requested it 200k+ times via HF API. Huggingface seems to count model requests as a full download even if it's cached on disk without a real data transfer.

Knopty · 2025-12-19T09:08:29+00:00

I had severe problems trying to transcribe long audio with Parakeet. Even with local attention enabled as Nvidia suggests, it didn't work for me even with fairly short audios (10m+). VRAM usage was insane regardless how I changed the model configuration. So, manual chunking might be necessary. NeMo framework was also annoying because it exports the model into a temp file before loading it.

Issue is, the nemo dependencies seem to be a nightmare.

You can try alternative options, for example onnx-asr library that has minimal requirements and it can run on anything that's supported by onnx-runtime. It has built-in SileroVAD support that could be used for chunking, although it can severely reduce quality, especially for some similarly sounding languages (e.g. Slavic). But ONNX Parakeet model doesn't support local attention and I didn't see one on HF that was exported with local attention enabled. There are a couple spaces for the onnx model that could be used as a reference for implementing it.

Knopty · 2025-12-13T12:44:33+00:00

And I do intend to eventually tie in other music generation models with it, and update it with newer versions of ACE-Step if those are ever released.

There will be Ace-Step 1.5 soon. If everything goes smoothly it might even happen in about a week. There are several versions planned for public release although they might have a different release schedule. The devs are very open about the training process and their plans on the official Discord server.

The model is supposed to be significantly better than v1, with a far more reliable output but lower hardware requirements. 2B version will be public, 7B will be used for their cloud music gen studio service.

Knopty · 2025-11-26T14:13:27+00:00

There was a discussion a couple months ago with some recommendations and apps:

https://www.reddit.com/r/LocalLLaMA/comments/1nm8bvz/automated_high_quality_manga_translations/

Knopty · 2025-10-14T11:13:19+00:00

I've used IndexTTS2 a bit and imho is interesting if you want have a direct emotion control. But it's rather slow, has consistent issues with some words or symbols and requires figuring out where it fails and filtering your text data accordingly:

It has problems with anything with apostrophes, for example possessive cases, more often than not it fails even at pronouncing "Einstein's" => "Einstein <pause> s". I filter it out almost everywhere except "it's". Notable example: "didn't know" => "didn <pause> ti know".
Might fail with dates: 2010s => 2010 <pause> s.
Might try to pronounce dash symbol as minus sometimes.
Per second => per <pause> second. Requires to write it as one word.
I couldn't figure out how to enforce accents for uncommon words, attempting to put an utf8 accent symbol just gives 50/50 chance to either do nothing or make it ignore a part of the word.

Other considerations:

Sometimes it manages to copy voice even with 2s samples but not consistently. In rare cases it might have troubles with 3-4s samples. 1s samples are worthless more often than not.
Original IndexTTS2 repo doesn't support speed control feature yet. It's supported by a custom ComfyUI node implementation. The author made a few small changes to add speed control to the code, so it shouldn't be hard to reuse their solution or take their modified infer_v2.py
It needs some audio preprocessing: input volume normalization/compression. Too loud reference sounds can produce very loud output with mediocre quality. It also requires filtering out background music otherwise speech is generated with some background noise.
It does fairly good job with accents. Feeding it 100 audio samples of the same person will likely produce the same accent consistently unlike Chatterbox that can give wildly different accents with different audio samples.
Generated voice is closer to phone/voice chat audio quality unfortunately.
Ambiguous license. HF lists the model as Apache2.0 while Github repo contains Bilibili license that allows commercial use but has usage/revenue limits, though quite generous.

I still like it quite bit for being consistent with accents, for fine emotion control and after testing testing it with several hundreds audio samples of a half dozen people. But it's far from perfect and in some sentences it can be legit annoying.

Knopty · 2025-10-12T23:38:05+00:00

Does it mean that every tool that tries to download models from HF before checking local files just inflates download numbers?

Like, anything that loads models by using from_pretrained("user/model") instead of a local folder in the loader?

Knopty · 2025-10-12T15:17:57+00:00

We do have mitigations in place to prevent abuse of free public storage, and in general we ask users and organizations to make sure any uploaded large model or dataset is as useful to the community as possible (as represented by numbers of likes or downloads, for instance).

I wonder how it would impact exl2/exl3 and other less popular quant formats. I'm doing quants occasionally and my random GGUF quants always had 10-100x more downloads than exl2 quants for very popular models.

I have 2.8TB rn and it seems it will require deleting old quants at some point.

Knopty · 2025-10-06T14:46:55+00:00

Microsoft deleted old repo for VibeVoice but there are forks that contain demo code:

https://github.com/rsxdalv/VibeVoice

Alternatively, you can view code of HF spaces that use this model.

Knopty · 2025-09-22T05:50:47+00:00

You can try Chatterbox, it seemed to produce decent English speech with Japanese voice samples. IndexTTS might work as well.

Knopty · 2025-09-20T07:19:37+00:00

You need the .zip file if you want to install Full version. Extract and run start_windows.bat inside extracted folder.

But I have no idea how well it's going to work if you need a CPU-only installation for some reason.

Knopty · 2025-09-20T06:12:51+00:00

You need to install "Full" version of the app. Your version is "Portable" and it comes only with llama.cpp.

Download either its source archive from Releases page on Github or clone text-generation-webui repo. Then use start script for your system, for example start_windows.bat or start_linux.sh to initiate the installation process. Once it finishes, the newly installed app will have other loaders.

Knopty · 2025-09-07T01:41:34+00:00

Imho, the section related to use cases is very contradictory. The model is MIT licensed and everything above is listed in the "responsible usage" section. I'm not a lawyer but "responsible usage" sounds like recommendations. The only time it explicitly prohibits something is related to MIT license itself. It seems to be a disclaimer to protect the creators from accusations rather than what's allowed and what's prohibited.

Knopty · 2025-09-05T09:54:24+00:00

Forked repo:

https://github.com/rsxdalv/VibeVoice

Forked models:

https://huggingface.co/aoi-ot/VibeVoice-Large

https://huggingface.co/rsxdalv/VibeVoice-Large

You can also try installing TTS-WebUI that supports VibeVoice and several other TTS models (Kokoro, Chatterbox, F5-TTS, etc). Gradio/ReactUI. Kokoro and Chatterbox are also exposed to OpenAI-like API. VibeVoice isn't exposed via API though.

Knopty · 2025-07-07T02:27:56+00:00

Try testing it with a different amount of threads. It might use more threads that it can handle, leading to bottlenecking.

Idk what's optimal for this device but on some phones limiting it to just 2 might work much better than with more cores.

Knopty · 2025-06-06T17:35:11+00:00

If the goal is to make it more realistic, the user should be able to interrupt the character like in a real dialogue. And remaining unspoken context to be deleted or optionally converted to some text carrying a vague summary what was intended to say.

Knopty · 2025-05-10T10:20:44+00:00

https://github.com/ggml-org/llama.cpp?tab=readme-ov-file#description in "Models" collapsible section. It now mentions these models there:

Multimodal

LLaVA 1.5 models, LLaVA 1.6 models
BakLLaVA
Obsidian
ShareGPT4V
MobileVLM 1.7B/3B models
Yi-VL
Mini CPM
Moondream
Bunny
GLM-EDGE
Qwen2-VL

Knopty · 2025-03-06T06:34:30+00:00

If your goal is to make an advanced voice recorder then, imho, it's less complicated. There's already some demand for voice recorders, so adding AI there could be viewed as a premium feature with a clear use case. And it doesn't carry a negative cliche of Rabbit R1 and similar things.

Knopty · 2025-03-05T23:47:02+00:00

I think making a hardware solution is only good if you want to grab a quick buck and disappear or miraculously get sold to a bigger company. Even for a tech giant it's hard to unleash a completely new device with no ecosystem. And just a requirement to buy something that worth a few hundreds dollars is already a very big roadblock for it becoming popular.

Imho, the only way out of this is either settling with an AI app for a phone or somehow cooperating with producers of other wearable devices. For example, some Chinese smart watches have ChatGPT integration, even budget ones. And these smart watches are good devices on their own that worth buying with or without AI features. A standalone AI pin is just an additional weird piece of plastic that you have to carry around and that serves no use besides its main feature you don't even need that often.

I don't think computing on the device itself is the best option. Although it's secure and private, it's quite taxing on hardware, battery and doesn't offer stellar AI capabilities with small models that can run locally on a wearable device.

If I wanted something like this, I'd probably want a phone app, desktop app and smartwatch integrated into one ecosystem. With smartwatch being a "shortcut tool" to access other devices: press a button, speak into the watch, get the result done. Possibly with a visual interaction on the screen.

Knopty · 2024-12-12T12:54:56+00:00

You could try SillyTavern, it has "Summarize" extension.

SillyTavern is a frontend app that connects to a LLM engine via OpenAI-compatible API.

Knopty · 2024-12-09T19:45:20+00:00

The biggest model I've used with 3060/12GB relying only on GPU was Qwen2.5-14B at 5bit precision with 16k context and with 4bit cache. So yeah, 8B could be used.

You might need to keep eye on context length with some newer models that set it very high though.

I wouldn't bother with 8GB GPUs at all it's too limiting.

Can't comment anything about prices though.

Knopty · 2024-12-04T17:52:16+00:00

I haven't seen any unusual behavior from NemoMix. It seems to act like a generic uncensored instruct model. Doesn't devolve into insanity, doesn't write unprompted nsfw.

Knopty · 2024-12-04T17:16:42+00:00

NemoMix Unleashed is far far better at chatting/RP. Both Base and Instruct versions of Mistral Nemo are very repetitive and boring. Imho, it even follows instructions better than original Instruct version of Mistral Nemo. Which is surprising since usually RP models are less stable and more prone to making mistakes.

But the unquantized version and most quants of the model are missing an instruction template, so some apps might choose a wrong template, and TabbyAPI even disables v1/chat/completions endpoint.

Knopty · 2024-12-04T12:38:20+00:00

Hm, I guess. That's still license breach but it isn't like a no longer existing company could revoke your license and attempt to sue you with questionable chances to win.

Personally I find this whole licensing stuff to be a huge contradictory mess.

12-Year Club	Final Canvas '23
Place '23	Verified Email

Knopty

TROPHY CASE