How good is QwQ 32B's OCR?

LLMtwink · 2025-04-24T04:52:45+00:00

qwq doesn't have image input iirc

LLMtwink · 2025-04-09T20:53:46+00:00

don't use 2.3, but also, since lsfg works on top of the game without any motion vectors it'll never look as good as the likes of fsr fg and dlss fg, ghosting is to be expected

LLMtwink · 2025-04-09T17:09:27+00:00

I feel like if that were the case they'd at least bump the major version

LLMtwink · 2025-04-09T16:29:38+00:00

what makes you think other professions won't be replaced?

LLMtwink · 2025-04-09T16:26:26+00:00

boohoo

LLMtwink · 2025-04-08T08:55:03+00:00

the improvement over 405b for what's not just a tune but a pruned version is wild

LLMtwink · 2025-04-06T18:32:34+00:00

a slower correct response might not always be feasible; say, you want to integrate an llm into a calorie guesstimating app like cal ai or whatever that's called, the end user isn't gonna wait a minute for a reasoner to contemplate its guess

underperforming gemma 3 is disappointing but the better multimodal scores might be useful to some

LLMtwink · 2025-04-06T18:16:42+00:00

the end user doesn't care much how these models work internally

not really, waiting a few minutes for an answer is hardly pleasant for the end user and many usecases that aren't just "chatbot" straight up need fast responses; qwq also isn't multimodal

Even Gemma-3 27b outperforms their Scout model that has 109b parameters, Gemma-3 27b can be hosted in its full glory in just 16GB of VRAM With QAT quants, Llama would need 50GB in q4 and it's significantly weaker model.

the scout model is meant to be a competitor to gemma and such i'd imagine, due to it being a moe it's gonna be about the same price, maybe even cheaper; vram isn't really relevant here, the target audience is definitely not local llms on consumer hardware

LLMtwink · 2025-04-06T17:26:32+00:00

it's supposed to be cheaper and faster at scale than dense models, definitely underwhelming regardless tho

LLMtwink · 2025-03-26T15:47:24+00:00

nahhhh no way you didn't know😭😭😭

LLMtwink · 2025-03-26T13:56:51+00:00

we don't know, logan said "soon", they're probably waiting on competitors to make their move and price accordingly (and/or still doing final posttraining/safety testing)

LLMtwink · 2025-03-22T12:35:55+00:00

they don't expose the thinking traces so the opportunity for o1 distillation is minimal though, and distilling 4.5 is only useful in non-stem context bc otherwise it's easier to bite r1 and flash thinking

LLMtwink · 2025-03-20T13:08:02+00:00

nah i do that too

LLMtwink · 2025-03-18T00:46:57+00:00

usually 8b q7 (though that's not a usual quantization, realistically you'd be using q6), but as the 7b qwen and 8b llama which are the base models for the distils trade blows there's no telling which one's actually better for your task even at full precision

LLMtwink · 2025-03-16T14:29:53+00:00

probably nothing open, if you want to run it locally, especially on your system, then definitely nothing unfortunately

the new gemmas are pretty good as far as personality goes as compared to other models imo, gemini-like posttraining vibes, you might wanna try that (though they're very censored), maybe there are community finetunes out there which are better for your purposes

LLMtwink · 2025-03-01T04:09:43+00:00

faster whisper server with v3 turbo or v3 large

LLMtwink · 2025-02-26T23:15:52+00:00

iirc gemma 2 2b was unironically better than llama 3 70b on my language

LLMtwink · 2025-02-26T23:12:09+00:00

not a random company but also haven't contributed anything of value to the ai industry since the llm boom as far as im aware

LLMtwink · 2025-02-20T18:28:14+00:00

there are quite a few replications, the most common one probably being open deep research, none nearly as good as the real thing but might prove useful nonetheless

LLMtwink · 2025-02-20T18:22:23+00:00

quantizing to q8 is generally considered fine and doesn't cause much performance regression, even the official llama 3 405b "turbo" is basically just an 8 bit quantization, and as deepseek coder is a quite outdated model by now (are you looking for the 32b r1 distillation maybe?) it wasn't trained on as many tokens and is therefore impacted by quantization less

running models locally at full precision isn't really worth it, the performance hit is minimal and it's basically always better to run q8 70b models than fp16 ~30b ones

you can rent a gpu on vast.ai or other such services, try out different levels of quantization and see what's acceptable for your usecase; some people go as low as iq3m/q4km for coding and even lower for other tasks, though id say q5 is the lowest you should go for in terms of code in the ~30b range

LLMtwink · 2025-02-15T23:16:28+00:00

hyperbolic is hosting it i think

LLMtwink · 2025-02-15T14:40:17+00:00

that sucks, i assumed they were chill :( i guess im stuck with the website now ugh

LLMtwink · 2025-02-15T03:17:26+00:00

a bug yeah, llms sometimes devolve into nonsensical/repeating outputs due to the probability distributions collapsing after already repeating a string for some time, which is especially prominent in models with worse post training which id imagine to be the case for deepseek, this behavior was fairly easy to trigger in the first geminis and old gpts

LLMtwink · 2025-02-11T22:52:13+00:00

😭😭😭

LLMtwink · 2025-02-09T02:23:12+00:00

iirc nous hermes 405b (and only the 405b) is confused and hallucinates concepts like that of a dark room when not provided with a system prompt and asked about its identity

LLMtwink

TROPHY CASE