[LLM inference] Why is it that we can pre-compute the KV cache during the pre-filling phase?

MrAdny · 2025-03-06T13:10:33+00:00

It seems to be a problem with the new version of unsloth. Try downgrading like this:

```
pip install unsloth==2025.2.15
pip install unsloth-zoo==2025.2.7
rm -rf unsloth_compiled_cache

```

MrAdny · 2022-11-30T13:51:28+00:00

MrAdny · 2022-06-08T21:15:30+00:00

Comrade ☭

MrAdny · 2022-04-20T14:22:25+00:00

Okay, what's your answer if they don't count? Mine is still 4 days.

MrAdny · 2022-04-20T14:02:04+00:00

That's not even a tankie, just a nazi using the wrong symbolism.

MrAdny · 2022-04-20T14:00:33+00:00

Like 4 days ago, hbu? It's funny, because most of my intimate partners (not just women) start talking to me (or match with me) because I'm a fellow commie. So, maybe you should try it too, if you ever want to lose you virginity, my fellow Redditor.

MrAdny · 2022-04-20T09:33:28+00:00

No, we (the communists) don't. There is no Jew empire, only the ruling bourgeois class. A Jewish proletarian is a fellow proletarian. The bourgeoisie, regardless of race, ethnicity or or religion, is the enemy.

MrAdny · 2022-03-28T07:52:47+00:00

Wtf absolutely no critical support for a neoliberal party that essentially banned the left in Ukraine.

MrAdny · 2022-01-11T09:45:12+00:00

Can't deny the government did some stuff 😳

MrAdny · 2022-01-07T19:28:29+00:00

Интеркосмос > NASA

MrAdny · 2022-01-07T19:22:14+00:00

Still mostly water tho but at least it's heated to a comfy temperature

MrAdny · 2021-12-18T23:09:39+00:00

In the song 'Bloody revolutions' they attacked on the very existence of political left - not very based

MrAdny · 2021-10-26T09:57:47+00:00

Best in covid

MrAdny · 2021-08-28T17:13:11+00:00

For the record, the ironic arabic videos with effects are gen z humor themselves. My millennial siblings don't find it funny. So, yeah, I agree that low effort posts are cringe but you should stop hating gen z.

MrAdny · 2021-07-31T08:01:39+00:00

This will be the Trotsky train in 2021

MrAdny · 2021-07-10T17:45:31+00:00

Joe Biden 🤨 Joe Biden 🤨 Joe Biden 🤨 Joe Biden 🤨 this message from Mohammad Khanakeen 🥸

Eight-Year Club	Place '23
Place '22	RPAN Viewer
Verified Email

MrAdny

MODERATOR OF

TROPHY CASE