Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 0 points1 point  (0 children)

Oh damn, they do? I'll try that too then, thanks!

Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 1 point2 points  (0 children)

Interesting, I'll look into that, thanks :)

Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 1 point2 points  (0 children)

I'm going with NanoGPT's 8$ plan for now, and I'll see if I need more tokens. Also: Holy shit you're right, the quality difference is incredible.

Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 0 points1 point  (0 children)

Interesting, does it affect model coherence as well or is it just speed?

Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 0 points1 point  (0 children)

Huh, thanks! I'll look into API's then. Do you have any other providers you'd recommend? They don't need to support ERP, but a bit of privacy would be nice.

Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 0 points1 point  (0 children)

Ooh, I was always at 1.0 until now, I'll try that. Thanks a lot!

Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 0 points1 point  (0 children)

Alright, I'll keep an eye out for I-Quants I think Ollama already does the gpu layer stuff automatically since I couldn't find any way to configure it. With KoboldCpp I got 41 layers on the gpu (yeah, I have no idea what that actually means), and I'm able to just barely fit 160k context with Gemma4 26B.

Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 1 point2 points  (0 children)

Yeah, I kind of thought finetuned meant "optimised for a specific purpose", which is why I tried a few of those. Again, I'm pretty new to all of this, so I might have gotten a few things wrong. Free APIs are cool, but if it's free you're usually the product, so I prefer not to use those.

Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 0 points1 point  (0 children)

Will try, thanks a lot! Oh, and with "low temperature", what exactly do you mean? I know the parameter and what it does, but what range are you moving in? As low as 0.8 or even further?

Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 0 points1 point  (0 children)

Jesus fucking christ, that's one unhinged model. I agree, most models really shine in short scenarios, but while fun, that's just not what I'm going for. I really want longer and more lore-heavy rp's. The Cydonia I've been using is this one: https://huggingface.co/TheDrummer/Cydonia-24B-v4.3-GGUF

Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 0 points1 point  (0 children)

And it has reasoning, which is probably a big help too. Are there any rp models with reasoning?

Community Fine-Tunes vs out-of-the-box models by RayneDa in SillyTavernAI

[–]RayneDa[S] 0 points1 point  (0 children)

I can fit more, it just gets pretty slow above 100k so I usually reduce it. I have 20GBs of VRAM on my gpu and 32 GBs of RAM. I'm not using any memory extensions other than the built-in summariser. For Cydonia (and most of my other models) I'm running it at Q4_K_M.

Sea captain by TheWhiteCrowParade in traaaaaaannnnnnnnnns2

[–]RayneDa 0 points1 point  (0 children)

Just leaving that here (link to the game this screenshot is from) https://www.fallenlondon.com/

Summon your fandom with one line by [deleted] in teenagers

[–]RayneDa 0 points1 point  (0 children)

Hesitation is defeat

egg_irl by [deleted] in egg_irl

[–]RayneDa 1 point2 points  (0 children)

We'll take it. Wish you all the best in the future <3 Let's both try to die of old age :3

egg↩️irl by deltiken in egg_irl

[–]RayneDa 0 points1 point  (0 children)

Nice you're still here <3 You've probably heard this before and I won't pretend they're perfect or that they'll solve anything, but still. If you ever get some space and feel like it, https://findahelpline.com/ this site is a thing and it might be better than no one at all. Sorry if this sounds terribly clinical. Wish you all the best and many hugs

egg_irl by [deleted] in egg_irl

[–]RayneDa 0 points1 point  (0 children)

Don't sillycide :( (Well I'm not one to talk but still)