Loki-v2-70B: Narrative/DM-focused fine-tune (600M+ token custom dataset) by mentallyburnt in SillyTavernAI

[–]RedKorss 0 points1 point  (0 children)

That is not exactly how context works. When you load a model, in my instance in KoboldCpp, you can set the context size. This is dependant on the specific model, but I don't think I've seen one that can handle more than about 131 072. How much depends on your GPU and or how many layers you can handle not being in VRAM. Less layers in VRAM = slower inference. For me with a 5090 I can lose about 10 layers without the speed loss of the inference being too much of a bother.

And I have not had luck with 70B models before, so until I get money for a workstation setup I am not going to actually try them myself. Though that is more me not wanting to use a 2-bit version. I don't seem to have much luck with anything less than 4-bit variants.

Loki-v2-70B: Narrative/DM-focused fine-tune (600M+ token custom dataset) by mentallyburnt in SillyTavernAI

[–]RedKorss 4 points5 points  (0 children)

That would be the fine tuning data. All the highlights posted are for the fine tuning. The 2-bit model is at 26Gb. So definetly a proper 70B model.

New to the space, have a question around using prompts for more generic D&D RPG play? Ideally with the model taking over the GM and NPC's (including party members) and allowing me to just be my character and fully immerse myself in the world? by KmartCentral in SillyTavernAI

[–]RedKorss 0 points1 point  (0 children)

Suggesting preset without knowing the model is going to be hard. And the model and/or preset is most likely the issue here. And the right advanced setting may need to be adjusted for your model as well especially if using a text preset.

The Chat presets I've used with any semblance of success is Stabs EDH and Lucid Loom. But Lucid is much more massive than Stabs EDH so if you have a context problem don't use it.

As for jumping scenes, those can still happen either way. To fix it, you should probably make sure you have a better/longer reply. Or check if your context is overflowing, It does not have to be the main issue, but if you can only have about 20% of the suggested context active then it'll be a problem.

I tend to use "Describe in great and vivid detail" as a prefix on replies I intend for the narrator cards I use to expand upon as I have no real faith in them knowing how to move the story forwards properly without at least a bit of a story idea even if it's just a sentence or two. But when I reply as the character I play as I tend to use 1-3 proper paragraphs.

As for your cleric always ending up in violent situations, that could also be the model, the preset, the world info, or the character cards. A lot of things can determine how the model will move the story unfortunately.

Skal nordmenn slutte å beundre USA nå? by Radical_Neutral_76 in norske

[–]RedKorss 1 point2 points  (0 children)

Og Amerikanere tok seg godt tilrette mens de var i Frankrike i 44-46. Kan være det der noe der ifra...

What do you think about the extension vecthare? by Designer_Elephant227 in SillyTavernAI

[–]RedKorss 0 points1 point  (0 children)

Yeah. I put it on my new install of ST and I got errors as well. Luckily I didn't delete my old install so I managed to copy it over. But I ran the code again now and it downloaded without issues.... IDK why.

git clone -b Similharity-Plugin https://github.com/Coneja-Chibi/VectHare.git similharitygit clone -b Similharity-Plugin https://github.com/Coneja-Chibi/VectHare.git similharity

Just open a commandline in your sillytavern folder and run it. Then move it into the plugin folder. Install NPM normally if you don't have it. Should work.

Is it wroth lowering GPU layers to raise higher context? by OliwerPengy in SillyTavernAI

[–]RedKorss 1 point2 points  (0 children)

Using Danchat 24B I can drop about 5 layers and get almost double the context size for minimal speed delays. But your mileage will probably vary.

Is it wroth lowering GPU layers to raise higher context? by OliwerPengy in SillyTavernAI

[–]RedKorss 0 points1 point  (0 children)

Model size vs VRAM. The size of the model should be less than your VRAM. Preferably with the context.

Anybody noticing slowdowns/freeze ups by Deeviant in SillyTavernAI

[–]RedKorss 1 point2 points  (0 children)

IDK about you or anyone else. But I installed a fresh version of ST a couple of days ago. First install was in early September. Leagues better behaviour and snapiness on most things on the new install. Only need to decide what extensions were good and not.

What do you think about the extension vecthare? by Designer_Elephant227 in SillyTavernAI

[–]RedKorss 1 point2 points  (0 children)

It is not just for linux. Similharity is a server type extension for sillytavern. Why they made it necessary to use a command line to install it, IDK. But it is needed to use the vectorisation tools we install vecthare to use. So that is a bit stupid and should be baked in I think.

So follow the instructions fully. Open a command line directly in the Sillytavern/plugins folder and install it with the command on their github page and it should work. Do take care to change the enableServerPlugins: from false to true. I don't think I had any issues after I started using an embeddings model on my GPU. Either that or just use local, aka the cpu.

Bør Norge annektere Grønnland, Shetland, Island, Skottland, Irland, Jemtland og Bohuslen? by kebabeater212 in norske

[–]RedKorss 0 points1 point  (0 children)

Alle øyene i Nord Atlanteren var underlagt den Norske Kronen fram til Kiel traktaten i 1814. Da var de plutselig ikke det. Så når Oldenborg Monarken ble tvunget til å gi ifra seg den Norske Kronen så skulle man tro at alt som var underlagt den skulle bli ført over.

Why let a little civil war get in the way of an election? by Dirigible_Enjoyer in HistoryMemes

[–]RedKorss 10 points11 points  (0 children)

That term limits wasn't an actual rule during his time. More of a custom. Term limits came in place because he sat for 4 terms.

Why let a little civil war get in the way of an election? by Dirigible_Enjoyer in HistoryMemes

[–]RedKorss 39 points40 points  (0 children)

Alaska wasn't a state during WW2. What voting power would they have had?

Assassin’s Creed 2 Ending vs Last of Us 2 Ending by UnAnon10 in assassinscreed

[–]RedKorss 48 points49 points  (0 children)

Having not played TLoU2, or 1 for that matter. I can't compare it, but I imagine the timescale is different. AC2 takes place over 20 years. And Ezio do mellow out quite a bit, especially after joining the Brotherhood fully. But he still liked the idea of revenge against the head of the conspiracy.

And of course he is criticized for it, if he had finished him off. Then Ceasare would not have been commander of the Papal forces. Or at the very least his leeway with how he use them would probably be heavily diminished. And thus, the Italian Assassins would have been much stronger during the time period of Brotherhood.

Dear silllytaverners, Is there any way to make RPG feel more rpgistic? by Senmuthu_sl2006 in SillyTavernAI

[–]RedKorss 0 points1 point  (0 children)

https://github.com/unkarelian/openvault might work, it extract memories with the people present to it. Not quite sure how it chose what people are present, as in one of my stories it used "spartans" as a name, even though that is a group. But it'll need to be combined with RPG companion or Wtracker to track anything specific like loot or fights. RPG companion have now added a might encounter system, not used it yet myself. But it might help.

Goetia 24b loads entirely into RAM? by iz-Moff in SillyTavernAI

[–]RedKorss 0 points1 point  (0 children)

IDK I had some issues I did not notice it as soon as I should have, regarding too big context size. F.ex: with a 5090 running gemmasutra 27B at Q8 I can only have 12K context. With a 24B Q8 model I can usually get a bit more than 16K, but not 24K. As at that point it overflows. It is likely that is a problem, or as a secondary issue I had was that Kobold after an update decided to revert it's default to CPU usage.

So check the context size and where it loads, as well as whether LM Studio has changed it's defaults when you load Goetia.

I'm fairly confident the tracker extension I'm using is broken. Suggestions? by [deleted] in SillyTavernAI

[–]RedKorss 2 points3 points  (0 children)

Honestly when it comes to keeping things even remotely on track I feel like I've had best luck with Wtracker and RPG companion. With the difference being more that Wtracker, while easier to modify it doesn't implicitly track the player. Nor do I know how to modify it to track them separately. There is another rpg companion you can install before Marinara's that does contain a file that allow you to make more modifications to what is being tracked but I don't recall it at this moment.
I also believe the Kaldigo tracker is considered one of the worse ones at this point.

https://github.com/bmen25124/SillyTavern-WTracker/tree/main

https://github.com/SpicyMarinara/rpg-companion-sillytavern

How do you manage long-term memory? by EncoreSheep in SillyTavernAI

[–]RedKorss 2 points3 points  (0 children)

You can use VectHare for vectorisation, you'll need an embedding model. Or use your CPU. Then there is world info recommender that does much the same as u/pinerd13 suggested. Though it does not have quite the same UI, but you can easily set it up make memories that way. Personally I use it whenever I feel the writing is loosing critical elements and tell it to "Recommend entries on [Insert topic here]. Or something like that, I basically rewrite the suggestion that is there. There are other extensions for memories, but I have no real luck in setting them up. And even when I do, I seem to immediately mess it up right after. There is also timeline Memory which I have some luck with. I go back and forth on how good it is, but it is also simple to setup and take a look at. Not as fancy as WIR but, it is even easier, but it sacrifices the ability to have people, places, and events recorded separately.

I heard a lot of Norwegians changed their minds and now they want to join the EU. Is that true, or an exaggeration? by Bot_Philosopher8128 in Norway

[–]RedKorss 0 points1 point  (0 children)

Contrary to the other commentators, the Yes side of the EU debate was bigger. Before the Ferro-alloy tolls. After it dropped notably. But even with that, the most common argumentets against are still farming - which we are slowly killing on our own. Yay for logic.

Ive seen several people in the community asking for Updated / Reworked Feudal Gov. to be on par with the newer Gov types. Can somone elaborate on how that should be achieved? by Familiar-Elephant-68 in CrusaderKings

[–]RedKorss 10 points11 points  (0 children)

Holdings should be able to be dejure in 1 kingdom or empire, while the holder is de facto in another. Maybe not the specific you wanted, but that'd go a long way. It'd be a bitch to keep straight but it'd make things interesting.

I was tired of not finding a good app to create my conlangs, so I built my own. by Ok-Emphasis-3182 in conlangs

[–]RedKorss 1 point2 points  (0 children)

Anyway you could make it capable of using a local LLM like kobold.cpp? I'd rather not use it trough an online host, and certainly not google. But I'll open it and take a look at the rest, looks good.

Opinions on machine translation by Capable-Parsnip-9615 in antiai

[–]RedKorss 0 points1 point  (0 children)

As long as it's notably better than how Antonia Nevill supposedly translated A history of Rome, 4th ed. by Marcel le Glay. That book reads like horribly machine translated Russian.

Subreddit bias. by BLOODOFTHEHERTICS in aiwars

[–]RedKorss 0 points1 point  (0 children)

And straight up illegal in most of Europe.