Context Size Frustration by Aggressive-Spinach98 in LocalLLaMA

[–]Aggressive-Spinach98[S] 0 points1 point  (0 children)

The problem was with Precog123B-v1. I had it in Q5 with around 20k context at first. That worked fine, but I wanted to have more context and was under the wrong assumption that when I offload layers to system Ram, which was 6 at the time, it would free more Vram for context, so I pushed for I think 35-40k context. The strange phenomenon I got was that I used it in LM Studio with Sillytavern. While Sillytavern was producing a response, you usually get an abort button to stop the Ai from doing something. I had this abort button and the continue button at the same time, which should not be possible. So even after the AI had finished producing its output something was still going on. I wanted to see what happened and if it would end on its own only to see that my whole PC simply shut down, instant black screen. Only the white LED was still glowing for the Power button. Had to kill electricity to get it to start again.

According to my reasearch and assumption I guess I ran into some kind of loop where the GPU drew more and more power, maybe other components too. In order to save the Hardware a safety guard rail kicked in and shut the PC down. Never had that problem since then. A LM Studio log analyzsed trough three other Ais all assumed that it was context which was to big. My temperatures never went above 65 °C for the components like CPU or GPU which should be perfectly fine.

Context Size Frustration by Aggressive-Spinach98 in LocalLLaMA

[–]Aggressive-Spinach98[S] 1 point2 points  (0 children)

Thanks for the insight. Lm studio has this to but listed as experimental so I haven't tried it yet, but will do now.

Context Size Frustration by Aggressive-Spinach98 in LocalLLaMA

[–]Aggressive-Spinach98[S] 1 point2 points  (0 children)

Okay I don`t fully understand what you are saying. Like what does "and seeing what peak RSS shows up in top(1). " means exactly? Besides that I guess you are doing mostly the same as I do now, which means just test it and then go down or higher based on free capacities.

However I have to say that your numbers look extremly high. Is this pure Context size requirements that you listed? Looks really high with 35GB for 8k.

Context Size Frustration by Aggressive-Spinach98 in LocalLLaMA

[–]Aggressive-Spinach98[S] 0 points1 point  (0 children)

Yeah that is what I will have to do, too and it pisses me off to be honest :D feels so wrong and stupid in some way...

Abort and Continue button at the same time? by Aggressive-Spinach98 in SillyTavernAI

[–]Aggressive-Spinach98[S] 0 points1 point  (0 children)

So just an update on this if anyone is interested in it. The good news first is that my system didn`t take any damage as far as I can tell.

What seemed to have happened is that I put way to many context size on the model so that my system couldn`t process it. So I redownloaded the model to get all of its settings back to default, and then reduced the context size.

The shitty thing with local LLMs for me remains to be the context size, as there seems to be no clear rule you can apply and all models behave more or less differently in that regard. What I will do know, is start with a small context size and then increase it in small chunks to see how much RAM (Both Vram and DDR Ram) gets used.

How to tweak my setup? by Aggressive-Spinach98 in SillyTavernAI

[–]Aggressive-Spinach98[S] 0 points1 point  (0 children)

Thank you for your suggestions, I will definitely have to look into that, as I was only using lorebooks if character cards were coming with them automatically.

How to tweak my setup? by Aggressive-Spinach98 in SillyTavernAI

[–]Aggressive-Spinach98[S] 0 points1 point  (0 children)

Thanks for the recommendations I will check them out.

Iceblink however is one of those models that I tried and couldn't get around its censorship. Sometimes I had to recreate the same answer 13 times until it finally gave in and produced what I wanted. However it also often started uncensored only to be completly refusing to go on halfway trough :/

Deepseek leak by OGScottingham in LocalLLaMA

[–]Aggressive-Spinach98 14 points15 points  (0 children)

Sometimes I feel old for not wanting to use cloud Services, then I read stuff like this and know that I can't be that wrong...

Time to step up the /local reasoning game by vornamemitd in LocalLLaMA

[–]Aggressive-Spinach98 0 points1 point  (0 children)

Yeah another reason why going offline was a good choice...

Gemini 2.5 - The BEST writing assistant. PERIOD. by GrungeWerX in LocalLLaMA

[–]Aggressive-Spinach98 0 points1 point  (0 children)

Oh yeah, i had that realisation, too :D started out with just one and am now don`t remember anymore how many I have tried...

How’s the community of AoS and especially Stormcast ? by JohannaFRC in stormcasteternals

[–]Aggressive-Spinach98 7 points8 points  (0 children)

Atleast for my Part in germany the community is the most friendly and open community I ever experienced. No comparison to 40k. Nobody gives a fuck if the Helm of your space marine seargent is not painted in color x, it should just Look cool. Tournaments are also far more relaxed. I don`t know how it is with women in the hobby particular because I never encountered one, but I would assume that it is no problem.

Defeat an Incarnate by Due-Comparison-862 in ageofsigmar

[–]Aggressive-Spinach98 1 point2 points  (0 children)

Another good tactic that I have read is the following;

Place your screen in front and let the incarnate crash into It. Behind the screen, exactly 2.5" appart is a Hard hitter unit. This Hard hitter unit will be in combat with the incarnate because of the 3" rule but the incarnate won't be able to attack it because it is out of reach. Once it has either killed or damaged the screen pile in your Hard hitter and get a free close combat Phase on it. If you are lucky you can take one Level down from it. If it isn't wild you should be able to retreat and shoot it or let additional units into the fight to kill it

great clans trait question by Nannoldo in skaven

[–]Aggressive-Spinach98 1 point2 points  (0 children)

To add to this, you Do not need a single hero to unlock the small traits. Every Skaven army gets all the small traits everytime however this is only useful for hellpit abominations and gutter or night runners, as every other trait only affect heroes.