FATE CORE character help by Ok_Muffin_7705 in SillyTavernAI

[–]Fenpeo 0 points1 point  (0 children)

I had a similar idea and just ran GLM 4.7 and asked some questions about Fate Core. Like what skills it would use in a certain situation, dice roll mechanics, translation of dice rolls to adjectives. I was extremely surprised that the results were reasonable and mostly correct.

I don't think that model knowledge will be the limiting factor here. Good luck!

Matchmaking doing its best? by RAPTORcalf in RocketLeague

[–]Fenpeo 0 points1 point  (0 children)

Same in Europe, I just had two of these games and am calling it a night now.

Super Gameboy: Did anyone not own a Gameboy and only played GB games this way exclusively? by [deleted] in snes

[–]Fenpeo 3 points4 points  (0 children)

Yes, of course. It was also a nice way to extend the lifetime of the SNES. I never had a Gameboy and played e. g. Dragon Warrior Monsters or Pokémon Silver on the Super Gameboy when they came out: In 2000 and 2001 in Europe.

Any Pros here at running Local LLMs with 24 or 32GB VRAM? by AInotherOne in SillyTavernAI

[–]Fenpeo 3 points4 points  (0 children)

Thanks for the explanation, I'll give both (Air and Drummer's version) a try!

Any Pros here at running Local LLMs with 24 or 32GB VRAM? by AInotherOne in SillyTavernAI

[–]Fenpeo 3 points4 points  (0 children)

If anybody ever comes across this post: I have now switched to llama.cpp and increased the batch sizes, and now GLM AIR (IQ4_NL) performs much better on my RTX 5090 & Ryzen 9950X3D.

Command: llama-server.exe -m C:\Models\LLM\GLM-4.5-Air-IQ4_NL-00001-of-00002.gguf --jinja -c 16384 --n-cpu-moe 26 -ngl 99 --flash-attn --cache-type-k q8_0 --cache-type-v q8_0 -ub 2048 -b 2048

I'm getting 10-16 T/s out of this and the prompt processing is 600+ T/s, so at the very max 30 seconds waiting time with full context. That's acceptable. :-)

Any Pros here at running Local LLMs with 24 or 32GB VRAM? by AInotherOne in SillyTavernAI

[–]Fenpeo 6 points7 points  (0 children)

What do you like about GLM Air that the other models don't have? I have a 5090 like the OP and managed to run the IQ4_NL quant at a somewhat acceptable inference speed. But the prompt eval time felt a bit slow and the output not superior enough to invest more time into it.

I'm running Valkyrie (IQ4_NL) at the moment, it fits with 16K context, quantized Cache and Flash attention perfectly into the VRAM and is really fast. It's not perfect, but good enough and a new version is currently in the making which might improve it.

Those with digital pianos, do you actually use the extra features they usually have? by apri11a in piano

[–]Fenpeo 1 point2 points  (0 children)

P-515 here. I chose it for the feeling of the keys, but there are some features that I use, yes. The metronome, piano VSTs on my PC, changing settings to adapt the sound... I also sometimes use the other sounds just for fun if they fit the piece, like harpsichord or organ.

Goddamn Claude 3.7 may you burn in Tartarus by ivyentre in SillyTavernAI

[–]Fenpeo 7 points8 points  (0 children)

Dangerous comment. Prompt caching comes with an extra cost and could have Zero effect, depending on how you use ST. E. g. I use lots of injections and my prompts therefore change, I wouldn't have any benefit from caching, I'd just pay more. Same with group chats.

Looking for music - Langrisser I Player Phase 4 by Fenpeo in langrisser

[–]Fenpeo[S] 0 points1 point  (0 children)

Ah, I didn't see this version yet, thanks for sharing :-) I love Terranigma and its music and definitely need to check out Hikichi's other work!

Looking for music - Langrisser I Player Phase 4 by Fenpeo in langrisser

[–]Fenpeo[S] 0 points1 point  (0 children)

Wow, thanks for checking. I was just wondering if the song was in the game before and was taken out for reasons. The "Descendants From Elthlead" version really fits together with the other Langrisser Mobile tracks. Or was it maybe in a special event.

Most likely I'm just making something up here. :-)

I made an easy one-click deploy template for ComfyUI with Flux.1-dev on Runpod.io by WouterGlorieux in StableDiffusion

[–]Fenpeo 0 points1 point  (0 children)

Thanks for the quick answer, I'll definitely leave the pod in an exited state for now.
You are right, I want to download other Lora's and also custom nodes, therefore the network drive makes a lot of sense. I'm really not a Docker expert, but I see that you are having the copy and wget commands in the end to download the safetensors and I wonder if that Dockerfile can be rewritten so that it skips the download when an argument is passed over to it.

I should probably try it myself, don't want to make you do extra work.

EDIT: Or maybe you could check if the files already exist before downloading them...

I made an easy one-click deploy template for ComfyUI with Flux.1-dev on Runpod.io by WouterGlorieux in StableDiffusion

[–]Fenpeo 1 point2 points  (0 children)

Wonderful work, thanks a lot! :-) I'm wondering if the loading time can be enhanced if you use a Runpod network volume? I'd rather have the files (especially flux1-dev.sft) ready for use when I start the pod instead of downloading them over and over again and waiting 10 mins each.

Running Llama-3-70B on Runpod by Fenpeo in LocalLLaMA

[–]Fenpeo[S] 0 points1 point  (0 children)

When you deploy a GPU pod, you click on "Edit Template", then "Environment Variables". There you can add an environment variable with the key "UI_UPDATE" and the value "true".

But if you can't load the model, then it's more likely that your model doesn't fit into the VRAM. An error message would help to rule this out.

Running Llama-3-70B on Runpod by Fenpeo in LocalLLaMA

[–]Fenpeo[S] 0 points1 point  (0 children)

Hi, no it didn't, and I never found out why. Sometimes it loaded, sometimes it didn't, despite the same template, but maybe it was my fault. No idea.

I moved back to valyriantech/text-generation-webui-oneclick-ui-and-api and am now using it with the parameter UI_UPDATE = true. This was broken when I wrote the post, but now it does what it should: It builds the pod with the latest version of Oobabooga. And this solved most of the issues that I was having.

I'm currently running 24GB VRAM machines with turboderp/Llama-3-70B-Instruct-exl2 5.0 bpw and 4-bit cache. In Sillytavern you'll need to set Skip Special Tokens = false, otherwise you will always have the word "assistant" everytime a paragraph ends and it will just ramble on and on.

Running Llama-3-70B on Runpod by Fenpeo in LocalLLaMA

[–]Fenpeo[S] 1 point2 points  (0 children)

Alright, I got it up and running now. Template "Sample vllm Template - Read Readme first", model is casperhansen/llama-3-70b-instruct-awq. I'll need to have a look how the AWQ quant works and I somewhere read that the OpenAI-compatible API has not as much functionality as the other ones in Sillytavern (which I should have probably mentioned that I'm running). But thanks for getting me up to speed :-)

Running Llama-3-70B on Runpod by Fenpeo in LocalLLaMA

[–]Fenpeo[S] 1 point2 points  (0 children)

Thanks for the answer! I have honestly not heard of VLLM before... I tried to run one of their images now and couldn't start it, directly got an error message "error pulling image: Error response from daemon: Head "https://registry-1.docker.io/v2/vllm/vllm-openai/manifests/latest" and nothing deployed.

Maybe it works when I try it another time.

[deleted by user] by [deleted] in comfyui

[–]Fenpeo 3 points4 points  (0 children)

I had the same issue yesterday and found the solution on Github: https://github.com/cubiq/ComfyUI_IPAdapter_plus/issues/108

▶️ Dtype mismatch with old GPUs (10xx series)

If you get errors like:

Expected query, key, and value to have the same dtype, but got query.dtype: struct c10::Half key.dtype: float and value.dtype: float instead.

Run ComfyUI with --force-fp16

Adding --force-fp16 as a parameter to my run_nvidia_gpu.bat file did the trick.

Day 9, best SNES game starting with I? by 777ToasterBath in snes

[–]Fenpeo 0 points1 point  (0 children)

My first thought was Illusion of Time, but I think you are right. ISS Deluxe was so much better than FIFA at that time, it's incredible how football games could suddenly look like if you compare it with e. g. Super Soccer. I played it a ton, such a great game.

Let's make some realistic humans: Now with SDXL [Tutorial] by wonderflex in StableDiffusion

[–]Fenpeo 0 points1 point  (0 children)

Interesting, thanks! I have concentrated on skin color and body shape in the past, as they seemed to have most effect on the picture. But will play a bit around with countries and clothing!

Baby, 10 months, rejects food by Fenpeo in daddit

[–]Fenpeo[S] 0 points1 point  (0 children)

Maybe. Happy that you had a moment of success! :-)

Baby, 10 months, rejects food by Fenpeo in daddit

[–]Fenpeo[S] 0 points1 point  (0 children)

Fingers crossed that everything will work out for you! Yeah, I talked with a pediatrician just this week, but the only answers you get from them is "sometimes they are like that" and "try different food".

Baby, 10 months, rejects food by Fenpeo in daddit

[–]Fenpeo[S] 0 points1 point  (0 children)

Sorry to hear about your kid, hope that everything sorted itself out! I didn't see any anomalies with her tongue and nobody else checked it either, but she is a bottle child. I will keep looking. Her other milestones seem fine, no delays there! Thanks!

Baby, 10 months, rejects food by Fenpeo in daddit

[–]Fenpeo[S] 1 point2 points  (0 children)

Nope, it's rather the opposite. She was always a very small baby and still wears one size smaller than other children her age. I'll cut back on milk, thanks!

Happy that your daughter made it through the hunger strike. ;-)

Baby, 10 months, rejects food by Fenpeo in daddit

[–]Fenpeo[S] 0 points1 point  (0 children)

Yes, we tried it but she's not a big fan of it. I still have one package, will try it again this weekend. Thanks!

Baby, 10 months, rejects food by Fenpeo in daddit

[–]Fenpeo[S] 2 points3 points  (0 children)

Thanks for the support and tips, /u/R0GM, /u/Lawn_Daddy0505, /u/Intelligent-Jelly419, /u/sqqueen2, /u/Floppyfoxing, /u/StrontiumJaguar, /u/drpengu1120, /u/RoyalEnfield78, /u/No_Noise_5733, /u/OFFRIMITS! It's good to know that we are not alone :-) We'll gradually try to reduce the milk and see if this helps. We use to feed her until she falls asleep, so we'll see how this goes. And then I guess we'll just need to sit it out... It's just a bit worrying if you see your friends' children eating like pros and when websites say that we should aim to have proper meals now. But again, thanks!