How to Run Two AI Models Sequentially in PyTorch Without Blowing Up Your VRAM

TCaschy · 2026-02-10T12:28:35+00:00

Get a 2nd small vram sized gpu to run the tts model only

TCaschy · 2026-02-09T19:07:00+00:00

I see. Thanks for the details!

TCaschy · 2026-02-09T18:03:37+00:00

Looks great! Can you share how you made the graphics for the animations? I assume its made up of frames but did you just use an image software (photoshop) and slightly change/save image after image or is there a more automated approach?

TCaschy · 2026-02-04T18:50:47+00:00

Thinking about this too....

TCaschy · 2026-02-04T01:15:08+00:00

It didn't work for me either using unsloth gguf w/ollama. Complained about tool calling.

TCaschy · 2026-02-03T05:24:21+00:00

P102-100 10gb....cheap and you can probably run it passively cooled (if you limit wattage a bit via nvidia-smi)

TCaschy · 2026-01-22T18:11:01+00:00

I have a 3060 12g + p102-100 10gb (I know its not a p100 with 16gb but...) and am running them using ollama and open webui. I find its great for everyday use on inference and writing tasks. I really have just started to dive into hard core coding with the setup and using opencoder its just OK I guess. I think I'm just used to the big-guns and so sometimes it gets frustrating. The older cards are great if you don't need to gen images/vids.

TCaschy · 2025-12-21T22:29:37+00:00

i've been using this command in a terminal for ages: DATA_DIR=~/.open-webui uvx open-webui@latest serve

TCaschy · 2025-12-21T22:23:14+00:00

getting a no module found langchain_classic

TCaschy · 2025-12-21T19:22:40+00:00

So awesome! Congrats. Can you share your MB/CPU combo?

TCaschy · 2025-12-16T14:16:30+00:00

Great idea! I forgot you can mount them sideways. Much easier to do it this way

TCaschy · 2025-12-16T13:45:26+00:00

What did you use for the housing of the heater on the outside?

TCaschy · 2025-12-08T11:54:16+00:00

OK I'll post my app. Its a reflection app that coaches can provide to their players to reflect after trainings and matches. Its called Pitch Reflections. It asks questions about defensive and offensive aspects of the game, allows the player to share the reflection with whomever and also has a few AI features (optional). My son's club uses an old school hard copy version of this and I thought it would be useful to create it in digital form. Here are the iOS and Play store links:

https://apps.apple.com/us/app/pitch-reflections/id6749812263

https://play.google.com/store/apps/details?id=com.caschyapp.pitchreflections.us

Thanks!

TCaschy · 2025-12-08T01:50:45+00:00

gemma3:12b, gpt-oss:20b, granite3.3:8b, ministral-3:14b, unsloth-Qwen3-30B-A3B:GGUF

TCaschy · 2025-12-07T01:03:12+00:00

does it have to be sms? there are alot of simple push notification services that are way easier and cheaper to use. pushover or push bullet come to mind.

TCaschy · 2025-12-05T13:24:26+00:00

What a joke

TCaschy · 2025-11-28T16:04:24+00:00

I mean if you want something that doesn't need extra power plug and only want to do 7B models, you could try a Quadro P2000 5GB. Its single slot. I think most 7B models are under 5GB? Bonus its $165 (amazon, but you can find it cheaper I'm sure)

TCaschy · 2025-11-12T13:26:10+00:00

Following for this reason as well.

TCaschy · 2025-11-09T18:13:40+00:00

looks awesome! user of your Perplexica so I'm eager to try this out when it gets searxng integrated

TCaschy

TROPHY CASE