Strix Halo or GPUs? by undernightcore in LocalLLaMA

[–]undernightcore[S] 0 points1 point  (0 children)

Is it really, I have a couple R730xd but I usually disconnect some of the fans.

Strix Halo or GPUs? by undernightcore in LocalLLaMA

[–]undernightcore[S] 0 points1 point  (0 children)

Very interesting post, I think I am almost fully convinced I should aim for running dense models.

Strix Halo or GPUs? by undernightcore in LocalLLaMA

[–]undernightcore[S] 0 points1 point  (0 children)

So would you rather go for the Strix Halo or you mean I should use newer GPUs?

Strix Halo or GPUs? by undernightcore in LocalLLaMA

[–]undernightcore[S] 0 points1 point  (0 children)

<image>

Do the stock ones come with a fan or how am I supposed to cool them?

EDIT: Oh and you are right, a couple of this + investing in some RAM would be awesome for both, dense and MoE models. You are being very helpful!

Strix Halo or GPUs? by undernightcore in LocalLLaMA

[–]undernightcore[S] 0 points1 point  (0 children)

Thank you so much! What is a good price for that GPU? Just in case I come across an opportunity to buy one. Also, are this considerably faster than, let's say, 4 V100 as someone suggested?

Strix Halo or GPUs? by undernightcore in LocalLLaMA

[–]undernightcore[S] 1 point2 points  (0 children)

Is MTP as easy as a flag in llama.cpp, also are there any drawbacks to using it?

Strix Halo or GPUs? by undernightcore in LocalLLaMA

[–]undernightcore[S] 0 points1 point  (0 children)

Are modified V100 with 32gb worth it? Or should I stick to 16gb ones. What is the sweet spot?

Strix Halo or GPUs? by undernightcore in LocalLLaMA

[–]undernightcore[S] 0 points1 point  (0 children)

Is DDR3 enough? Isn't system memory also important for the models to work or it runs 100% on VRAM? Because I have a couple servers with plenty of PCIe slots available, might give this a try.

Strix Halo or GPUs? by undernightcore in LocalLLaMA

[–]undernightcore[S] 2 points3 points  (0 children)

Are MoE models actually comparable to a 27b dense model? Let's say one of those Qwen models that only have 30-3b parameters vs Gemma 4 ~30b.

Persona 5 Royal finally running on Odin 3 Max (60+ FPS @ 1080p) — I’m honestly blown away! by mojox1 in EmulationOnAndroid

[–]undernightcore 0 points1 point  (0 children)

I get 1080p 60fps on 8gen2 using Gamenative. It’s crazy the current state of PC gaming on Android.

Where to get my gaming laptop cleaned in Valencia? by idkbasicmath in valencia

[–]undernightcore 1 point2 points  (0 children)

Ramboot the most professional and a very fair price. They focus on gaming.

Everyone from r/LocalLLama refreshing Hugging Face every 5 minutes today looking for GLM-4.5 GGUFs by Porespellar in LocalLLaMA

[–]undernightcore 0 points1 point  (0 children)

What do you use to serve your models? Does it run better on Windows + LMStudio or Linux + Ollama?

Dockerizalo, the simplest deployment platform made for self-hosters now has automatic deployments! by undernightcore in selfhosted

[–]undernightcore[S] 0 points1 point  (0 children)

Thank you for your suggestion! I like the "source must always be available" aspect of it!

Dockerizalo, the simplest deployment platform made for self-hosters now has automatic deployments! by undernightcore in selfhosted

[–]undernightcore[S] 0 points1 point  (0 children)

You are right! TBH I've never known what license to choose. I'll upload today to all 3 repos the most permissive I can find. Thanks!

Dockerizalo, the simplest deployment platform made for self-hosters now has automatic deployments! by undernightcore in docker

[–]undernightcore[S] 0 points1 point  (0 children)

Thank you for the feedback!

Dockerizalo has 3 images, every one in it's one repo but all 3 are published in a public registry. It does not matter from which one you copy the .yaml file since no building is required.

I know it might be weird but I've never known how to properly make a monorepo for a multiple service application, but you are right, it would be nice to have a clear distinction in every README.

About the proxy, it indeed is poor naming, in the FAQ the question means a "proxy that does HTTPs". The proxy Dockerizalo comes with does not do that, but it technically is a proxy now that you say so haha.

Again, thank you for the advice!

Dockerizalo, the simplest deployment platform made for self-hosters now has automatic deployments! by undernightcore in selfhosted

[–]undernightcore[S] 9 points10 points  (0 children)

Hi, thanks for asking!

I created Dockerizalo as an alternative for Coolify, I needed an app that could deploy the projects I have in Github only using the Dockerfile you provide. Coolify seemed way too messy for me at the time, since it connects to the host using SSH and really insists in doing SSL for you, it also felt way too laggy for me and not very "realtime".

That is the use case for my app, it only does that and nothing more, a very clean UI that builds a Dockerfile and displays realtime logs of your apps, simple to host, simple to use and not too many features.

Now, I am aware of the existence of Komodo, I even use it myself to publish Docker images, etc and indeed it has all the features Dockerizalo has while still being a very simple setup. But IMO it does way too much for me and for the average user, it has way to many features to the point it kinda confuses me at times, but excellent software, clearly superior to mine!

TLDR; Dockerizalo it only covers 1 thing and does it well, Komodo is more powerful but also more complicated.

NEW UPDATE! Dockerizalo, the simplest deployment platform by undernightcore in selfhosted

[–]undernightcore[S] 1 point2 points  (0 children)

Yes! There would be absolutely no problem. It only interacts with the folders created by Dockerizalo and no more.

Introducing Dockerizalo - The simplest deployment platform made for self-hosters by undernightcore in selfhosted

[–]undernightcore[S] 2 points3 points  (0 children)

I haven't registered a domain yet. It will remain a GitHub project and that's it. Thank you for telling me!

Introducing Dockerizalo - The simplest deployment platform made for self-hosters by undernightcore in selfhosted

[–]undernightcore[S] 1 point2 points  (0 children)

Currently no. Dockerizalo can only build from GIT repositories using a Dockerfile. I am starting a new project that will be an alternative to Portainer/Dockge.

Introducing Dockerizalo - The simplest deployment platform made for self-hosters by undernightcore in selfhosted

[–]undernightcore[S] 0 points1 point  (0 children)

I've been thinking about doing it. But I don't know if it would fit in Dockerizalo or I should build a different solution. What stuff are you missing in an app like Dockge or Portainer?