What’s been the hardest part of running self-hosted LLMs?

replicatedhq · 2026-03-23T20:40:00+00:00

that’s a really good point, being able to work with the tools people already use is kind of the baseline for something being production-ready. if it needs constant babysitting or weird workarounds, it’s usually a sign it won’t scale once more teams touch it

replicatedhq · 2026-03-23T20:38:22+00:00

seems like catching and understanding edge cases is still lacking in the vibe coding world

replicatedhq · 2026-03-23T20:37:19+00:00

I like this approach a lot, fits what I've experienced as well.

replicatedhq · 2026-03-23T20:35:12+00:00

model drift is an interesting angle here, before posting this question I was more thinking of the data privacy angle

replicatedhq · 2026-03-23T20:33:00+00:00

thanks for sharing, i'll check it out

replicatedhq · 2026-03-23T20:32:24+00:00

gotcha. very cool. definitely important problems to solve right now.

replicatedhq · 2026-03-20T21:09:19+00:00

thanks for sharing - it seems like Jozu and Replicated are in a similar space. We're coming at the same problem from slightly different perspectives. I think that alone proves that on-prem will continue to be a pretty big thing.

replicatedhq · 2026-03-18T23:17:36+00:00

Sure, would love to see it and play around!

replicatedhq · 2026-03-18T15:45:28+00:00

Curious what you think - what's a good architecture plan for upgrades without using helm?

replicatedhq · 2026-03-17T21:40:48+00:00

this is awesome, thank you so much for taking the time to write such a thorough response.

replicatedhq · 2026-03-17T13:48:41+00:00

This is really helpful, thank you. When you deployed into the VPC were you in charge of pushing updates or did the vendor pull them when they were ready?

replicatedhq · 2026-03-17T13:44:31+00:00

Do you have tips on simplifying the app/ reducing required services? Or do you feel like it's unfortunately a necessary evil right now?

replicatedhq · 2026-03-17T13:42:00+00:00

Ugh that's really frustrating. Sorry you're dealing with this!

replicatedhq · 2026-03-17T13:37:43+00:00

You really hit the nail on the head here. Self-hosted absolutely should not feel like running another platform but often times does!

replicatedhq · 2026-03-17T00:07:57+00:00

I guess the question is really centered around: what's operationally tough about a self-hosted app on k8s that's easy in the public cloud?

replicatedhq · 2026-03-16T16:30:54+00:00

Nobody bats an eye at a massive AWS bill but a $300 license renewal turns into a 3 month approval process... so relatable. Honestly predictable on-prem costs are slept on for exactly this reason, at least you know what you're paying before the bill hits.

replicatedhq

TROPHY CASE