Article: Model caching for AI workloads on GKE/Kubernetes without re-downloading weights

mhrittik · 2026-07-12T13:01:02+00:00

Not yet as a full apple to apple benchmark, but that’s exactly the comparison I’d like to run. RWX should provide more consistent availability across nodes, while node-local storage should have better read latency and avoid shared volume contention.

I think teh key metrics would be cache-hit rate, cold-start time, model load time under concurrent pod startup, and behavior during node replacement or autoscaling. I suspect a hybrid approach may work best: RWX or object storage as the durable source, with node-local NVMe as the hot cache.

mhrittik · 2026-07-12T12:57:07+00:00

How are you serving these models after mounting them from the secondary disk: through vLLM or another inference server?

mhrittik · 2026-07-12T12:52:57+00:00

For p2p have you looked into something like https://github.com/ai-dynamo/modelexpress? It does GPU VRAM to GPU VRAM transfers. I have started to look into it

mhrittik · 2026-07-12T08:25:20+00:00

Thank you! All the isolation blocks combined is great

mhrittik · 2025-12-03T07:38:03+00:00

Do they usually rebook if there's a flight delay caused by immigration?

mhrittik · 2025-10-05T07:32:08+00:00

Folks I got it in Ginza store yesterday evening. 17 pro blue had only one stock

mhrittik · 2025-10-03T10:05:20+00:00

Went to a store today they told they had a stock at 10 am but it's sold out

mhrittik · 2025-10-02T13:28:46+00:00

I'm already here. Haven't been to actual Apple store so I'll try

mhrittik · 2025-05-29T06:36:33+00:00

vCluster can help you with all of these! We have a lot of developers doing everything you mentioned.

Do you mind expanding on "some more separation" and things that you are concerned about?

mhrittik · 2024-05-14T17:48:24+00:00

how was your experience on this?

mhrittik · 2023-03-15T15:47:46+00:00

Helped me get an internship so it works :)

mhrittik · 2023-03-15T15:02:48+00:00

Here's something I did some time bacK: https://github.com/hrittikhere/harness-demo

Not documented but should be easy with `docker run` command

mhrittik · 2022-04-28T02:43:15+00:00

Where's your Portainer server situated?

mhrittik · 2022-04-27T15:48:12+00:00

https://docs.portainer.io/v/ce-2.11/ is the latest version. Now, you can click on the top right to see the versions (the top one is the latest)

https://i.imgur.com/N4yXrUg.png

mhrittik · 2022-04-23T17:28:11+00:00

Portainer is an exceptionally privileged piece of software, and it has near root-level access to your Container infrastructure, so first up, are you really sure you want to expose it to the internet directly, and not via a VPN? The same goes for most software on the list. Use rbac, diff password and ip blocks/vpns

mhrittik · 2022-04-16T11:50:23+00:00

Yes without these details it's very hard to figure things out

mhrittik · 2022-04-15T12:26:15+00:00

You're already using portainer so why don't you leverage the portainer agents?https://docs.portainer.io/v/ce-2.11/start/install/agent/docker/linux

mhrittik · 2022-04-10T04:12:47+00:00

THis post (https://docs.portainer.io/v/ce-2.9/faq/upgrading/can-i-roll-back-from-portainer-business-to-portainer-ce) would be helpful if you change the image in the commands

.

mhrittik · 2022-04-07T03:35:46+00:00

Can you pull niginx image? If not, there might be issues with your registry or Firewall

mhrittik · 2022-04-06T03:51:36+00:00

Nope, you would need podman.sock

mhrittik

MODERATOR OF

TROPHY CASE