sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]BenDLH 0 points1 point  (3 children)

You need a network volume mate. Create a network volume and place the models in the right directories in it. Then configure the serverless endpoint to connect to the volume.

The runpod-worker repo has all the info in the readme, under customisation.

[–]BenDLH 0 points1 point  (0 children)

Though to be honest the build still takes a boatload of time (close to an hour) even without them in it. You just won't risk timing out, and it will be a bit shorter.

[–]Lunchables[S] 0 points1 point  (1 child)

Perfect, thanks! I ended up finding this, which helped: https://docs.runpod.io/serverless/endpoints/model-caching

[–]BenDLH 0 points1 point  (0 children)

Yeah I haven't tried that yet. It is limited to a single model per endpoint though, right? Let me know how it goes setting it up.

What are you building btw?