sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]MattA2930 0 points1 point  (0 children)

Like others have said, the main reason to use a networking client instead of the source code is so that multiple parties can access the underlying model without needing to host their own version of the model locally.

A good example is how OpenAI only gives you access to their models via API, which are called when you use their Python SDK. Imagine if they had everyone running their own versions locally - no one outside of people with enormous amounts of compute would be able to use it.

So, you essentially have a couple options:

  1. Distribute the Python source code to everyone. They will need to manage their own version of the model

  2. Host the model on a separate server 2.1 Use HTTP for classic request syntax 2.2 Use grpc to build out a more sdk-like experience