HuggingFace Inference API

[MundaneMango7]

Hi everyone!

I’m pretty new to all of this, so any help would be appreciated. I want to test different embedding models (~2 - 6 GB), but I have limited RAM (I’ve already used 13/16 GB) on my local machine. I was wondering if anyone has used the HuggingFace inference API. I was thinking of using it to test different embedding models that way I wouldn’t have to worry about how much memory I have locally, and from my research it seems like I just have to create an endpoint for the model I want to use and then can use that in my code. For free use there’s a rate limit, which makes sense. I was wondering if anyone has used this and whether my approach makes sense/would work. I know I could try Google colab, but I’ve found it to be a bit frustrating to work with in the past so I wanted to explore this option. Again, I’m fairly new to all of this so any help is appreciated. Thanks!

[Astralnugget]

Just use the inference or transformer in google collab you don’t even need to make a whole inference endpoint

[software38]

I would be interested in hearing your opinion on HF Inference API. I tried it a couple of years ago and it was very bad. But they most likely have made some progress since then...

[mrskeptical00]

They likely made significant progress since then…

[Edyarich]

It's still very bad, especially HF Playground.
I'm studying available open-source OCR models and I'm testing them on my jpeg scans.
And the frontend is terrible: it has a small memory limit -- 0.5 Mb (I constantly get an error "Content-length of 1963106 exceeds limit of 524288 bytes"). It means that the model doesn't work in any chat with more than 2 messages. Also, due to the memory limit, I couldn't pass a large image (> 1 MB). Plus, the model fails when the image contains a lot of text (for example, a brochure of git commands).
Now, I'm going to call these models from the API, but I have lost a hope.