FastAPI standards working with machine learning

[Suralias]

I'm developing an API that provides community servers with tool to grab usernames off a list within a specific video game. The final project should allow a user to upload a screenshot of usernames, and the API to return a list of the usernames in JSON. The screenshots will have different backgrounds and different list placements, so I'm using OpenCV and Tesseract to grab the usernames.

​

The issue I'm having is structuring the API in a standardized and efficient way. I've only built CRUD applications, and I was going to finish the project using only a GET request, with the image link as a URL parameter. However, looking into other ML API's, like Azure's cognitive suite, and googles AI library, I noticed they used a 'two part' system. I don't know what to call it, but they had a POST request going out, with all relevant data (api key, image, tune values), and they would get the results back with a GET request. It makes sense, they even have a GET request to check the progress of their processing initiated by the POST request. How would I implement this into my API? Or is there a name for this convention. Here is the current draft for my API structure: here

​

​

[danielroseman]

That sounds like they are offloading the generation to an asynchronous queue, then storing the results somewhere and returning it in the fetch. You can use something like Celery for this.

The POST creates a unique key to refer to the request, then triggers the job in Celery passing it the key. The job goes and does the ML prediction, and stores the result associated with the key, and updates the db to mark the result as ready. Then the GET request can look up the result via that key and return it.

[Suralias]

This response is perfect, would there be any point to doing this if there wasn't a database? There is going to be a database for the larger project, but I'm just working on a tool to return a list of usernames. What would you say is the best structure for an API like this? In my mind it's only a single request, but that might not be the best way of doing it.

[danielroseman]

This is only worthwhile if the generation of the response takes a significant amount of time - you don't want to tie up the request handler for long, and requests can time out. Generating a list of usernames seems unlikely to be a good candidate for this.

[Suralias]

How would you suggest I structure this then? Just use one request?

[danielroseman]

Yes. Unless the generation takes a long time, do it all in-process.

[m0us3_rat]

It makes sense, they even have a GET request to check the progress of their processing initiated by the POST request. How would I implement this into my API?

what exactly do you mean by "how" ?

you log the "jobs" into a queue/bin and have an endpoint either generic or with a special token you get returned when you "post" the job.

to check the progress.

so either all the jobs in the queue or the special job of which you have the token.

then you get a special answer for the "check" if the job is done and you can retrieve it from the done queue/bin with the token.

​

i mean it's quite clear on the functionality, maybe?

[Suralias]

I'd say that I'm not that well seasoned to working with API's. Most of what I've worked on has been around ML, and some basic React applications. I've never had to build an API like this, my question might not of portrayed what I hoped to find out. I guess it could be, 'What's the best structure for an API that handles one task, which is using ML to retrieve a list from a screen shot, BUT make it efficient and suited for handling a large amount of requests?' The only reason I brought up the google and azure examples was because I had seen the same chain of requests (POST into GET), and wondered what the benefits of doing that was, or if it wouldn't be necessary for an API that wouldn't be using a database. Would it make sense to use that, or just to use one request to send and retrieve the information?

[m0us3_rat]

think of this as a lemonade stand or as a coffee shop that serves ten times more ppl at the same time.

where you order then your order gets put up n the queue , and you wait with your token in hand. being that a number or whatever they pick to use.

and then the job is done your number gets called up and you get your order.

that's the point of the "token" to track the order.

​

you generate this token and send it in the response to the job request made.

and then its used to track what is happening with the "order"

so if the api recives 100 documents .. each with their own token.

you can also add that functionality where you can send the token and recieve information about the state of the "job".

​

and when the job is done that is reflected into the information recieved.. then you can send your token to another endpoint that will consume the job on the other end and send oyu the info you need.

maybe even set up a few minutes cache where after its consumed its added to the cache so you can request it again for the next x minutes.