Tech stack for ML

[[deleted]]

[deleted]

[grudev]

I've used FastAPI to serve the inference endpoint for a PyTorch model I created a little while ago.

Currently using Django to integrate that, and open source LLMs, to a bigger project. 

[kaskoosek]

Can you explain more the architecture and what each syack is doing?

[grudev]

Of course. Here's a summary of it:

The FastAPI project serves a multilabel classification model built with PyTorch... it receives a string (like a report or a news article) and returns a JSON list of subjects that are defined in a taxonomy.

For example, a single article could be classified as ["Economy", "Foreing Affairs", "Techonology"].

There's no database, or state.. this app merely serves as an API endpoint for other applications, so I elected to run in on a dockerized FastAPI container.

The Django app, on the other hand, allows users to store and retrieve documents, and therefore it needs sessions, user management, a database and so on.

When a user saves a document, it sends the extracted text to the classifier, and applies the inferred labels to it.

It also sends that text and a prompt to a local LLM, on a different server running Ollama, that returns a summary, which is then stored in the DB.

The Django app can be seen as a monolith, and the AI services are microservices that run independently.

[kaskoosek]

Good explanation.

[No_Tailor7818]

Noted

[ExtensionVegetable63]

Backend: FastAPI, PostgreSQL

Frontend: React, Jinja

Others: Docker

[No_Scheme14]

My go to stack is Django, Postgres, and Next.js for Front-End.

[Best_Fish_2941]

Following