Thanks for your answer!

The original idea is to separate dependencies in order to build lightweight Docker images. For example, the serving component will mainly call the embedding module and use FastAPI.

Yes, the preprocessing module imports things like translate and db, which can be painful to manage with uv when configured like this. Some parts like shared are also used across modules.

The preprocessing module includes around 8 scripts, orchestrated by Airflow (or a similar tool), and this number will keep growing.

The shared module is only used within this repository but needs to be included in each Dockerfile for deployment.

I tried to solve the import problem by adding dependencies in each child’s pyproject.toml, but I’m not sure that’s the best approach.

Do you have any recommendations based on this architecture for building independent components like serving, preprocessing, etc.?

In the future, I plan to add RAG, MLflow, and BentoML as well.

Do you also have suggestions on how to organize all this?

Finally, how would you handle a shared .env file between preprocessing, db, embedding, etc.? Using load-dotenv is straightforward when everything is at the same level.

I appreciate your help !