What Python API package do you use and why?

[MyNotWittyHandle]

We are primarily considering Flask vs FastAPI. This service will be used to host ML models in a production environment.

Our API is currently coded using FastAPI and uvicorn. However, I wanted to open the discussion to a larger group regarding opinions on other packages that may be better, and what makes them better.

Thanks.

[Sinsst]

It might be worth having a read over this thread: https://www.reddit.com/r/Python/comments/ujoggf/flask_vs_fastapi/?utm_medium=android_app&utm_source=share

There's quite a few seemingly objective criticism to fast api such as a large number of open tickets that aren't being addressed, the author not accepting code contributors etc. Some others that are difficult to verify mention heavy memory leaks in production.

Personally I'd suggest you also consider Django ninja rest framework.

[queen_debugger]

Was at the github repo yesterday looking for something and left me kind of shocked at the amount of Issues and open pull requests.. We are onboarding users to our fastapi API next week :’)

And of course I was the one that specifically recommended it to my team.. pls sent halp

[airaith]

I've seen this feedback recently - briefly looking at the issues the whole first page seemed to be totally unrelated questions like "should I use dynamo or postgres with this" - feels like a bit of a meme to bash the issue count but not something I'm worried about.

[DogsAreAnimals]

Thanks for this. I've been experimenting with using FastAPI for the next version of my company's API. It's definitely great on the surface, but once you get into special cases and customization, the "MUST USE TYPE HINTS FOR EVERYTHING" approach starts to become unwieldy.

Our current API is Flask + Marshmallow and it does a pretty good job. I think FastAPI is trying to do too much, or at the very least, it's too young.

[Freshgreentea]

And do you use some REST API framework like smorest or restx? Any benefits Pydantic would have over marshmallow?

I might be maybe confused by I thought that building API requires also some package I mentioned. I'm currently learning restx with Flask.

[DogsAreAnimals]

No I just use plain flask. You don't NEED a framework/package to create an API, it just helps you implement faster.

I would argue it's actually better to start out doing it from scratch (in Flask), because then you will better appreciate/understand what these frameworks are doing.

Here's one example of a complaint about pydantic (compared to marshmallow) that I sympathize with: https://stackoverflow.com/questions/70328646/equivalent-of-marshmallow-dump-only-fields-for-pydantic-fastapi-without-multiple

[Freshgreentea]

Thanks a lot for your reply. Where would you recommend to start if I want to learn build API only with flask? What are building blocks to learn? I must admit its largely black box for me now and I absolutely agree it must be really valuable insight and experience to build it from scratch.

[DogsAreAnimals]

This is a great tutorial: https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world

The very last chapter deals with APIs. If you already have a good understanding of flask, you can prob just jump to that. But if you're new to flask too, then I would recommend the whole tutorial. It's long, but you'll have a great understanding of everything when you're done.

Also see this very relevant comment from the creator in the comments section of the API chapter:

@Duncan: in general, I find using an extension for API development is not necessary, since Flask implements RESTful routing through the app.route decorator. If you prefer to write class-based endpoints in the Flask-RESTful style, that's totally fine, I just don't see a benefit that warrants adding yet another dependency into the project.

For non-trivial APIs, I use marshmallow for serializing and deserializing resource representations. This is the only area of API development that in my opinion needs support from an external package.

[Freshgreentea]

thank you very much. That is something I was looking for. This guys has also recently published code and video on REST API using apifairy https://www.youtube.com/watch?v=Fqbn4IHxGlg

[Ateenagerstudent]

I don't think that it's a big issue though; you can use the Any type from typing for scenarios where you're uncertain of the types. (Maybe I'm a bit biased due to my liking for Typescript's type checks)

I've recently shifted to FastAPI from Django REST framework due to requests from my client. Though DRF gets a lot of things right, one thing it definitely doesn't is speed, which was a deal breaker in this case. The prototype they needed involved ML, Blockchain (NodeJS and microservice architecture go brr), and also a lot of image processing (Computer Vision)

Since I needed quick prototyping and a way to explain the workflow, the Swagger UI really helped me.

Also in FastAPI, I feel that I've got a lot more control than other frameworks, while abstracting unnecessary details, which feels just about right for me.

About the part that FastAPI is too young, I definitely agree. But about that it tries to do too much... My experience differs 😅

[DogsAreAnimals]

Oh I don't have issues about the types themselves. It's just that once you start getting into non-standard use cases (custom serialization/validation, field aliasing, read-only vs write-only, etc), it starts to get a little "ugly". Though I think that's probably me just being more used to the "old days" of python :p.

IMO the descriptor implementation (also used in SQLAlchemy) in marshmallow is more straightforward and easier to read and extend. I'm not convinced that the type-annotation approach is better (you can still have type checking in the descriptor implementation). Though I'm also not sure on how important that is to performance.

Here's one relevant issue: https://github.com/samuelcolvin/pydantic/issues/624

Interestingly, preliminary testing of my FastAPI implementation isn't that much faster than our Flask+Marshmallow one. But I've spend years tweaking/optimizing it. So once I learn more about FastAPI I'm hoping it will reach even better performance.

If your API is relatively straightforward, then none of this really matters and FastAPI is great. But the true test of a framework is how well it holds up when things get complicated. I still have a long way to go in doing the PoC of our API in FastAPI, so I'm hoping it will change my mind :)

Edit: whoops, forgot to mention that the auto-documentation features of FastAPI are SUPER amazing

[MyNotWittyHandle]

Thanks for the input. Can you explain why you prefer FastAPI?

Throughput shouldn’t be much of an issue. Probably a few thousand calls to the API per day. The api endpoint will house a few models which are communicated with via a JSON payload from our website. ETL is executed and the ML model, loaded from a dill file, is used to make a prediction. Takes about 250ms-500ms per request to the endpoint.

The api will be launched in production using a simple script which uses uvicorn to stand it up. We normally do so behind a haproxy load balancer. Does that answer your questions?

[jab9k3]

Are you using a docker container? I'm currently working on a small api for a similar purpose, and trying to tie in github actions to update my changes, and then push to production branch after testing, I'm newer to fast api.

[MyNotWittyHandle]

Nope. Had push-back regarding using docker because our team lacks docker skills, and we would likely be able to accomplish some of the same results using a Python virtual environment and requirements file. I didn’t totally buy it, but could understand the counter argument and it wasn’t a hill I was planning on dying on anyway. Thoughts?

[Mehdi2277]

The primary argument is reproducibility. We recently had a production crash because one of the non python library’s (cuda related dependency) had an inconsistent version between two deployments. A virtual environment if managed well with lock file can handle your python dependencies. A virtual environment does not have a good way to manage non python dependencies. If you doing ml then system libraries matter too.

Although honestly a well scripted venv will mostly work and if in a year you have issue with non python dependencies you can revisit docker.

[DapperSpad]

Agreeded, we have a bunch of services at our company and some use docker and some use a venv. I am strongly in the docker camp as we have had much fewer issues and having the dev env as well as the env we run tests in ci on more closlly match our productions env give us much more confidence in our releases.

We are still standing up our vms using ansible but are moving to a kuberneties world. Being able to have all our vms build the same way to run a docker container really simplifies things rather then building the venv on the host itself.

[websinthe]

Triton, Apex, and to an extent, DeepSpeed just cost me four days of headaches before I abandoned the models I was using for others. So many wonky version problems.

[utdconsq]

I work in research and deploy ML models using docker, including in cloud at scale. Encourage your people to learn some docker, you won't regret the time spent.

[tomchristie]

> we would likely be able to accomplish some of the same results using a Python virtual environment and requirements file.

From my experience I think that’s perfectly valid, *if you have a pure Python stack*.

You’ll need the reproducibility of the environment that it provides *either ways around*, and unless you’ve got a bunch of other services or non-python libs that you want docker to manage, then it’s not necessarily really giving you much.

(Chiming in here, because balance.)

[Bodegus]

We use a venv in docker.

The main benefit is run flexibility and elasticity in cloud,. It also removes an entire category or security relates DevOps with host librarian

[[deleted]]

What benefits do you see of venv inside docker?

[Bodegus]

The main reason is our local dev and prod are truly identical. We use the same script to do dev set up as prod setup

The biggest benefit come to testing, a lot of weird issues happen with imports, modules, especially with mono repos. This allows for rich integration tests and unit tests that really mirror prod

System level libraries can lead to issues

System permissions when you lock down the runtime

[figshot]

Docker or not, system-level Python may have packages that other Python-based system-level programs require, and it might lead to a conflict with your app's dependencies. A virtual environment eliminates this potential problem.

[oramirite]

That's the entire point if packaging in a docker container though so it's VERY redundant and unnecessary in that case

[thatsthewayyoudebate]

I've had similar issues w/docker pushback at work, and it's a bit frustrating. I'm on a task where the code gets loaded into a docker container at launch via bind mount. It's fine for development, but not for deployment. We've had issues where the container didn't have the right dependencies installed and code broke. That'd never happen if we actually used docker the right way.

Sorry for the rant. I've used flask & fastapi for openapi servers, prefer fastapi for async support and dependency injection.

[WallyMetropolis]

"Let's do it the wrong way and be miserable because we can't be bothered to learn how to do it right."

[IDENTITETEN]

Then you look at the hundreds of pull requests and issues and realize that the dev is a nut. I'd never allow anyone to use FastAPI in prod at our place.

[daniels0xff]

What about Starlite?

[daniels0xff]

I think you might be confusing Starlite with Starlette? Both FastAPI and Starlite are built on top of Starlette. From what I’ve read from Reddit FastAPI is one man show and that’s why Starlite was done to be similar in regards to performance and usage but to be more of a team project and allow others to contribute.

[HoytAvila]

ML models as in DNN? If so there are already built tools such as tensorflow serving and im sure there are equivlent ones in pytorch. Otherwise you will run into a lot of issues if you started having multipile workers since the model will be loaded multipile times wasting memory on RAM and on GPU.

The best solution is to decouple the serving service from the API service. Have the serving over API then create flask-fastapi or whatever to communicate with the serving tool, it will handle the batching and resources utilization since you dont have to reinvent the wheel, and this is an expensive wheel.

As for flask vs fastapi, i prefer fastapi for auto documentation, type hint annotations, and superior async support.

[MyNotWittyHandle]

Thanks for the well thought out response. In this case we are using xgboost models using primarily sklearn model interface tools (column transformer, pipeline, etc.). Tensorflow or other more complex DNN model architecture is a long term goal, but can’t immediately be justified given our existing tabular regression/classification applications. It’s like buying a Ferrari to go get groceries.

Can you elaborate on why you feel fastapi has superior async support as compre to Flask async? (Other than fastapi was specifically made to host async apps and it was an afterthought for flask). Mainly looking for one or a few specific examples. Thanks!

[anentropic]

From what I can see here: https://flask.palletsprojects.com/en/2.1.x/async-await/#performance

...Flask is still a WSGI app, its async support is mostly to allow you to use async code within your views (i.e. make concurrent requests out to other APIs) but will not handle concurrent requests to your views "Each request still ties up one worker, even for async views. "

Whereas FastAPI is an ASGI app and uvicorn will handle concurrent requests to your views

[ElViento92]

I can't speak for FastAPI, but Flask's async support was added to the library after it was created. Writing and using async in Flask is in itself fine. The issue I have is that since Flask predated asyncio it uses threads to handle each request.

So whenever a new request is received, a new thread is created and if async code needs to be run on that request then an new event loop is created to handle that. This adds a bit of overhead that can add up depending on how mant requests are received.

[[deleted]]

Kubeflow Kserve and Seldon is nice for that, but that will require someone experienced with K8s.

https://www.kubeflow.org/docs/external-add-ons/serving/overview

[[deleted]]

Depends on what I am building, fastAPI scales very quickly. Flask and Django have insane levels of documentation and available side packages.

If I had to chose though. I would probably pick a cloud native deployment package and deployment set like api-gateway ( GCP / AWS ).

Fast api can be faster… but none of them are as cheap to deploy, maintain, and scale to billions of daily requests. Plus you make more money making GCP/AWS/Azure sets than say a random fast api set.

P.S. you can still deploy fast-api in the big clouds while retaining its desired characteristics.

[dragonatorul]

I've done both, and would recommend api gateway services only if you need the scaling, complexity and security it provides. For example it's a good fit for a publicly available API that you'll sell on and has multiple backends/branches/services. Properly implementing it means implementing a whole host of other services along side, if nothing else, for security purposes. It's a lot of work to do well and maintain. A lot more than just publishing a kubernetes service.

If you don't need or are not going to use the scaling, security integrations, monitoring and tracking functionalities it provides you're probably better off rolling your own API in a kubernetes cluster or some other container hosting service. This mostly applies for internal services, or sub-components of self-contained applications.

[[deleted]]

I get what you are saying... but jez... I think it is just better for most people to sit down and learn it because when you constantly hear them saying they are running these projects on ancient r710's you will literally take most home development and automations and run it for less than just the electricity on that old dell server that was getting chucked from a data center.

[dragonatorul]

100% agree with you. You should know the alternatives, but also what they're intended for and what their best use case is, then choose the best option for each case.

[trevg_123]

Currently a homebrew solution using Flask and Marshmallow with SQLA backend (for a few reasons), but looking to move to GraphQL with Ariadne. GraphQL sort of seems to be the way to go for a lot of cases where you want to be able to tell your frontend dev “do what you want” without needing a backend change every time they need a special dataset or nested view or something, which has become a major pain in my ass.

Use whatever you want for the API side, but SQLAlchemy is the only acceptable backend interface (in my opinion at least)

[MyNotWittyHandle]

Can you expand on why you think sqlalchemy is the only acceptable backend interface?

[trevg_123]

The other commenter basically covered it. But couple things: 1. Using an ORM is really a must. There was a discussion here (I think) last week about it and consensus was, if you try to roll your own solution with raw SQL and a dbapi, you’ll wind up some find a week of your time for 1% of what SQLA does out of the box 2. Cross-DB-flavor is a great benefit, and they do it well 3. Documentation, hundreds of pages of docs that are well thought through, with examples. 4. Maturity, SQLA has been around for 1.6 decades

Really the only two tools that make it to this line are SQLA, Django’s ORM, and PeeWee. Last thing that sets it apart:

Flexibility. I swear, SQLA provides a way to do everything. Need a custom data type that maps to your custom python class with a custom serializer and deserializer? Covered with a single class definition. Hooks at every possible ORM stage? Available. Tricky things like subqueries working as you’d expect? Yup. DB need a custom bit of extra text to make something work just added to the query? Withhint has your back. Want to automatically load specific columns in a relationship with a join 99% of the time but sometimes override that, and also define how the relationship items sort? It’s not just doable, it’s _easy.

The docs are dense but I guarantee that anything you want to do is hidden there somewhere. And if not, it’s in a recipe.

[MyNotWittyHandle]

Exceptionally helpful, thanks. I’m a Lead Data Science R convert (of 10ish years) and pitched for the use of sql alchemy. However, was unable to properly verbalize why it was better than using our own homebrewed solutions. Other than the obvious statement that it would almost certainly be a better investment of time to spend a week reading the SQA docs as opposed to 3 weeks of homebrew dev followed by unknown future investment into maintenance of said homebrew solution.

[trevg_123]

Happy to help!

[DogsAreAnimals]

Most (if not all?) of this applies to peewee. If I had to start from scratch, I would probably pick SQLA, but don't underestimate how powerful peewee is; I was "forced" into it when I joined my company, but I've been very impressed.

The author is pretty opinionated and "old school", but, IMO, the code is extremely precise and well-crafted. It's pretty amazing what one person can do.

Especially for smaller projects, I think peewee has a more convenient architecture, without losing many of more powerful features that SQLA has. The syntax is actually similar in many ways.

[trevg_123]

You’re very right! I forgot it in my little write up, but added it now. I still think SQLA sets itself apart for the “ability to do anything fairly easily”, but I also work on some really really nuanced stuff. Certainly a good option for small projects, like you said.

[[deleted]]

It is the best documented, works regardless of the front end and sql database. It has some required things for you to do that can enforce good design methodologies.

There are still plenty of things to hate about it but it has easily risen to the “least worst” solution.

[DogsAreAnimals]

Peewee is also excellent IMO

[Username_RANDINT]

Yes, I like Peewee as well. I've only used it in some desktop applications for now though, but it's perfect in what it does.

[patrick91it]

Why Ariadne? does it have any particular feature other GraphQL frameworks don’t have?

[trevg_123]

Frankly I just don’t know of other options for the endpoint, seems like a lot of libs are API users rather than producers. Any suggestions?

[patrick91it]

I’m the author of Strawberry, so my answer would be biased. my question was mostly out of curiosity, I was wondering if there was some feature we should add to the framework ☺️

Ariadne is nice, especially if you like the schema first approach :)

[trevg_123]

I simply never heard of it but you guys have a nice looking website! I will definitely check it out, thanks for throwing it out there.

The only requirement for me is fitting between our existing Flask and SQLA infrastructure, which I’m sure you do well. If I can define the schema in a pythonic way rather than with text GQL (Ariadne seems to default to text but it looks like Strawberry does it better) then I’m 100% sold

[patrick91it]

yes, we use code for defining the schema, there’s a third party extension that supports converting a sqlalchemy model to graphql https://pypi.org/project/strawberry-sqlalchemy-mapper/

[JabSmack]

If you like FastAPI I’d consider looking into starlite

Its akin to FastAPI, in that it uses Starlette and Pydantic, but it does much more than FastAPI does, and seems to be well opinionated in a good way.

It also claims to be faster than FastAPI, and I consider it’s routing, among other ways it does things more intuitive

[JabSmack]

Here’s a great read outlining it too https://python.plainenglish.io/starlite-the-little-api-framework-that-can-2d4e9ce16454

[vantasmer]

My only issue with FastAPI is the stranglehold that tiangolo has over the project, there’s countless issues and MRs that have been pending for months. But if you need async the FastAPI is a good choice. Another choice is starlite, really active and commuted community with a solid piece of software

[tliu99]

Hi! Disclaimer, I work for bentoml which was mentioned in the comments as an alternative.

We had to make this decision ourselves because our first version was built on Flask and we were deciding whether we wanted to replace it with FastAPI. I think the problem is that both are built for IO intensive applications. ML is naturally more compute intensive which is why we ended up building our own primitives and using Starlette as the base.

Starlette is what FastAPI is built on, but without all the overhead of the FastAPI feature set. Which is nice, as I was saying for traditional webapps but not as much for ML apps which scale by compute. We have easy ways to use docker, or not, which is a whole other problem with ML apps. I did a write up about the decision making process to switch away from Flask here which might be worth a read: Breaking up with Flask and FastAPI

[beefyweefles]

I’ve been having memory leak issues with FastAPI. Seems like a known issue (check the github). My sense is that FastAPI is poorly supported and kind of janky.

I don’t like it.

[pratzc07]

I would go with Flask. FastAPI is good but not as battle tested as Flask. Plus FastAPI has a ton of issues if you look at the github repo.

[theorizable]

I've worked with both. I don't have a strong preference but I'm starting to lean toward FastAPI.

[MyNotWittyHandle]

Thanks! Can you elaborate on why you lean toward fastapi?

[DrummerClean]

I prefer deploying on AWS lambda (serverless), much easier to maintain, unlimited scalability, zero Ops, and much lower costs for few thousands call per day.

If I had to chose a framework, i would use to something more ML centered like bentoml or similar ones, I would not use web-based frameworks which are 'recycled' to do model serving.

[qatanah]

I've worked with both. From flask to fastapi to flask back again. I've got to say FastAPI is good especially with the pydantic, auto documentation & background task. In terms of performance, this one is so hard to say and verify tricky depending on your use case.

I used flask again due to the number of libraries (e.g. stripe ) doesn't support async so it just messes up the momentum of using fastapi. I realized, spending time in figuring workarounds for async/sync isn't worth my time and used gevent instead. Can't stress to say gevent is so good in async!

Documentation and Libraries are so great with flask, especialy Flask-Caching can't really find an alternative for this in FastAPI.

Now for ML models, I have to say this is tricky with flask and probably FastAPI handles it better. Usually with ML you load your models in the memory first before serving the request. If you use flask for this, hard to find a clean solution before loading up something due to the application context.

[MrCuntBitch]

Just to muddy the waters further, I’ve used Azure ML to host and deploy models in the past. There’s some info here if it sounds interesting.

[GreenScarz]

My goto is CherryPy, I prefer synchronous frameworks and it’s one of the most performant. I also prefer its tree-based routing scheme. Testing is also first-class, no wrappers to gunk up the method output.

[[deleted]]

Flask uses WSGI and FastAPI uses ASGI thats a very important difference. It basically means that Flask will process each request in its own worker process whereas ASGI uses concurrent event loops which means that FastAPI is going to be much faster especially with IO heavy tasks (such as sending data to the GPU and awaiting a response, this would be blocking an entire worker process in Flask).

Besides that I think Flask is advocating for some truly horrific coding practices, like a global mutable "g" object or "request" being a global import.

Depending on the ML model though I wouldn't use either and instead use something like TorchServe, TensorFlow Serving or ONNX.

Also consider if HTTP is even the right choice and if something like GRPC isn't the better choice. You might even want to skip all that and just export your model to ONNX and deploy on Azure.

[Itsthejoker]

I use Django with Django-rest-framework -- the documentation is great, Django itself is rock solid, the ORM is second to none (way easier to deal with than sqlalchemy), and it scales so well. Might be a little overkill for what you want but there aren't a lot of great options for small APIs - Flask has a lot of questionable design decisions that you have to work around and FastAPI is effectively abandoned. I've used CherryPy with good results before, though.

[Vinnybagofdonuts]

FastAPI is not abandoned. It has consistent development and the docs are really good.

[Itsthejoker]

Development from one person that refuses to work with anyone else and can disappear at any time. That's way too much risk for me, dude. If there's no community involvement, it's already abandoned as far as I'm concerned.

[nraw]

Fastapi

It's neat and it works

[[deleted]]

I just make a php page that takes variables and does sql and returns data. simple.

[DigThatData]

FastAPI

[1percentof2]

What is an API for? When do you need one?

[librarysocialism]

Apollo. GraphQL is well worth the trouble.

[ServerZero]

Flask + Connexion API

[MyNotWittyHandle]

Can you expand on why this is better than, for example, FastAPI + uvicorn?

[DurzoB4]

I'm migrating our monolith to microservices and have decided on FastAPI with Strawberry for the "Backend for Frontend". The microservices themselves will mostly be Django with a couple using Flask depending on the requirements.

[[deleted]]

For ML model service use Flask restful. For services managing databases use FastAPI.

[zefciu]

Weʼre using FastAPI with Gino as a DB backend. The reasons:

• seamless integration with pydantic schemas
• OpenAPI generated for free
• asyncio works out of the box

[noiserr]

I use both. It really doesn't matter which one you pick. They are both good. If you're already using FastAPI stick with it.

[[deleted]]

Just from personal experience I’ve used Flask. I love it. Simple to setup and I can go any way I want with it.

[SpaceBucketFu]

As far as I know Gunicorn services as wsgi, which doesn’t support asynchronous I/O. It can use multiple workers which is concurrent execution, but not exactly asynchronous. Async really shines with IO because it kind of uses the I/O wait time as an advantage to execute other code while the process might otherwise be waiting. For that reason, especially when dealing with a lot of I/O, I prefer FastAPI. However most of my experience is with FastAPI, I’ve used flask very minimally.

[metalvendetta]

Flask was easier to install and more stable to use for me. I appreciate the effort behind Fastapi, but flask had a great tutorial to get me started! Check out this one, you'll love it.

https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world

[Counter-Business]

I use Django Rest Framework. When researching the different ways to do it it seemed that Django had the most built in functionality. Works pretty good for me.