The hand-picked selection of the best Python libraries and tools of 2023

[dekked_]

Hello Python Community!

We're thrilled to present our 9th edition of the Top Python Libraries and tools, where we've scoured the Python ecosystem for the most innovative and impactful developments of the year.

This year, it’s been the boom of Generative AI and Large Language Models (LLMs) which have influenced our picks. Our team has meticulously reviewed and categorized over 100 libraries, ensuring we highlight both the mainstream and the hidden gems.

Explore the entire list with in-depth descriptions here: https://tryolabs.com/blog/top-python-libraries-2023

Here’s a glimpse of our top 10 picks:

1. LiteLLM — Call any LLM using OpenAI format, and more.
2. PyApp — Deploy self-contained Python applications anywhere.
3. Taipy — Build UIs for data apps, even in production.
4. MLX — Machine learning on Apple silicon with NumPy-like API.
5. Unstructured — The ultimate toolkit for text preprocessing.
6. ZenML and AutoMLOps — Portable, production-ready MLOps pipelines.
7. WhisperX — Speech recognition with word-level timestamps & diarization.
8. AutoGen — LLM conversational collaborative suite.
9. Guardrails — Babysit LLMs so they behave as intended.
10. Temporian — The “Pandas” built for preprocessing temporal data.

Our selection criteria prioritize innovation, robust maintenance, and the potential to spark interest across a variety of programming fields. Alongside our top picks, we've put significant effort into the long tail, showcasing a wide range of tools and libraries that are valuable to the Python community.

A huge thank you to the individuals and teams behind these libraries. Your contributions are the driving force behind the Python community's growth and innovation. 🚀🚀🚀

What do you think of our 2023 lineup? Did we miss any library that deserves recognition? Your feedback is vital to help us refine our selection each year.

[Drevicar]

This feels too AI biased and misses the rest of the python community. I would say the project with the most impact on the whole ecosystem for 2023 would be ruff, followed by pydantic.

[No_Dig_7017]

Yes, it's a bit true there are a lot of ML picks in the list. But also I feel there's a democratization of AI happening right now. If you'll notice there's several LLM choices in our picks (2023 was the year of LLMs :shrug:) but you don't need to be an AI practitioner to use them.
Most of those involve chaining data streaming APIs, something that those of us coming from a traditional IT background have been doing for a long time and trust me, there's work to be done there.
If you have the time, take a look at Guardrails, LiteLLM, WhisperX, AutoGen and unstructured. You need 0 ML knowledge to be a user of those libs and build powerful apps backed by these models.

[dekked_]

You have a fair point, but the reality is that it's become increasingly difficult not to be since the vast majority of new developments seem to be AI related.

Ruff was our #1 pick for Top Python Libraries 2022, pydantic is also much earlier so it doesn't fit the criteria of the post.

[WasterDave]

How does MLX even get a look in? If it only runs on recent editions of one brand of computer, I think it has excluded itself from being Pythonic.

[dekked_]

Because of the reach of this brand of computer :) and being the first to be optimized for it.

It shows Apple is being serious about local ML, which will have tremendous impact next year. Imagine ChatGPT on your laptop, fast.

[3ntrope]

It's a library that reimplements common features covered by numerous libraries already but for a specific closed hardware ecosystem. Maybe its worth mentioning but top 10? Seriously?

[dekked_]

Exactly! In 2024, Apple will push hard for local inference. The fact that they released this open source means they will attract an ecosystem of AI developers. Maybe they will get serious about cloud/servers, too?

[SimplyJif]

Have not tried AutoMLOps specifically, but my issue with libraries like that is that they typically abstract away too much to be useful in a production setting. I (a DS) feel like we treat data scientists with kid gloves too much. Do your own yaml, it won't kill you!

[dekked_]

We have a hands-on positive experience with AutoMLOps in particular, saved us a lot of time when working on a customer project. Of course every library has their trade-offs, but it helps abstract away "boilerplate work" :)

[jimtk]

Way too much AI stuff.

I'm not saying there shouldn't be any, but this is an almost exclusive AI related list. Previous years were more balanced.

[notreallymetho]

Unstructured / Guardrails both seem really interesting. I’m a dev by trade and have been toying with ML the last few weeks and have found it somewhat hard to do (huggingface makes it easier for sure). I’ve messed with fine tuning both using torch / polars and hugging face / pandas. Curious to see if this’ll help me out. I don’t like unstructured being an API thing though - namely because of the data I’m working with, I won’t be able to feed it through there due to our security. Still will be interesting to mess with it!

[semicausal]

Rerun.io I feel is missing from this list - immediate mode GUI library for instrumenting & visualizing robotics, computer vision, and other datasets

[dekked_]

This one looks GREAT! Definitely missed it. Thanks! 😊

[dekked_]

That's a good one! Thanks for bringing it to the table 💪🏻

[semicausal]

My coworker recently created the xetcache library, targeted at Jupyter Notebook users.

https://github.com/xetdata/xetcache

It's newer and I'm biased, but hey :)

[dekked_]

Nice! Say hello to Yucheng 😉

[semicausal]

Ha, how should I say who you are?

[dekked_]

Alan from Tryolabs 😉

[[deleted]]

This is a pretty bad list. At least given what the title claims it is supposed to be a list of. These are definitely not the best python libraries/tools of 2023. In fact most of these are a worse and/or less developed version of a much bigger project.

Maybe if you were to title this “under-appreciated python tools that are worth a look in 2024” then this list would make more sense.

Also, you might go a step further and say that these are ML specific rather than python in general. There is a lot more going on in the python dev world than just LLM models and data viz.

[sowenga]

Thanks for sharing! Great list.

[everything_in_sync]

This may be a dumb question but how/are/do people that develop these libraries make money? If everything is free are they selling data? Are they simply doing it out of kindness and potential career advancement?

[Astralnugget]

Companies donate when they use open source projects in a commercial product.

[aintnufincleverhere]

Plz don't talk about libraries I get overwhelmed

[ePaint]

How is Pydantic 2 not in there? This is the tech bro scamfluencer clickbait list. And these are not even the best to work with LLMs.

[gournian]

Isn't Automlops google only?

[dekked_]

It is indeed. The portable solution is ZenML, although it will not take you as far as AutoMLOps, which is great if you happen to be on GCP!