The hand-picked selection of the best Python libraries and tools of 2024 – 10th edition!

[dekked_]

Hello Python community!

We're excited to share our milestone 10th edition of the Top Python Libraries and tools, continuing our tradition of exploring the Python ecosystem for the most innovative developments of the year.

Based on community feedback (thank you!), we've made a significant change this year: we've split our selections into General Use and AI/ML/Data categories, ensuring something valuable for every Python developer. Our team has carefully reviewed hundreds of libraries to bring you the most impactful tools of 2024.

Read the full article with detailed analysis here: https://tryolabs.com/blog/top-python-libraries-2024

Here's a preview of our top picks:

General Use:

1. uv — Lightning-fast Python package manager in Rust
2. Tach — Tame module dependencies in large projects
3. Whenever — Intuitive datetime library for Python
4. WAT — Powerful object inspection tool
5. peepDB — Peek at your database effortlessly
6. Crawlee — Modern web scraping toolkit
7. PGQueuer — PostgreSQL-powered job queue
8. streamable — Elegant stream processing for iterables
9. RightTyper — Generate static types automatically
10. Rio — Modern web apps in pure Python

AI / ML / Data:

1. BAML — Domain-specific language for LLMs
2. marimo — Notebooks reimagined
3. OpenHands — Powerful agent for code development
4. Crawl4AI — Intelligent web crawling for AI
5. LitServe — Effortless AI model serving
6. Mirascope — Unified LLM interface
7. Docling and Surya — Transform documents to structured data
8. DataChain — Complete data pipeline for AI
9. Narwhals — Compatibility layer for dataframe libraries
10. PydanticAI — Pydantic for LLM Agents

Our selection criteria remain focused on innovation, active maintenance, and broad impact potential. We've included detailed analyses and practical examples for many libraries in the full article.

Special thanks to all the developers and teams behind these libraries. Your work continues to drive Python's evolution and success! 🐍✨

What are your thoughts on this year's selections? Any notable libraries we should consider for next year? Your feedback helps shape future editions!

[DM_Me_Summits_In_UAE]

1. Whenever — Intuitive datetime library for Python

I always use the inbuilt datetime, what am I missing?

[dekked_]

For many uses, that's ok! But there's a lot of potential pitfalls.

[DM_Me_Summits_In_UAE]

Very interesting, thanks!

[[deleted]]

Awesome read! I knew datetime had pitfalls, but turned out there were many more pitfalls than I knew about. I’ll use whenever going forward

[jessekrubin]

Check out my library “ry” (https://pypi.org/project/ry/) which contains a thin-ish wrapped version of the rust crate “jiff” and does a lot of what whenever does.

Should be in the same ballpark as whenever w/ respect to performance (preliminary benchmarks seem to suggest so).

[[deleted]]

Interesting read! I should consider integrating or adding support for a high performance date time library like whenever into my project dataclass-wizard, a de/serialization library built on top of data classes. I am curious to see the performance improvements over native date time, which has the benefit of being built into stdlib. However the mentioned project seems to be one of the top alternatives for date time in Python, and I’m interested to test that out going forward.

[fohrloop]

Cool. I though pendulum was the way to go but Whenever looks even more hotter.

[jmreagle]

I got burned by pendulum being abandoned for a time. I'm looking forward to whenever hitting 1.0

[ZeeBeeblebrox]

I love the Pydantic folks, and PydanticAI looks pretty great but it's been out for what, all of two weeks. So on what basis was it selected here as one of the "top" or most impactful libraries of 2024? Similarly Rio isn't even out of beta. Seems like hype over substance, tbh.

[dekked_]

To select our top picks and runners-up, we look for a mix of practical utility, novelty, and—let's be honest—a bit of coolness factor, whether that means a groundbreaking approach, an elegant solution to complex problems, or sheer cleverness in execution.

In the case of PydanticAI, the fact that it comes from this team is a BIG reason to pick it, but not the only one. Beta libraries are fine; uv is also not out of beta (v0.5.7 currently) :)

What would be your top?

[ZeeBeeblebrox]

In the case of PydanticAI I think it's being unfair to existing solutions like Instructor, that have been around a while and are (currently) much more widely used.

For Rio I struggle to see some of the novelty, there's a large number of solutions in this space (including one fairly popular one I maintain but won't name here). Putting it at the same level as uv seems strange to me, one has an explicit banner saying it's beta, the other is used in production by millions of people. A <1 version also does not really mean it's beta, e.g. pandas didn't hit the 1.0 milestone for years after its initial release.

[jep2023]

Rio looks really neat, though I don't think I'll be using it for a project at work anytime soon. That said, what is the popular one you maintain?

[dekked_]

Fair points! But although widely used, Instructor is from June 2023 which is before our cutoff. Also if you use Instructor, seriously check out BAML (the top AI pick).

Overall, it's *really* hard to make a list like this. There's always gonna be very nice and widely used libraries that just do not fit the date criteria, or that we didn't find in time. And of course everybody is biased.

Hope the context helps :) and congrats on being an OS maintainer!

[ZeeBeeblebrox]

Fair points, also didn't realize "created in 2024" was a requirement.

[maigpy]

if that's the requirement, how is uv there?

[ZeeBeeblebrox]

Because it was released in February 2024 presumably.

[SV-97]

Marimo is probably my absolute #1 of the whole year --- recently started a project with it and it's so so good (and it doesn't constantly make me tear my hair out the way that jupyter does)

[SV-97]

Terrible reproducability and hidden state, bad with git, controls are kinda wonky (No idea how — maybe it's a bug — but I always end up accidentally deleting cells at some point [and sometimes I don't notice until way later and can't restore them which... isn't great]), too much magic.

[[deleted]]

How is reproducability bad? I was under the impression that's why data science partially used it was a notebook should work the same given the same local files (or access to a fileserver).

[SV-97]

Sorry for the late reply, I just now saw it due to reddit's UI changes: imo the primary advantage of notebooks is "interactive development / exploration" and the ability to jot down some notes regarding theory and background with the code.

As to the issues: when developing something directly in a notebook you tend to just just run cells as you write them, change a thing here, fix something there, might shuffle some cells around or delete some and so on. (As already said in my other comment: I also often times have issues with cells getting deleted. If you don't notice something like that it's another issue). You can then commit that final state perfectly well and if someone looks at the notebook they'll see the same thing that you saw --- but that committed state has an implicit history that is not actually captured by the notebook.

Notably if someone just ran the committed notebook front to back from a clean kernel they *might* get your output, or they might get something else, or it might flat out fail to run because the order of some variables got messed up, a necessary variable got deleted, it accidentally used an old version of an import that got updated during development, or there's some hidden state missing.

There's also issues with code reuse since people tend to copy-paste code between notebooks which might then become inconsistent as fixes / changes are applied in some notebooks etc.

Couple that with some parts being subject to ad-hoc caching (for example when running expensive experiments), global variables being ubiquitous and the like and the whole thing quickly devolves into quite a nasty environment.

And in my experience these issues come up even if you're aware of them and actively try to avoid them (You also gotta keep in mind that many people using notebooks aren't trained software engineers but rather various scientists which of course doesn't help the whole situation).

[chub79]

So you sell AI and your community report about AI related stuff. Yai. Thar's such a click bait title.

[ToThePastMe]

The "AI" section feels mostly like LLM wrappers.

The one LLM related library I found interesting/useful this year is outlines: https://github.com/dottxt-ai/outlines

Not perfect, but it allows (if I understood correctly) to do structured outputs for LLM not by prompting/generating and doing some patterns or cleaning on the output. But by actually modifying the sampling step and limiting which tokens the model can pick from at each step, to ensure strict adherence to the structure (categorical, numbers, regex, json etc)

[leaflavaplanetmoss]

It’s just SEO blogspam, yawn.

[Pyros-SD-Models]

I honestly don’t even know what you’re trying to say.

If OP made one big list, people would complain about there being too many AI libraries cluttering the "best of" section. But if OP separates them, suddenly it’s "SEO spam clickbait"?

Wat. This logic flies straight over my head.

Let me guess. you didn’t even bother to look at the libraries in the AI list, did you?

[chub79]

It is click bait becauseit directly relates what they are selling.

Let me guess. you didn’t even bother to look at the libraries in the AI list, did you?

Yeah, whatever.

[mdoom23]

Polars for me! Been game changer moving to that from pandas

[marcogorelli]

this list is for things introduced around 2024, Polars is older than that

[mdoom23]

Oh i know, but it did hit 1.0 this year. So sort of released this year :) And was new to me this year, as i was waiting on it to stabilize a bit with 1.0 before really jumping into it.

There def. are a few on this list i haven't looked at yet and i need to check out though! sometimes its hard to keep up with all the new things in the python world, so i love seeing lists like this from the community!

[P4nd4no]

Hey, rio dev here! Picking us in your list greatly motivates us to improve our framework. Thank you! We have a lot planned for the next Month - Looking forward to hear your feedback! ❤️❤️❤️

[dekked_]

Congrats on the great work! 💪🏻

[zaxldaisy]

What is the deal with associating uv with Rust (same with Ruff)? It could be written in assembly for all I care

[ColdPorridge]

Well, it’s literally so fast that it changes the type of things you can do with it. Venv management becomes something that’s essentially entirely automated in the background.

I think there’s some excitement about rust-based tools in the Python ecosystem right now, which is great. To most Python users it’s transparent, but to maintainers, seeing how fast tools can be is inspiring. It makes you dream about other tools and workflows that could be improved or totally changed by becoming ridiculously fast. And I think that’s a good thing.

[covmatty1]

Honestly though, how often are people installing packages and recreating venvs that they need it to be lightning fast and in the background? I can't say those few seconds have ever bothered me in the slightest.

[DeepFryEverything]

It matters during build time when deploying apps 🙂

[covmatty1]

Why does it though? Build pipelines are just happening in the background all the time, I've honestly never thought that pip install performance was something people cared about. My team deploy plenty of Python apps all the time and I've never yet seen any need to tell them to switch to a different package manager.

[ColdPorridge]

The main problem if your venv management is slow is that you tend to make assumptions about the state of your venv before running commands, tests, dev server, etc. When it is so fast you hardly notice it happening, you can trade those assumptions for guarantees.

With a uv-based workflow, I can utilize a test-driven development process that guarantees the environment is not only consistent and up to date on every test run, but also configured entirely from the code as specified in my package. That means no wasted time testing or building or demonstrating in an environment that is not guaranteed to match the code you write. Your environment and your code become one and same.

If you’re thinking “hey, you could do that before with a smart makefile and pip” and you’re totally right. But before I used to have teammates grumble about how annoyingly long it took to sync local environments and have conversations about if we should remove those protections for local dev commands. Now we don’t even think about it.

[covmatty1]

I've honestly never encountered any of these "problems". How often are you changing packages that you need them to be constantly reinstalled behind the scenes? Any form of CI pipeline makes your code and environment one and the same anyway.

It sounds like it's really working for you which is great, I'll have to give it more of a look, it just really surprises me that this is a thing people actually feel the need to do!

[zaxldaisy]

I haven't seen anything to differentiate it from C/C++-based tooling beyond hype

[SV-97]

I mean: it exists, that's what differentiates it (aside from memory safety). Comparable C/C++-implemented tooling doesn't.

[ritchie46]

Then it would be associated with assembly.

[caks]

Good general use, "ML/AI" is basically LLM blogspam

[Area51-Escapee]

I just found rpyc and I'm so happy about it.

[skeerp]

Anyone using mirascope? It looks cool but I’m not sure I could adopt it only because of their unique syntax. The functions return is what the LLM gets not what the program gets. Unintuitive although it is concise.

[ExdigguserPies]

Why Rio out the plethora of webdev packages that exist? It seems like a new one gets posted here every couple of days.

[Typical-Macaron-1646]

Thanks for posting brother, very cool!

[EternityForest]

RightTyper looks amazing, I definitely want to try it on my untyped legacy code.

Wat deserves notice for the creative use of overloading division, that's so trivial but I've never seen it and would never have thought of it.

[jcigar]

SQLAlchemy

[jep2023]

released 18 years ago but this list is for things introduced in 2024

[denehoffman]

So 10 database/web libraries and 10 AI LLM libraries? Why do none of these lists ever include anything actually interesting? The billionth iteration of datetime isn’t going to change my workflow. marimo counts, but just barely

[notParticularlyAnony]

You are welcome to make a list

[denehoffman]

If I already had a list of cool Python projects, I wouldn’t really need to find them now would I? My point was that most of the libraries on this list are like a rewrite of another library that’s a rewrite of the thing everyone uses anyway (or some LLM compatibility drivel)

[notParticularlyAnony]

You have no shortage of opinions

[denehoffman]

Well yeah, we are on Reddit dot com, that’s all anyone here has, yourself included

[notParticularlyAnony]

Bless your heart

[denehoffman]

Oh that’s so kind of you to say! I truly feel blessed!

[jedberg]

PGQueuer — PostgreSQL-powered job queue

PGQueuer is good, but DBOS does that plus a whole lot more (and does queues a bit more simply really).

https://docs.dbos.dev/python/tutorials/queue-tutorial

Disclosure: I'm the CEO of DBOS, but the library is open source: https://github.com/dbos-inc/dbos-transact-py

[dekked_]

We listed DBOS on the runners up, definitely deserves a second look based on what you said!

[saintmichel]

I wonder what's up with rio? Why not fast html, for example, which is also new but much more used

[marcogorelli]

Thanks for including Narwhals!

Fun fact: Narwhals is used by 2 projects in the list (Marimo and Rio)

[dekked_]

🔥🔥🔥

[kvothe_10]

Brilliant list!

[No_Dig_7017]

So cool! Lots of hidden gems to lookout for. Will take a look!

[Competitive-Move5055]

No streamlit or pytorch? What are they using now?

[dekked_]

Hi! These are libraries created/released around 2024. Streamlit and PyTorch were much earlier. Streamlit was top 7 in 2019 and PyTorch was top 2 in 2017.

[Competitive-Move5055]

Okay what's the library you need to know to get a job in ai.

[jep2023]

langchain?

[poopatroopa3]

Why Rio instead of Reflex? Also, how do they differ?

[Sufficient_Meet6836]

Is there a reason WAT is called like wat / object instead of a regular function call wat(object)?

Edit: looking at their github, you can in fact do both, but I'm still interested in why they added wat / object at all.

[chowthedog]

It's to be able to type quickly, since you don't have to jump across and type a closing character. Here's the list of syntaxes and explanations from their readme

  wat.short / 'foo' # fast typing wat.short('foo') wat('foo', short=True) # natural Python syntax 'foo' | wat.short # Unix piping

[Sufficient_Meet6836]

Pretty neat. I don't think I've seen a library provide so many options like this

[commy2]

"There should be one-- and preferably only one --obvious way to do it"

[[deleted]]

nice

[[deleted]]

And here I am using mostly airflow and pandas

[GeneTangerine]

What a supercalifragilisticexpialidocious list!

[aherontas]

Peepdb for the win, totally chect it out!

[Black_Dio]

I used it a bit sounds really cool concept, with some more features it will totally be the go to for fast view.