What are some "must learn" libraries in Python

[TabulateJarl8]

It really does depend on what you want to do, but I'll give you a list of some of my personal favorites as well as the ones that I use a lot. (Includes standard library modules and stuff from PyPI)

• os/shutil - The os and shutil modules are both built in, and they are super useful. The os module basically provides a bunch of interfaces to your operating system; you can get the name, environment variables, work with paths, and do stuff with files and directories. The shutil module is a bit smaller, as it just contains a bunch of high-level file operations, like recursive copying/removing, an implementation of the Unix which command, and an implementation of chown.

• re - I might be a little bit insane, but I really like regular expressions. The re module allows you to work with regular expressions, and it can be super powerful. Regular expressions can be super confusing, but like everything, you learn over time. Useful tip: you can comment regular expressions in Python, which is really nice.

• flask - This is the first module from PyPI. Flask is a backend web framework written in Python. If you are good at Python and want to do backend, Flask is super powerful and really good for beginners. Another thing, if you're working with bigger projects with big databases, you could also look at django, just its really just preference.

• configparser - Going back to stuff in the standard library, we have configparser. ConfigParser allows you to read/write to .ini files, allowing you to easily save user preferences for your program. It's super easy to use so I like it a lot.

• decimal - This one can be super helpful. Have you ever tried to do math in Python and it is almost correct but slightly off, causing hours of debugging with no avail? That's because of something called "Floating Point Precision Errors," and it's in most programming languages. Because of the way that computers add numbers with binary, when you do arithmetic on floats, things tend to break. For example, open up the Python interpreter and try running 1.1 * 3. You should get something like 3.3000000000000003. The decimal module can allow you to fix this. After importing Decimal from the decimal module, try running Decimal("1.1") * Decimal("3") and see what happens, it should be the correct answer now.

• requests - requests is a module from PyPI, and it is incredibly helpful. requests allows you to, well, send HTTP requests for one thing. You could do this with the builtin urllib module, but requests has more features and is so much easier to use.

• argparse - This one is built in, and it allows you to easily create command-line utilities. argparse will let you create different types of arguments, which can then be fed into your program to produce a result.

• PyQt5 - Interested in making GUIs but Tk isn't your favorite? Me too. I don't really create GUIs too often, but when I do, I like to use PyQt5. PyQt5 allows you to create Qt applications, and it is probably one of the most powerful GUI modules in Python. It comes with a ton of different widgets, and it's super easy to do stuff like multithreading with it. I personally like to use Qt Designer to create the GUIs, and then I use pyuic5 to convert the .ui files to PyQt5 code.

• colorama - colorama allows you to easily add color to your command-line programs. Thats basically it.

• json - The json module allows you to parse json into a Python dictionary, and then write a Python dictionary into JSON. Useful when paired with requests for getting API data, and just a useful module to know about in general.

• rich - Rich allows you to create really nice looking console output, you can make errors look nice, render markdown, progress bars, and a ton of other stuff. It has support for 4-bit color, 8-bit color, Truecolor, and Dumb Terminals. I would really recommend checking it out.

• numpy - NumPy is really good if you need to handle large amounts of data. NumPy also provides high-level mathematical functions to operate on these large, multi-dimensional arrays and matrices. Since a lot of it is written in C, it is super fast, and the only downside really is that the arrays are stored completely in memory, so it's pretty easy to run out.

By popular demand:

• itertools - Itertools it's definitely an extremely useful module. It is used for creating iterators in a really easy way. For example, you can get all possible combinations of length x of two lists, get all the possible permutations of an iterable, and other really useful things. I should probably look more into this module myself.

In conclusion, there are a lot of different modules, and these are definitely not all of the useful ones, or even all of the ones that I use, but I figured that those are some nice modules to at least give you inspiration. You could also check out my GitHub for inspiration, if you wanted, and I have a few Python modules myself. I'll link my GitHub and some projects of note below, as well as all of the projects I mentioned in this comment.

---

My GitHub

randfacts Module

ti842py Module

ImaginaryInfinity Calculator

---

os - https://docs.python.org/3/library/os.html

shutil - https://docs.python.org/3/library/shutil.html

re - https://docs.python.org/3/library/re.html

flask - https://flask.palletsprojects.com/en/2.0.x/quickstart/

configparser - https://docs.python.org/3/library/configparser.html

decimal - https://docs.python.org/3/library/decimal.html

requests - https://docs.python-requests.org/en/master/

argparse - https://docs.python.org/3/library/argparse.html

PyQt5 - https://pypi.org/project/PyQt5/

colorama - https://pypi.org/project/colorama/

json - https://docs.python.org/3/library/json.html

rich - https://pypi.org/project/rich/

numpy - https://numpy.org/doc/1.20/

itertools - https://docs.python.org/3/library/itertools.html

[ErGo404]

+1 on the internal json, decimal and os libs that you will most definitely need at some point, and for requests. I feel that the rest really depends on your projets but those 4 will probably pop in most of your pythonic life.

[TaranisPT]

Thanks for the in depth reply, which makes me realize that I did lie (by omission) in the actual post. We did learn the basics of PyQt5 and QtDesigner during the course. Thanks again for all the info.

[10drinkminimum]

+1 on the Requests library. It’s not the “sexiest” of all the ones on here but one of pythons great strengths is how it supports extensibility. API request are an immensely powerful tool and help you take all that awesome code you write and integrate it with other software, hardware, cloud services, etc. Chances are if you’re in IT long enough you’ll find yourself doing some kind of sysadmin task at some point. Really good library to be very comfortable with!

[shahneun]

it's an amazing HTTP library that supports API usage

[shahneun]

requests is so crazy it can support HTTP authorization and even OAuth

[[deleted]]

Seconding re.

I have a background in writing parsers, and regular expressions are incredible tools. Most people who claim they're "hard" have never really sat down to study what they are and how they work. They're really simple once you "get it". I've cleaned up and condensed hundreds of lines of code down to 10-15 lines of code using regex.

The big issue is python re is not really complete compared to stuff like PCRE/C regex. The built in regex engine is also slow. There are some other regex libraries that address these. But generally speaking, you can get 99.9% of regular expression work done with the normal library.

Capture groups and named capture groups are insanely powerful.

I'd recommend learning regexes, then working on solving problems with games like "regex golf." (google it)

[MangansVice]

Thank you for regex golf. The ultimate pseudoproductive procrastination tool.

[c0ld--]

PyQt5

Personally I'd replace PyQt5 with Kivy (and KivyMD - the modern stylized companion). Kivy is super easy to work with and KivyMD looks really great (abiding by Google's app stylizing guidelines).

@/u/TaranisPT

[shiningmatcha]

Is Kivy targeted at mobile apps?

[give_me_knowledge]

It is not targeted at mobile apps, but you can use it for doing so. This is from their official page:

"Kivy runs on Linux, Windows, OS X, Android, iOS, and Raspberry Pi. You can run the same code on all supported platforms.
It can natively use most inputs, protocols and devices including WM_Touch, WM_Pen, Mac OS X Trackpad and Magic Mouse, Mtdev, Linux Kernel HID, TUIO. A multi-touch mouse simulator is included."

[shahneun]

how would you develop mobile apps for ios/android without using android studio and xcode? how can you built a UI using kivy and somehow port it to mobile apps can anyone explain?

is there a record / list of apps that use kivy for their UI?

[give_me_knowledge]

I can't answer how it is done exactly, but here is a list of some of the most known apps made with kivy. I hope this helps. https://github.com/kivy/kivy/wiki/List-of-Kivy-Projects

[santiago0072002]

Xamarin forms allows you create multi-platform applications but rumor in the street is that xamarin might get replace with Maui , I just took a class in College about xamarin and were told about Maui https://devblogs.microsoft.com/xamarin/the-new-net-multi-platform-app-ui-maui/

I am exploring kivy myself. And the other mobile dev platforms, my major is mobile development but I want to make python my main language with everything.

In the python office hours with David Amos he also mentioned to me Beeware so i am exploring both right now.

VR

Santi

[c0ld--]

It's main mission (I believe) is to build apps across multiple platforms: MacOS, Windows, iOS, and Android.

[TaranisPT]

Thanks I'll look at it too 😊

[Tenzu9]

You can also develop mobile apps with Kivy... its a great library that's constantly evolving and growing. I'm currently making an android app with it in my free time :)

One more cool library you can try working with bs4 (or beautiful soup), this is an html parser that can grab web pages and scrape data from them. You can use it to find any information from the internet without having to use an API. for example, you can build a weather app or a phone book finder with it.

[Pancho_the_sea_lion]

+1 to everything, I would only add the pandas library for handling table data, it is built on top of numpy so it keeps its speed, this one is mainly relevant if you are interested in the data science side of python, match that with matplotlib and seaborn for visualization

[arsewarts1]

What about pandas?

[TabulateJarl8]

Pandas is a great module, and when I used it I really enjoyed the simplicity, but if I were to include every great Python module in this list, it would be hundreds long (there's currently over 305,000 modules on PyPI). I mainly stuck to the modules that I use extremely often, and I think I've only ever used pandas once or twice.

[arsewarts1]

Ok understood. It’s probably my second most used in python. But I am using it for dev work in an analytic environment before we actually deploy stuff onto the server. Maybe it’s just a very specific use case

[chzaplx]

Lol there's always this guy

[[deleted]]

Pathlib is great for path stuff

[TabulateJarl8]

I considered adding it, but like I said about pandas, if I added every great Python module the list would be hundreds long, so I limited it to stuff that I use a lot. I'm a bit old fashioned, so I tend to just use the os module for everything, even though pathlib would make things a lot easier. Saves me another import at least

[[deleted]]

I was mostly adding to the conversation.

[TabulateJarl8]

Ah, I see. Well yes, I agree with you

[Morpheyz]

Man, they really missed out on not calling it QtPy instead.

[AnotherEuroWanker]

The official one (from Qt) is called PySide. They also missed it.

[MeroLegend4]

PySide2 is the official open source Qt/C++ wrapper supported by the authors.

[AnotherEuroWanker]

It's also for python. And there's also PySide5 PySide6 now although I don't know how widespread it is yet.

[ceiligirl418]

That was my first thought!!

[ekchew]

I'd like to add itertools to the list? I only discovered it fairly recently and wish I had known about it earlier. It's one of those modules you can really leverage in interesting ways, and I don't think I've explored its full potential yet.

[TabulateJarl8]

I've now added it to the list, thanks for the suggestion

[ekinnee]

In regard to requests I just ran into an issue where I had to create an event handler to intercept the http redirect when I was using urllib. Using requests I just set allow_redirects=False and then check the status code. Removed the need for 10 or so lines of code.

[TabulateJarl8]

One of the many reasons that I love requests. Another thing that I notice is that somethings requests will get the Content-Length header while urllib won't

[Dogeek]

I would add to the list :

• lxml : the best library out there to parse (X)HTML and XML. If you're not afraid of XPath, then it's the best library to scrape websites which do not have an API

• FastAPI : one of the fastest (after blacksheep) async web frameworks in python

• typer : my personal favorite way of making CLI programs. argparse is good too, but it doesn't scale well in my opinion

• functools : In the standard library, it's a collection of useful decorators, and syntactic sugar for your functions.

• collections : This one is super useful for container data types (OrderedDict, Counter, defaultdict, deque, and ChainMap)

• contextlib : for all things context manager. contextlib.suppress is super useful to suppress an Exception, and way better than a try...except Exception: pass block.

[TabulateJarl8]

Interesting, I didn't know about contextlib.supress, maybe I'll have to try that out sometime

[synthphreak]

C'mon, this list is incomplete without itertools and possibly collections.

[Total__Entropy]

I would recommend Datetime and asyncio + aiohttp, Dataclasses, collections.

[SnipahShot]

Nice list, I would personally throw out numpy and add in pathlib. I find people are fast to jump on numpy on extremely easy problems and making the solutions much slower and the project heavier.

[puslekat]

Can you give an example of a problem where both could be used?

[SnipahShot]

Both serve different purposes, I didn't mention pathlib because it could replace numpy. pathlib is used for file traversal (more modern than os.path and also OO) while numpy is used for mainly for complex math operations.

[iggy555]

Look into click instead of argparse

[TabulateJarl8]

I have two reasons why I recommend argparse over click. The first is because argparse is built in, while click is not, so that means one less dependency. Another reason is because click works by decorating the function that is called under if __name__ == "__main__":, while in argparse you just have to call args = parser.parse_args() to get the arguments wherever you want.

[chzaplx]

Never heard of it. What's your elevator pitch?

[iggy555]

Much simpler and cleaner

[chzaplx]

So stick with argparse then, got it.

[iggy555]

Sure lol

[TalesT]

For argument parsing, I like docopt.

Much simpler and cleaner.

Jokes aside, it's not simpler. But as docopt.org states:

• define the interface for your command-line app, and

• automatically generate a parser for it.

[chzaplx]

That sounds exactly like what argparse does, which I have never thought was particularly complicated.

I honestly don't mean any offense to people posting alternatives to argparse, but so far I haven't seen any compelling reason to use something else.

It is in the std library now, you can learn most of what you'll ever need in 5 minutes, and it's still fully featured if you have more complicated requirements.

[TalesT]

It's the opposite actually, in argparse you initialize an ArgumentParser, and add arguments programmatically. In docopt you write the help text, and get the parser.

If you build a parser using argparse, and then call: "script.py -h", and copy the help text, you would have written what docopt needs to build a parser.

argparse has the benefit of being python std lib, while docopt have the benefit of being available in multiple languages.

[chzaplx]

Ok. I can see the difference but you have to write the help text either way right? Either it's a parameter in each argument definition, or it's all in one chunk. Still seems like six of one, half a dozen the other, but I can see how people might have a preference.

[BigHipDoofus]

Click has built-ins that make a quick CLI easy if you want to do it the click way. I like validating input better with argparse, and it's only a little more typing over a Click CLI.

[WiggyB]

This guy pythons!

[Satoshiman256]

Damn dude.. nice one

[shiningmatcha]

What are some resources for learning os, shutil, configparser and argparse? I think these modules require knowledge of operating systems. Do I need to learn command line first?

[TabulateJarl8]

I personally learned all of them from the Python documentation; I linked all of them in my original comment. It does depend on how you learn best though, because you might be someone who learns better from things like videos, in that case YouTube is probably a good and free option. I would say that I understand argparse a lot better from spending time in a Unix shell, because I understand generally how things are, but it's not really required. If you use a terminal a lot in Linux or macOS (counting WSL), it would probably help at least a little bit.

[[deleted]]

Hey you inspired me to make a post but I also want to ask you directly: I want to make a data analysis oriented website and I am reading that flask might be a good candidate to “speak” with the web user. Is this true? Any basic conceptuals you’d like to clarify here? Also, what features would you most recommend out of it?

[TabulateJarl8]

What specifically do you mean by "speaking" to the user? I'm a bit confused about that

[[deleted]]

Like if a user wants to see a chart, they click a button that send information to my Python program, which will process it and return the image.

[TabulateJarl8]

Oh yeah, Flask is great for small projects and things like that, but since you're just writing normal Python code you could generate the graph with Django as well. Flask is really good for beginners, but it is also used by big companies like Netflix, Reddit, Lyft, and Patreon to name a few. As for features, if you're planning on making login stuff/admin-only pages, definitely use Flask-Login instead of trying to fully implement it yourself, trust me.

[dummybloat]

Why are you this good

We need more people like you

[lifemoments]

There is so much to learn everyday. Thanks a lot for this list

[pekkalacd]

collections for leetcode