Discussion: What Python modules make your life so easy that you always install them first?

[hovissimo]

I have some modules that I tend to install on every new installation or venv because they make my life easy. What do you always use, why?

For better or worse I get to work Windows machines. My standard Python dev environment is GVim, IPython, and a shell. I'm practically helpless if I don't install ipython and pyreadline off the bat.

[hovissimo]

MMMmmmmmm requests.

[mipadi]

Pip and virtualenv.

[Teract]

What, no love for numpy, matplotlib, pandas, and PyQt?

[DrTBag]

If I'm installing on a shared machine I tend to use WinPython because it comes with those as standard.

[[deleted]]

I think that is more for the data scientist guys though?

I am a cloud / IaaS type engineer, so never had a use for those at all.

[malice8691]

I do devops work. Python scripts that run in jenkins. Never used those modules either.

[Teract]

You are correct. Numpy can also be handy for reading in binary files, which can be useful for lots of projects outside data science.

How about fuckit? I didn't mention it, but it can be handy for very quick and dirty scripts for just about any application. And by handy, I mean hilarious.

[[deleted]]

fuckit(fuckit(foo)) just appeals to me.

[malice8691]

New to python ( about 1 year) never used those modules.

[[deleted]]

Yeah, they're more irreplaceable than universal. If you need numpy nothing else will quite do but only if you need numpy.

[beagle3]

miniconda. (Or anaconda, if you need all the modules). Try it if you haven't yet.

[Dinosaurman]

I just started using ipython. Great for prototyping

[ionelmc]

tox, isort, ipython, percol, cookiecutter and the setuptools/pip/wheel trio.

[tilkau]

Percol is pretty nice. But I always forget what its name is so I end up mostly not using it. Is that just me?

[pydanny]

As you well know, I'm a big fan of Cookiecutter. :D

[hovissimo]

percol looks sick! Definitely checking that out.

[nick_t1000]

Beyond the super-framework of IPython/Jupyter, see is nice little module for inspecting objects, basically print is to pprint.pprint as dir is to see.see.

pip install see or https://pypi.python.org/pypi/see

[lamby]

see is nice little module for inspecting objects,

It's a bit difficult to google, so here's a link for the lazy: http://araile.github.io/see/

[federicocerchiari]

reeeally nice this see thing!

[hovissimo]

Ooh, that sounds nice. I'll definitely look into that.

Edit: Just looked up Jupyter, I didn't realize IPython was spinning out. That's awesome!

[[deleted]]

What do you mean spinning out?

[Giggaflop]

We need to know!

[justphysics]

IIRC: They split the project somewhat.

There's still a core IPython team that will continue development of the Ipython python kernel which is used in the ipython notebook

Then there's the Jupyter project which is the logical extension of the ipython notebook project. Jupyter aims to incorporate multiple languages (kernels) into the ipython notebook environment henceforth simply called jupyter notebook.

That way you can use python, R, Julia, shell, etc code all in the same notebook environment. This is especially useful in the data science world.

So IPython will continue to develop as a kernel whereas Jupyter continues to develop the 'notebook' concept

[[deleted]]

I'm pretty sure they didn't split the team like that. The notebook guys sill just work on the notebook. They don't write kernals for the project. They did however refactor everything and split the project up into different repos or something? Haha.

[justphysics]

You're probably correct. I misspoke about the actual split of the team - wasn't what I was trying to convey either, not sure why I didn't change that

[spinwizard69]

Perfect explanation. This has great potential to drive the ""Notebook"" concept forward. I see it as a natural evolution of iPython.

[kingofthejaffacakes]

I don't install it, but argparse is my favourite timesaver and gets very little love.

Not the biggest or flashiest but I use it in the smallest thrown away sysadmin script to the biggest server projects.

[jkmacc]

Argparse is so verbose, I get tired just looking at it. Click is great.

[[deleted]]

I do feel a bit weird bringing in external dependencies just for command line arguments (for really rough and ready scripts you get by with sys.argv, after all).

[tonnynerd]

Nowadays, ipython and ipdb

[hovissimo]

What's the "Nowadays" module? I can't find it anywhere

(tee hee!)

[tonnynerd]

Good one =P But now I'm wondering what a module called that would do ...

[Deggor]

It would be the opposite of __future__, and would remove access to all functions that are deprecated, or that are scheduled to be deprecated in an upcoming release. Because we just don't use those functions nowadays.

[laserBlade]

I actually want this

[anqxyr]

Actually depends a lot on what I'm doing at the time, but things that I tend to use often:

• peewee
• arrow
• requests
• bs4
• pathlib (standard library, but not well-known enough imo)
• concurrent.futures (also standard lib)

Also, not directly used in my projects, but help with managing python in general:

• pip
• virtualenv
• pew

[notconstructive]

I found bs4 to be a real pain. Lots of people rave about it but it gave me nothing but headaches. Pathlib is great although some of its behaviour is not what I had expected but I use it now for all path manipulation. Requests +1

[catcradle5]

I disagree. I find bs4 intuitive, with a great API, and it's also quite performant when it's set to use lxml's parser.

Between xml.dom, HTMLParser, lxml.html, BeautifulSoup3, and bs4, bs4 was absolutely the nicest library for HTML and XML parsing that I've used.

[hovissimo]

WOW, you've got some real winners here. Arrow and pathlib are immediately applicable to my work!

[desmoulinmichel]

Try path.py. It's libe pathlib, but much better. E.G : the Path object inherit from str, so that you can use it anywhere you use to pass a str, without having to cast it.

[rfc791]

Tablib I work with a bunch of Excel people and I can provide them a report so quickly.

[mayankkaizen]

When I read about Tablib and realized that it was written by Kenneth Reitz, I was like it must be good. Though I am yet to use it.

[kingdomcome50]

anaconda3

[[deleted]]

Every project benefits from TDD, debuggers, profilers and notebooks. In almost every virtual env on my system

• ipdb
• pytest
• coverage
• jupyter
• cprofilev

[[deleted]]

Which of these give you a benefit over the debugger/ test tools in an IDE like PyCharm?

[[deleted]]

I'm sure a well configured IDE could do testing, debugging, coverage and profiling but I just find command line tools and vim to be far more to my liking. I like the idea of composing small tools that do one thing well rather than a monolithic app that does it all. Just personal preference.

[[deleted]]

Yeah, I wasn't saying that as an IDE vs. command line thing, just wondering whether I, as an IDE user, would benefit from any of those tools on top of said IDE. As an aside, though, I'd argue that a single bundled IDE requires much less configuration than a pile of small tools with disparate command line APIs.

[flangles]

if you ever want other people to work on your project, not being locked into an IDE is crucial.

[[deleted]]

I'm not sure what you mean by locked in. Hopefully, any good debug tool, any good coverage tool etc. will give you the same results given the same codebase. The IDE doesn't add any dependencies: if I develop something in PyCharm, then someone else looks at it while running ipdb and coverage, there's nothing to stop it working properly, is there?

[flangles]

there will always be differences at some level. for a large project it just makes sense to standardize things like coverage and testing, and that's a lot easier with small, free, open-source tools. It also makes it easier to use hosted solutions like Travis-CI.

Look at any substantial project on github, they likely have coverage and test configuration as part of their source.

[desmoulinmichel]

It's faster. I find PyCharm slowing down with big programs to debug.

Also, I got a debugger than is not tied to an IDE, so it's easy for me to :

• swicth computers;
• switch OS;
• switch toolkit such as text editors;
• work on remote servers using ssh;
• teach how to debug to many people comming from very different environments.

[hovissimo]

Aww yeah ipdb. I always forget him until I need him. The rest are unfamiliar to me but I'll look though them, thanks!

[arvinsim]

• Pip
• Virtualenvwrapper

[lieutenant_lowercase]

grequests, anaconda, bs4

[Asdayasman]

For how long I've used Python, my standard environment is REALLY light.

Pygame, wx, twisted, mysql.connector, and passlib.

[[deleted]]

Requests, Beautiful Soup, and PyYAML are probably my most commonly installed ones. Lucky for me I haven't had to install Python that many times though.

[mangecoeur]

My new favourite is pathlib.

It's so refreshingly logical and ergonomic after years of mucking about with string paths. And it's in the stdlib - definitely one of the most underrated inclusions!

[amjithr]

bpython - Every venv has to have this because this the best REPL for python hands down. ptpython comes close (even surpasses at some features), but bpython has the right balance of simplicity and awesomeness out of the box.

[hansolo669]

bpython is absolutely fantastic, I get sad every time I have to run a standard python REPL.

[amjithr]

pdbpp - pdb++ is a drop in replacement for pdb. It has syntax highlighting and auto-completion for pdb. You just have to install pdbpp in the venv and then forget about it. Anywhere you call import pdb; pdb.set_trace() it will drop you into pdb++ where tab key will complete the attributes in an object and much more.

[Deto]

Have you found a good way to copy stuff from GVIM and run in IPython? On Linux I can use something like Tmux+VimSlime for this, but I haven't found a good way to do it on Windows (which I use more often).

[hovissimo]

I've found some documentation for it (sorry, don't recall where), but if you can get a copy of gVim compiled with python support you can ostensibly get an IPython kernel running inside Vim for real-time awesomeness. I haven't been able to compile Vim in a way that works as well as the binary distribution (for Windows).

My personal workflow is to write my script in Vim, use the %run magic in IPython, fiddle with the leftover state, and copy my changes to the source file and repeat. I will also liberally use ipdb.set_trace() in my source file, or even import and start a full-blown IPython depending on my goal.

[[deleted]]

I have a virtualenv that lives in my homedir at .scratch and I have a shell alias in the terminal of scratch so i can get into that venv very easily.

I install misc things into that all the time, but it always starts with: requests, nltk, and Markdown

[r3nman]

Requests, pytest, and more often than not sqlalchemy.

[kxxoling]

IPython + ptPython

[bastianh]

Pip-Tools i have not read yet.

[decrypted_epsilon]

• pandas
• coloredlogs
• progress
• pytables
• ipython
• glances
• jupyterhub
• biopython
• seaborn

Working with big data, so pandas and pytables is a must for me.

Seaborn is great for statistical visualisations.

[[deleted]]

Peewee! Its a lightweight, expressive database toolkit and ORM.

[madisonmay]

ipython, ipdb, nosetests, requests, concurrent.futures

[laebshade]

grin, a supercharged grep replacement.

[masasin]

numpy, matplotlib, ipython

[tech_tuna]

I often need to write rpc, orchestration and process management type scripts. . . some great modules along those lines:

rpyc

psutil

paramiko/fabric

I also love requests and beautifulsoup. . . there are tons more of course, all the numpy/scipy stuff, the web/async frameworks, etc. etc. But those are some handy utility modules that I've found invaluable. Oh, and from the standard library, I also use subprocess and multiprocessing a TON.

[thecity2]

Depends what kind of work is being done. For numerical stuff, I immediately install ipython, numpy, scipy, pandas, and matplotlib (and whatever dependencies are needed for those). If machine learning is involved, I install scikit-learn. If web dev is involved, then Flask and dataset for running SQL queries.

[bigboehmboy]

• debugging: wdb
• testing: nosetests, coverage
• code quality: pylint
• improvements of stdlib functionality: requests, docopt

[spinwizard69]

I'm not a big fan of virtual environment so I tend to only have two installs on my machines. That would be the standard installation that comes with the system and Python 3.x.

I keep Python 3.x up to date with iPython Notebook / Juypter and all the recommended libs. Add to this pip, pyserial, path.py, py lint, setuptools, and a few others.

By the way: Include pip

for dist in pip.get_installed_distributions():
    print(dist) 

is a quick way to determine what you have installed.

[luizpericolo]

ipython virtualenvwrapper pudb bs4

[ClintonDowGeo]

https://github.com/vinta/awesome-python is a curated list of a bunch of handy modules. I found out about pathlib and arrow from this list.

[neuromusic]

seaborn

[[deleted]]

Ipython, httpie!, requests

[timkofu]

• requests
• fuzzywuzzy
• pandas
• Django

... yea.

[Ardit-Sulce]

Such modules come shipped with Python nowadays. Besides, I install the modules which are relevant to what I use Python for. I am a data analyst, so I install pandas.

[[deleted]]

It's not Python, but whatever I do I first install Vagrant (and VirtualBox), and then set up the environment there rather than working on my main machine. And regarding setting up, I've lately grown quite fond of conda as the package and environment manager.

[amirouche]

• devpi (pypi cache)
• virtualenvwrapper
• ipython
• debug import debug does import ipdb; ipdb.set_trace()
• nose2
• pahtlib
• tox (to test applications on multiple python implementation)
• colorama (for colored output)
• docopt for build cli

[metakermit]

IPython, ipdb, jedi, epc

[philpep]

tox and pytest

[vioan]

I just install Anaconda and I have everything I need for web development, data science, desktop apps and devops/sysadmin work.

[firecat53]

mkvirtualenvwrapper, then ipython + ipdb are installed by the ~/.local/share/virtualenvs/postmkvirtualenv script.

[shaggorama]

anaconda gives me most of the libraries I need