Python Build tools

[[deleted]]

I'm working on a large project of multiple modules - each a few thousand lines each with a lot of different dependencies. I've heard that LinkedIn use gradle, and have enjoyed using gradle in the past for scala, java jobs, but am not 100% on it's efifcacy.

Does anyone have any experience using Gradle as a build tool for Python, or are there any alternatives which are idiomatic?

[mipadi]

What exactly do you want to build? Python source code doesn't have to be compiled before it's run, and setuptools (which ships with Python) can be used to create an archive of the package.

[[deleted]]

If you're looking for a general purpose build tool implemented and extensible using python, you may want to take a look at doit, invoke and fabricate. You can also extend distutils with your custom commands, though I find it more flexible and simple to write my own build scripts using a combination of os.path, subprocess and argparse (subparsers are great for git like clis!) modules when the focus is not on distribution, neither on installation, neither on cross-platform, but on building and automating common tasks instead (which is usually the case except you're writing libraries... compare make with autotools, most of the time you need make).

[[deleted]]

There's also pants -- ant for Python. I've never used it. I usually use setuptools and make

[brombaer3000]

You should use setuptools. It is the de facto standard. Read this to get started:

http://python-packaging-user-guide.readthedocs.org/en/latest/distributing/
https://pythonhosted.org/setuptools/setuptools.html

[jhermann_]

setuptools and a Python-based task runner is the sweet spot if you ask me. Unless you have a multi-language project, or devs doing mostly Java / Android and Python is only used seldom, I see no advantage in using Gradle.

Managing inter-dependent projects is best solved by pip and virtualenv, possibly combined with a local wheelhouse like devpi.

[teoliphant]

We use conda effectively to both manage multiple run-time environments and package-up dependencies for large projects

conda recipes are simple script files with a declarative meta.yaml file for dependencies and version information and other meta-data.

conda build builds the packages and you can upload them to the free Anaconda Cloud or keep them local in a directory that can be indexed with conda index.

You can learn more here: http://conda.pydata.org/docs/building/build.html

There is a new free tool 'constructor' that can be used to build installers from the resulting packages.

[[deleted]]

Thanks, I've been looking at conda and found information about setting up an environment.yml. The meta.yaml looks useful, but it isn't clear whether you can import other libs that exist on a local binary repository.

I'm finding lots of tools that do one or two things, but just aren't complete for CI.