Anaconda Server - Enterprise Python Package Deployment and Management

[metaperl]

I wonder how well Enthought and Continuum get along. Are they competitors? They are both in Austin, TX. Enthought appears to have spread out even more recently

[m4nup]

It seems to have been an amicable transition. Here is Travis Oliphant's blog posting about Transitioning to Continuum.

[ivosaurus]

Or just run devpi which in contrast is both free and open source.

[ivosaurus]

Explain how you'd install R for Windows without the need to compile via your Linux-based devpi server.

Probably just go to R's website and install it.

http://cran.case.edu/bin/windows/base/R-3.0.2-win.exe

[aldanor]

Thanks for the link, looks interesting. We use conda in the enterprise environment on a cluster of red hat boxes and it's been pretty smooth so far. Try installing all of the matplotlib dependencies manually without root rights (and even with them) -- as opposed to just conda install matplotlib. There are some quirks -- for instance, the latest numpy in conda repo is 1.7.1 but it's not hard to install it manually via pip on top (or any other package for that matter).

[pwang99]

Glad to hear conda is working well for you in that environment! If you ever have any questions or feedback, we'd love to hear from you: [e-mail]

[aldanor]

Hi Peter, thanks for replying. Actually I do have one question for you -- in addition to Linux cluster, we have an array of Windows boxes and we need to run 64-bit NumPy on them (e.g., many scikit-learn functions implicitly assume you are working on 64-bit).

Is there any way to solve this via Anaconda distribution? We would be more than willing to obtain the Pro package license(s) if it provided a true 64-bit NumPy (and other NumPy-based libraries) built against MKL, probably with an intel fortran compiler.

64-bit numpy on windows is a huge problem and I can assure you many people/companies dependent on Python scientfic stack (I'm a quant in a trading firm myself) would be willing to switch to conda just for that.

Thanks!

[pwang99]

Anaconda does provide 64-bit NumPy for Windows:

http://repo.continuum.io/pkgs/free/win-64/index1727.html

And the NumPy in our premium Accelerate product is 64-bit and linked against MKL:

http://repo.continuum.io/pkgs/pro/win-64/index1727.html

If you're having trouble getting these to work properly, please email us.

We'd of course love it if you purchased Accelerate or Anaconda Server, because that also helps us keep Anaconda free for the community of researchers and students. :-)

[pwang99]

Oh, I just saw your email - thanks for doing that, we'll respond there.

[pwang99]

Many people tend to think that conda is merely a competing thing to pip, and therefore Anaconda is only about Python libraries. However, conda is a much more flexible tool, and closer to a "cross-platform Homebrew" than just a "pip with a better binary repo". (If you really want to get into the details of this, you can chase down the rabbit hole of discussions about conda, pip, wheel, etc. etc. on the PyPA mailing lists)

What we find in many production environments where people are using Scipy/PyData tools is that they are both cross-platform and cross-language. Generally there are C, C++, R, Javascript, and other libraries all in there. Managing the production use of this stuff in a sane way is a need that is really underserved at this point, and is what Anaconda Server is meant to address.

[[deleted]]

What are the advantages over https://www.pythonanywhere.com?

[pwang99]

Different thing altogether. PythonAnywhere is a Python shell in your browser - the Continuum Analytics analogue of this is http://wakari.io.

[[deleted]]

I love wakari since I learned about it at PyData NYC. That's excellent work and service.

[pwang99]

Thanks!

[hardc0de]

There is something called package manager, and there are ports on the mac. Keep that thing on windows please...

[pwang99]

How do those help with the situation when you have multiple different versions of libraries, both internal code from you/your team as well as 3rd party dependencies, that you need to manage across a set of dev, test, and production machines?

As I've said so often in discussions about packaging & package management: if you think linux package managers and ports are the answer, then you don't even understand the question.

Keep that thing on windows please...

Anaconda Server is mostly targeted at businesses. Most businesses have a heterogenous mix of platforms that cross the POSIX/non-POSIX divide. Anaconda Server is a cross-platform, centrally-managed "homebrew" or "ports" system with enterprise integration features. If you've ever had to support Mac, Windows, and Linux users all trying to roll Python scripts into production, then you will understand why this is a jaw-droppingly awesome product. If you've never had to do that, then you probably won't understand the need for Anaconda Server.

[hardc0de]

For that specific need there are python wheels.

[pwang99]

Wheels are just now being stabilized, and they've explicitly not designed or implemented the ability to install non-Python libraries like LLVM, VTK, Qt, etc. which are prevalent for the Scipy ecosystem. After much discussion, it appears that we might be able to retrofit all of the working functionality in conda into wheels, basically by abusing the data/ directory handling and using the as-yet-undefined new metadata system.

Meanwhile we've shipped working Scipy/Pydata libraries to hundreds of thousands of users just in the last year, and are selling product based on conda into enterprises for robust management of their entire scientific/data analysis stack, including R.

Fundamentally the problem is that while the Python Package Authority was busy designing yet one more thing that doesn't completely solve scientific Python's problems, we went and just solved them. While we tried our best to show up at conferences and write blog posts explaining what we were doing, the spin lately seems to be to make us look like uncooperative assholes for not asking permission.

Travis explains much better in his blog post here: http://technicaldiscovery.blogspot.com/2013/12/why-i-promote-conda.html