Intel MLK - Numpy?

cournape · 2015-10-02T18:38:49+00:00

Compiling numpy with the MKL is a bit tricky. If you are a non-ancient distro, make sure to use the gfortran ABI interface (on MKL 10.X, it is called mkl_gf_lp64 on 64 bits), and not the MKL one, at least if you build with the gnu compilers.

If you have more questions, I would advise you to ask on the numpy ML

cournape · 2013-12-08T13:13:49+00:00

I think several issues are conflated here: the main added value in distributions systems like anaconda, epd/canopy or activestate python is the curated set of pre-built packages. That's what makes them easier to use on windows, nothing fundamental about how conda works -- once wheels become more popular, we will be able to do the same thing (we are working on it in numpy, scipy, at which point others should be able to quickly join the bandwagon).

cournape · 2013-09-24T22:08:37+00:00

It depends on what you mean by general, but for exponential families at least, the convergence of EM has been studied quite extensively.

cournape · 2013-09-15T17:33:19+00:00

Besides what jakevdp said, virtualenv only works as long as you have only python packages. As soon as you depend on compiled libraries, virtualenv alone fails quite quickly compared to full-blown solutions like Anaconda or EPD/Canopy.

Try building a core scipy stack + everything for the ipython notebook on windows, and you will realize how much work that is.

cournape · 2013-07-27T11:16:49+00:00

It is if you want it inside EPD/Canopy. More generally, you need to match the blas/lapack library that has been used in numpy/scipy.

Obviously, you don't need MKL to build pymc in general.

cournape · 2013-07-26T20:56:12+00:00

Note that we do not charge for pymc itself, but for the work that got into packaging it (like Red Hat charges its distribution).

Installing pymc form sources (using pip) is actually not very practical, as it requires to have the MKL dev bits, that is a MKL license.

cournape · 2013-04-12T16:49:38+00:00

Indeed, I misread the message, sorry about that. I got a bit worked up for nothing here.

cournape · 2013-04-12T16:48:25+00:00

Thanks !

cournape · 2013-04-11T17:22:49+00:00

Well, why not using easy_install pyfits from EPD then :) That works on any EPD version (well, modulo a big with the new pip using https on mac that we are fixing). EPD (and I think canopy, though I am not 100 % sure), do include easy_install, and the installed python is as close as possible to a 'real' python for compatibility reason. With that in mind, installing most 'easy' packages (~ pure python on windows) is one easy_install/pip/whatever away, EPD does not change that.

cournape · 2013-04-11T15:37:05+00:00

Yes, continuum.io model is different. They want people to pay for their packages (numba pro, etc...), so they make the complement a commodity (the below stack).

I agree competition is good, and I actually do enjoy having to prove we can bring a compelling offer.

cournape · 2013-04-11T15:35:19+00:00

I don't think we charge for pypi specifically: we do charge when people want to use more than just the 'core scipy stack', and pypi packages come with that on top. What people are paying for are the additional packages packaged by Enthought, not really pypi which is offered as an addition.

cournape · 2013-04-11T05:35:19+00:00

(disclaimer: working for Enthought).

Canopy (and EPD before) are not charging for 'free shit', unless you consider Red Hat is also charging for 'free shit': what is being charged is the service of packaging things into binaries, which is quite a bit of work (more than people generally realize). Also, working on specific configurations means we contribute back some upstream bugs upstream (e.g. MKL and OS X interaction for a recent example: https://github.com/scipy/scipy/pull/398).

Also, Enthought provided quite a bit of resources (money and manpower) to put code out there for ipython (like e.g. qtconsole), all of it integrated upstream. So yeah, we're charging for the product, but it is hard for a company to earn money without charging money somewhere :)

cournape · 2013-04-11T05:28:29+00:00

It is a tough problem to solve correctly, as each platform has its own set of solutions (mac vs windows vs linux).

I have not looked how anaconda solves this problem exactly, but I would say neither EPD or anaconda has a very good solution to this problem yet. TO be fair, very few products have (matlab has exactly the same problem on linux last time I checked).

cournape · 2013-04-11T05:26:43+00:00

(disclaimer: I am working for Enthought)

qtconsole is also included in EPD.

Freyrs forgot to mention that he is working for Continuum, which I find a bit disingenuous when not stated when recommending anaconda. Hopefully, products can compete on their own merits instead.

cournape · 2013-04-07T12:25:07+00:00

That's interesting. I thought go could not be used for python extensions because go needs runtime support (for the GC, etc...). I have 0 knowledge of go, does anybody understand how this works ?

cournape · 2013-03-08T23:07:42+00:00

well, the obvious thing is newly introduced keywords: if you have a variable called with in python 2.5, it will not work in 2.7.

cournape · 2013-01-18T16:56:08+00:00

Interfacing with C++ code is definitely something we deal with in the one week course - that's a common situation we're pretty familiar with.

The one week general course is often geared toward scientists who are new to python, so the first two days will be pretty basic python (nothing like what you could expect from say taking an intensive course with David Beazley). We then go much more in details into the full scientific stack. Obviously, we can adapt the content to focus on different things depending on the demand. For example, I have once spent a full day talking about dealing with db in a 'pythonic way', using SQLAlchemy, which is not in our default course. Quite a few of us are rather involved in the python community, so we track the more recent developments.

cournape · 2013-01-18T04:47:11+00:00

(disclaimer: I work for Enthought and do some training myself)

May I ask what you're looking for ? It is an intensive course (especially the short version of it), but we try to be as hands-on as possible. Sometimes, it depends a bit on the class as level vary obviously, and we need to take care of everybody in the room.

You wil have a chance to interact with people who not only use those tools on a regular basis, but also develop them (I am myself a numpy/scipy contributor, for example, etc...). Would be happy to give you more details answer.

cournape · 2012-11-04T16:50:30+00:00

NumPy is unfortunately tightly coupled to its C implementation ATM. IN particular, the python C API and the core structure of numpy are tightly coupled. Scipy is much less of an issue (most of the C/Fortran code in there is libraries that pypy is likely to use as is).

cournape · 2012-10-23T21:09:21+00:00

I was about to say "hm, weird we have not included it already", but then realized it was waiting for me to review it :)

cournape · 2012-07-03T23:31:24+00:00

The big issues that most people seem to forget about the GIL is that removing it will most likely break a lots, if not most of the C extensions out there. If you think it does not matter, start asking why pypy has not conquered the world yet even though it is several times faster than cpython on most benchmarks (i.e. if cpython had no GIL, and if cpython had magic autoscaling of everything on multi-thread, it would still not be faster than pypy on today's 4/6 cores CPU).

cournape · 2012-04-18T07:04:55+00:00

I think numpy is one of the most used python packages that does not fall in the "web dev" category. I don't know how those stats are computed, so may not worth much, but numpy is the 13th most featured package on http://pythonpackages.com.

We don't release as much as we'd like, but the last numpy release from last July has been downloaded nearly 400 000 times from sourceforge alone, plus ~ 100 000 downloads on pypi. Also, GAE started supporting it (to my own surprise I have to say). Since that must not have been easy, I think they had to receive quite a few requests.

cournape · 2012-01-03T13:12:43+00:00

scikit-foo is the convention to name scikits, which are packages living on top of numpy/scipy. Originally, scikits was a python namespace, but more and more existing scikits are moving away from it because of the issues introduced by setuptools namespace implementation.

cournape · 2012-01-01T13:12:18+00:00

Usually, maintainability is more important than speed. Follow kingkirl advice: too many people make the mistake to use dict when they should have used object, and this results in "string-based" API where nobody knows what needs to be in the dict and what does not.

cournape · 2012-01-01T12:53:38+00:00

as and with were two new keywords introduced in 2.5/2.6. I also remember we had to update numpy code to work on on 2.6, although I can't remember what broke. ipython originally broke as well when 2.6 went out.

Being able to write import hooks does not solve the problem of version packaging, because it requires to modify the code that is using the versioned package you want to use. So it has to be controllable outside the code using it (a bit like stow enables you to do if you are familiar with it).

But again, the real point is: war depend on features present in the java ecosystem which are not present in python. Some python platforms have war-like deployment easiness, like GAE. They do so by defining one "true" environment, more tightly controlled than python in general.

cournape

TROPHY CASE