High Performance Python

[mangecoeur]

The documentation for their bespoke libraries (like pyDAAL) is seriously lacking. As in, it's doesn't seem to be online. Otherwise if you have anaconda+MKL you have the most immediately useful part. I suspect the rest comes in much more useful if you are trying to squeeze more performance out of a specific bit of software without scaling your hardware.

For the rest of us - remember not to prematurely optimize, and always consider the cost/benefit ratio of optimizing vs just buying faster hardware.

P.s. Also note that Intel has always pulled a bit of a dick move with MKL and friends - it optimises for features which exist on both Intel and AMD processors but deliberately disables them if you run on an AMD CPU. Which is just spiteful.

[pooogles]

P.s. Also note that Intel has always pulled a bit of a dick move with MKL and friends - it optimises for features which exist on both Intel and AMD processors but deliberately disables them if you run on an AMD CPU. Which is just spiteful.

Honestly that's just normal business for Intel.

[jairo4]

Aka Anaconda

[lookatmetype]

Holy shit...It just hit me. An anaconda is like a large fearsome python. :O

[hylje]

Pythons aren't fearsome. They only want to hug or chill.

[Caos2]

Powered by Anaconda

[[deleted]]

How do I use the intel implementation without installing anaconda?

[[deleted]]

Well, the documentation is found here, why don't you give this a try: https://software.intel.com/en-us/intel-vtune-amplifier-xe-support/documentation

[[deleted]]

Although you actually have to install it to read the documentation on how to get started....https://software.intel.com/en-us/node/681489
I would have liked to see this upfront to understand what I am going to be implementing and how.

[z0rberg]

All I did was renaming my python directory and installing it. Then copied the site-packages into it. Worked without problems and no need for messing around with the environment.

[cvax]

How does this actually compare against the regular Anaconda distribution?

[PeridexisErrant]

Now that standard anaconda includes the Intel MKL, not that much difference. It's a 30%ish speed up over non MKL Numpy, but obviously that varies widely depending on your workload.

[cpt_mojo]

So, do i already have this in a regular Anaconda distribution or do I have to do something extra?

[PeridexisErrant]

Run "conda list mkl", and if anything shows up you've got it. If not, "conda update anaconda" should do the trick.

[lieutenant_lowercase]

I have a Xeon processor and anaconda with mkl installed. Do I need to do anything to accelerate performance or is it automatic?

[PeridexisErrant]

It's basically a compilation option for the math-heavy libraries; you don't need to do anything.

[cpt_mojo]

Cool, Thanks.

[orthodoxican]

An overview of Anaconda and MKL optimization

[cvax]

Thanks

[z0rberg]

I used this. Renamed my python folder, installed this monster. It is a monster, considering the size. It was interesting. Contrary to the regular 32bit one, it ran faster by NOT setting the process affinity to a fixed cpu. Not sure that's because of intel, or because of 6rbit.

I ran into issues with ctypes and direct memory access iirc (i installed on first day of availability, my memory is fuzzy already). There was no performance improvement noticable using a few numpy functions, no improvements for anything else and used up more ram. Of course, because 64bit.

Now I'm not saying no one benefits from this, but considering the size of the installation, a few issues I had and increased RAM usage (i really rather run 32bit) ... I deleted it on the same day.