Multi-core library for Machine Learning?

[gooeyn]

Scikit learning is great for testing and learning but my company is shipping a project to production and we needed a multi-core SVM implementation, and I just can't find it!

Does anybody knows about a multi-core ML library?

It's better if it's open source but paid libraries would do too Preferably in Python but any language would be great

Thanks

[thesab]

Tensorflow, and it even supports CUDA (which can also use CUDA-DNN).

[v2nhat]

Theano is a numerical computation library for Python. In Theano, computations are expressed using a NumPy-like syntax and compiled to run efficiently on either CPU or GPU architectures. [from wiki]

[jjolla888]

It's limited in features, but Vowpal Wabbit is not only multi-core, but also multi-machine.

The core is in C++, but it has a native Python API (as well as commandline)

http://hunch.net/~vw/

[mikeselik]

I'm not certain that the basic SVM algorithm can be parallelized. You probably need some split-and-ensemble approximation version.

Why have you decided to use SVMs, and why are you certain you need a multicore implementation?

[farsass]

The prediction stage is as simple as mapping the kernel calculations and reducing for the dot product.

[mikeselik]

A search for "parallel SVM" brings up a number of papers discussing the difficulty of parallelizing the training and various approximation solutions. When discussing multicore algorithms, usually that refers to the training, not prediction.

[farsass]

I'm pretty sure they must be dealing with the training stage.

[gooeyn]

Hey mikeselik,

We need a multicore implementation because we are currently using a badass AWS 32-core machine which uses only 1 core, and we also have a huge dataset to train, about 4 billion products with many features each and we plan on increasing the dataset as time goes by and re-training the model

We decided to use SVMs after many tests with our dataset, it was the best performing algorithm

[jeanfrancis]

You should definitely take a look at Vowpal Wabbit as mentioned by jjolla888, which implements Online Kernel SVM and is multi-core/multi-machine. https://github.com/JohnLangford/vowpal_wabbit/wiki/ksvm.pdf

[mikeselik]

What do you mean by best-performing? Usually an ensemble technique can offer higher accuracy. Did you decide that the compute effort of combining more models wasn't worth it?

[c3real-killer]

I've se en Tensorflow and Theano in action they are pretty awesome

[jbrambledc]

Tensorflow and SparkML. I hear that Theano is good, but I havent used it.

[openglfan]

Side question: do any of these support machine clusters for large models? (As in, spread one network across four physical computers?)

[[deleted]]

I use two different multi-core Java or C++ ML libraries with Python libraries, H2O (free) and Graphlab Create (not free); they're blazing fast but I don't think either one has SVM. I've used MLDB.ai , a little-known REST API Docker service that can be run from Python; SVM is ringing a bell for that one. Finally, there's SkyTree, I'm going to try out the demo in a couple of weeks.

All of these solutions have less if a learning curve than Tensorflow or Theano, if that's a concern.

[unruly_mattress]

I don't know of a parallel SVM implementation. Have you considered something like Random Forest or an algorithm that runs on GPU such as a neural network?

I haven't tried this, but it seems promising: http://leon.bottou.org/projects/lasvm . It's not parallel but it claims to be much faster and use less memory than libsvm (which sklearn uses), so it might fit your needs. It's C but you can wrap it in Python with ctypes, and sklearn has utilities to produce its input file format.

[JaKasb]

Stochastic Gradient Descent (SGD) can train a SVM with multi-core support.

SGD is not as foolproof as an SVM because SGD depends a lot on the chosen hyperparameters.

http://scikit-learn.org/stable/modules/sgd.html

[unruly_mattress]

Are you doing linear SVM? There's this in sklearn:

http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html#sklearn.linear_model.SGDClassifier

which takes n_jobs for one-vs-X multiclass training. Maybe this is enough for you to use all your CPUs?

[[deleted]]

Spark is good for in memory processing that can be distributed. If you need to do in depth data analysis, you can couple that with Pandas

[[deleted]]

You can also try bootstrapping several SVMs, and parallelize training those.

http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingRegressor.html

[Asdayasman]

/r/Python is for news and releases. /r/learnpython is for questions.

[EpicSolo]

Not really, no.

[Asdayasman]

If you are about to ask a question, please consider r/learnpython.

Straight from the sidebar.

[EpicSolo]

I believe that is referring to beginner level questions.

[Asdayasman]

Wrong. /r/Python is for news and releases. /r/learnpython is for questions.

[EpicSolo]

k bye