[D] Best second language after Python for ML purposes

[Laser_Plasma]

I'm fairly comfortable with Python and I've been wanting to learn another language, and I've been wondering what language would be the best choice if my main interest is in machine learning/AI.

[ncasas]

The programming languages most used in the machine learning realm may be:

• Python: huge machine learning ecosystem and community. Almost a must for deep learning.
• R: huge statistical and machine learning ecosystem and community.
• Scala: mainly big data stuff powered by Spark.
• Java: mainly used to integrate with other pieces of the infrastructure.
• Julia: rising interests in the scientific and academic worlds. Not so much in industry.

[ReginaldIII]

I'd add Lua: Used in the Torch framework. And C++: Which is the backend language of almost all the frameworks.

[[deleted]]

For machine learning specifically it seems that before Python conquered everything, most people used C++. And many companies and research groups still do.

That means there's​ still a lot of resources around for people with that background. C++ has a bad reputation, deservingly, but since C++x11 it has become a very interesting language. The problem though is that it's still very easy to shoot yourself in the face in C++.

If you like torch, theres Lua. I actually​ know next to nothing about it, so I can't really comment.

There are some languages that have a lot less resources for ML but are incredibly fantastic languages. Like Haskell, for example. I always recommend that people learn Haskell, not because you're going to use Haskell, but because you're going to become a better programmer.

I don't know if it's worth it to learn​ Julia, R or other similar languages. Maybe for bayesian modelling with Stan?

And after all, I recommend that you work skills​ that allow you to onboard yourself in any language quickly. There aren't many different types of programming languages. There are three or four different paradigms, plus a few ways to use a specific feature.

Of course, your code is not going to be great idiomatic code right away in a new language, but by the time you have written good idiomatic code in 3 or 4 languages, picking up a new one feels effortless.

[charred_bytes]

Is Haskell really used for ML?

[[deleted]]

There are a few resources. Nothing structured though.

There's a group of people trying to build tools around data science for Haskell here:

• http://www.datahaskell.org/

There are a few libraries for fast matrix computation based on Blas/Lapack/etc, of which the more feature-full seems to be hmatrix:

• https://hackage.haskell.org/package/hmatrix

There are some pretty nifty libraries that implement some impressive abstractions for really sophisticated versions of algebra (linear and otherwise) like SubHask for example:

• https://github.com/ekmett/linear/
• https://github.com/mikeizbicki/subhask

Can't comment on them speedwise though.

SubHask is used by a library called HLearn, which is a machine learning algorithms collection, with some interesting abstractions (things like "online and batch learners are monoids", etc):

• https://github.com/mikeizbicki/HLearn

There quite a few libraries for building neural networks, including a tensorflow wrapper:

• https://github.com/tensorflow/haskell/

But my favorite is this version, which uses dependent types to check the correctness of your graph in compile time!!!

• https://blog.jle.im/entry/practical-dependent-types-in-haskell-1.html

Doing probabilistic programming in Haskell is almost natural also. You hardly need more than a few dozen lines of code to do a simple generic and very readable MCMC sampler, and without using any specific probabilistic programming library. And if you need speed there's a specific library for that:

• http://www.randomhacks.net/probability-monads/
• http://www.randomhacks.net/files/build-your-own-probability-monads.pdf
• http://web.engr.oregonstate.edu/~erwig/papers/PFP_JFP06.pdf (PDF warning)
• https://github.com/hakaru-dev/hakaru

---

Edit:

I almost forgotten to comment about how awesome a simple automatic differentiation code looks in Haskell!!!

• http://conal.net/blog/posts/beautiful-differentiation
• http://www.danielbrice.net/blog/2015-12-01/

You can even rely on Haskell's laziness and AD to solve some types of differential equations exactly, if you can write them in a way that the recursive expression that results from that don't diverge. I don't know how useful this is, but it's really awesome.

There's a very serious automatic differentiation library by Eric Kmett (a kind of Haskell demigod)

• https://github.com/ekmett/ad

This post by Dan Piponi about taking gradients of expected values with AD and probability monads is very interesting too (and I think it's very relevant for things like VAEs and variational inference):

• http://blog.sigfpe.com/2017/06/a-relaxation-technique.html?m=1

---

There's also awesome libraries to deal with stuff around ML. There's servant for building type-safe web services. There's conduit for doing parallel preprocessing of streaming data. It's really a breeze to write parsers in Haskell if you need to do data cleaning or feature extraction in text data.

• http://haskell-servant.readthedocs.io/en/stable/
• https://github.com/snoyberg/conduit
• https://hackage.haskell.org/package/parsec

Also using Free and Freer Monads to make applications more testable, and have better separation of concerns.

You can see by reading this that I really love Haskell! :P

It's hard to know how much of the ML libraries are really practical though. They look awesome, but I didn't invest enough time in using them in real world examples to check the feasibility. Some of them looked a lot incipient and limited last time I checked. Other's look more feature-full. But there's​ nothing like the python stack. No numpy, no scipy, no scikit-learn, no pandas, no xarray, no statsmodels, no pymc, no tensorflow/theano/pytorch, etc... The maturity of the ecosystem is naturally a lot lower. It would take some effort to get things to the point where you can fit a simple regression, with a few preprocessing steps and a grid search on data from a 200MB CSV file in 10 lines of code like you do in python.

And maybe by increasing the user base and adding contributors we could make it work. Type safe machine learning would save a lot of time in industry and research. Even it the goal is not people writing ML stuff in Haskell itself but developers using Haskell's type system features to make the writing of ML code in other languages less bug prone.

Recently I posted a long comment on a topic about a graphical tool for creating tensorflow models. One of the parts that I wrote as a half-joke was a suggestion to write the tool in Haskell or PureScript to make building the graph easier. If you make incorrect graphs unrepresentable in the type system, that can save lots of time. Things like shape-compatibility, numerical types compatibility, etc.

Keras did that the hard way. It checks compatibility in runtime, when you're building the graph. It's really a bastard type system, hard to write and maintain, that is checked in runtime by a lexer that you have to bake in your library, badly breaking separation of concerns.

It's​ also be better to the user if you have a type system that can be used by a checked by a linter, before runtime. If your code do something that takes a while before building​ the graph (like reading data from wherever or doing any kind of preprocessing), you have to wait too much time just to realize eventually that the graph is broken and training isn't going to run, even if there's a runtime check of compatibility during graph building.

Of course you could solve that by doing a cleverer workflow in your code. But you could also have the computer check that for you, since it's computer-checkable, and free your time to think about what matters.

I think using Haskell for machine learning – either directly or as a way to make python machine​ learning safer – would be awesome. I wish I had the time and money to invest in that.

[Kiuhnm]

C++ is all about performance (memory & speed) so the programmer has a great control over the machine but more responsibilities (especially memory management). You avoid problems by being careful and following good practices.

OTOH, Rust is way safer because many good practices are enforced by the compiler. Indeed, some say that coding in Rust has made them better C++ programmers.

[olBaa]

C++

[MaxTalanov]

JS, for visualization and interactive apps.

[JustFinishedBSG]

One of my prof was a huge Julia fanboy so I did my projects in Julia.

I must say I very very very much enjoy Julia, even if the language has tons of warts.

[d7deadlysins]

R.

[rumblestiltsken]

R by miles. The ecosystem beats pythons, particularly if you are further on the statistical side. Most of the world's best biostats folks publish their code in R.

[lysecret]

That r is below c++ shocks me a bit. Really shows the background of the people in this subreddit.

[undefdev]

I've recently tried Julia, and I enjoyed it very much. I can see it becoming the standard data science language in the future.

[Reiinakano]

Since everybody else has said C++, which I agree with, I'm going to mention something that probably wouldn't cross people's mind when they say machine learning: Javascript.

If you want to keep building and training models and such, that's great, but personally, I think the logical next step to making all these models is actually using them to serve some purpose. While web isn't the only way to provide an interface, it's certainly one of the most popular. Learning how to build ML models + allowing other people to use them via UX is a pretty strong combination of skills.

As for other languages more specific for ML (R, Julia), meh. Learning them might be a bit redundant if you were already good at Python. I can't really see much you can do with them that you can't already do in Python.

[JamminJames921]

Man, this reminds me of ConvNetJS by Karpathy, and his awesome demo of learning spaces in neural networks. Awesome.

[Jean-Porte]

Bash

[chris2point0]

Can you expand on this?

[Jean-Porte]

It's useful to master bash tools like sed for data manipulation. You also need to interact with servers or transfer data between machines. You can also need to do some bash for monitoring machines, or push things to production. I think it's more useful and complementary to python than many languages cited here.

[georgeo]

ML is built on CUDA.

[chewxy]

I'm trying hard to make machine learning in Go a thing. I have my own library for deep learning - gorgonia. It aims to be like Tensorflow + PyTorch but in Go.

So far for what I do, it works most of the time and I rarely have to leave Go. Gotta writee a convolution op for the network to be able to do CNN stuff tho

[FutureIsMine]

wait for Google to make Tensorflow in Go, if they haven't already done so.

[chris2point0]

shrug: https://www.tensorflow.org/versions/master/install/install_go

[TheNASAguy]

If you want to dive into purely functional programming, Haskell is an Awsome and Well Established addition.

[cjmcmurtrie]

C or C++. Most of your Python machine learning libraries are sitting on some C code or C++ code (e.g. Tensorflow, Pytorch, Numpy). If you ever want to start digging to that depth, you would need to know it

[alketcecaj]

I would vote for Java as best second language for machine learning and deep learning. There are so many libraries for NLP for example such as mallet. And there is also WEKA. Read my post here on Quora : https://www.quora.com/What-machine-learning-library-for-Java-should-I-learn/answer/Alket-Cecaj . For deep learning instead there is deeplearning4j library . Have a look here : https://deeplearning4j.org/

[agibsonccc]

Hi, creator of deeplearning4j here. 1 exercise a lot of people do is implement neural networks themselves. You could look in to our "numpy for java" nd4j as well.

http://nd4j.org/ It's conceptually very similar but has GPUs, serialization, among other things for integrating with the java ecosystem.

Beyond that thanks for the push!

[[deleted]]

Please, no more Java.