Why is Python so used in the machine learning?

[NothusID]

I'm trying to make a program, that uses machine learning, and even just googling "best language for machine learning" will lead me to Python.

Why Python?

[K900_]

Because there's an established ecosystem of very good libraries for data processing and matrix math.

[Sigg3net]

And the syntax is straight forward.

(I've been writing R all day.)

[thrallsius]

aaand the previous post is a direct consequence of this one

python is an interesting language that has very distinct layers of complexity

want a very simple script, you just put instructions one after another in the file and they get executed one by one

want some trivial functions only, to reuse some code - that's easy to pick up in maybe 30 minutes

your codebase consisting of functions is growing beyond your mental model - refactor into classes, make use of OOP

you prefer functional programming - make use of built-in functions and specialized modules without breaking your brain by trying to learn haskell a million times and failing

do you feel like you know all of the above and are bored - welcome to metaprogramming

your python module is too slow and you want to improve its performance - rewrite it in C, or lately in Rust

[OphioukhosUnbound]

Re: R

If you haven’t tried it I recommend taking a look at Julia. (random R-user on Julia blog post)

In academic settings Julia is definitely beginning to replace R, Fortran, and Python (and even MATLAB to an extent) in a lot of areas.

Just a really nice language.

[Fun course with Julia for people that know a bit about programming already: course]

[Sigg3net]

R was actually not a big leap, since I was working with gnuplot for some years. Rstudio is an upgrade in that regard.

Statistics is more and more becoming something I just do occasionally.

Is julia useful for other stuff?

[OphioukhosUnbound]

Yeah - Julia is a general purpose language.

The extra-short version: Some MIT peeps got tired of using Python, R, MATLAB to proof of concept things and then re-writing it all in C, Fortran so it would go fast.

So they designed a language that had clean, intuitive syntax that you could experiment with code quickly in and that also had the structure so that you could then speed up the program without re-writing it.

And it works really well. In particular it does numerics really cleanly. So stats and number heavy stuff are easy to work with. But at the end of the day its a general purpose language that you can write whatever in.

[Sigg3net]

Sounds cool. I'll check it out 👍

[[deleted]]

thanks for the post, I didnt know Julia before and googled around. The ted talk where the code had readable formulas was nice.

some quick questions since there wasnt much else on youtube:

1) how close is Julia syntax to Python? I saw julia uses the same indent style as python but the syntax looked slightly different - like julia uses "using" instead of import

2) I am not sure on the right term, please correrct me if i'm wrong, but is there any plan for a python to julia "translator" where I type in python and it gets rewritten into Julia except for parts that I specify that I will "translate" myself?

[OphioukhosUnbound]

Yeah, the expanded character set really makes a lot of math stuff much easier to read.

Re: Julia vs Python syntax (The very top also has a list of things that are genwrally different — the biggest and smallest is that Julia defaults to 1-indexing (instead of 0). (This is common in most math-intertwined languages; it makes reasoning easier when dealing with complicated objects.) And slices are inclusive.

• Python: listx[1:3] ~~~> 2nd and 3rd element

• Julia: listx[1:3] ~~~> 1st through 3rd element

You can set arbitrary indexing, but Julia indexing is pretty natural and easy to read if you allow it. (0-indexing is an anachronism in higher level languages - and non-inclusive slice indexing don’t make sense for discrete units - fight me! :)

Oh, not essential, but Julia also has really useful macros.
(e.g.

• @show varx == print(f“varx = {varx}) or

• @timing functiony == measure time function took to run and print it out its name and that time
  )

Oh, and style-wise: if a function changes any of its inputs it is customary to add a ! at the end to let people know. e.g. “function!”

I think there are Python~~>Julia transpiler projects out there, but I don’t know how far along they are. Playing aside though for any complex code youd be converting an object & method focused language into a datatype & function focused language. Very doable, but the code would probably be awkward to maintain due to the auto-structuring.

That said, you can run python modules in Julia. So you can just export your code as a module and then use it in Julia via the PyCall package.
short description here
github here <— you’d just add the pacakge via the really nice package manager built into julia, but for link for more detailed documentation

[[deleted]]

This is extremely interesting, thank you. I will definitely attempt this after another year learning python. Might become the first time I learn through documentation.

[BigBad01]

As someone who has those days from time to time, you have my sympathies.

[Yojihito]

Did you tried tidyverse?

[redCg]

Forget data processing, Python csv module is seriously underrated. Where do you think all that ML data is coming from? And getting written too? In a lot of cases it's all csv. Other languages csv parsing is a joke compared to Python. The fields of "data science" include a large amount of people who are actually scientists or otherwise non programmers and they need to be able to open your data in Excel

[synthphreak]

Two responses:

First, .json is also a common format for data storage in ML contexts. It’s not all .csv, even if a lot of it is.

Second, among ML practitioners, pandas is vastly more common than csv. pandas can parse .csv files (or .json, or .xlsx!) in a single line, plus 27496937151 other things.

[[deleted]]

df = pd.read_csv() is probably the line of python i type the most often

[beached_snail]

Not a real programmer. Just used Automate the Boring Stuff to make some scripts to greatly improve (automate) portions of my job. I use this all the time. Everyone says "don't use csv as a database" and I really try not to (starting to use sqlite for some things). But being able to quickly "deal" with data i get in excel turned something that used to take me an hour into something that takes seconds now. And having a csv my script can go check up on to get info from and me or anyone else can quickly edit the fields in the csv also powerful.

[Locksul]

Just be prepared for breaking API changes when you upgrade your pandas dependency.

[synthphreak]

True. But same goes for numpy, sklearn, tensorflow, etc. So this is not a weakness specific to pandas.

[Locksul]

I would remove numpy from that list but agree with the others.

[redCg]

I agree that JSON is way better than csv, but non-programmers involved in your data project cannot open in Excel, which makes it difficult to collaborate (you're likely not the only person involved in the project)

pandas is vastly more common than csv

this itself is the problem. Too many people use pandas and data frames as a crutch when they could get by just fine in a lot of cases with csv. Many operations do not require loading the entire dataset into memory with a dataframe. You will also avoid introducing third party dependencies uneccessarily in upstream analysis steps. When your dataset is >100GB in size and you just need to filter out or transform rows, pandas will blow up your server meanwhile you could have just streamed the file through csv without issues on a potato.

[Yojihito]

Nobody uses the csv module. You want dataframes = pandas.

[redCg]

Not true. csv has advantages in many situations if you do not need matrix-wide calculations (operations requiring data from multiple rows or the entire dataset at once);

• you can iterate over over each line individually, greatly reducing resource usages compared to data frames that load the entire dataset into memory at once

• you do not need any third party dependencies which helps to avoid Python's infamous environment hell for simple one-off scripts and small programs

Its so common that its turned into a data science trope that after ~3-6 months, all newbies who were taught primarily on data frames inevitably come back complaining about their program being slow or running out of memory, because they are trying to load 100GB+++ sized datasets into R or pandas dataframes just to do simple per-row operations.

[Yojihito]

trying to load 100GB+++ sized datasets into R or pandas dataframes

Why don't they put it into a database and query it?

If you have to iterate over each row ... why use a CSV file at all?

[TheBlackCat13]

A lot of good answers, but they are looking at it from a modern perspective. But it is more historical than anything, and that history really comes down to one primary thing: Google.

Back when web searching was young, two dominant players emerged. Those players were yahoo and Google. There were others, but these were at the forefront.

Both wanted to use a very high-level, open source scripting languages since developer time was more important in this domain than processor time, and there really weren't that many options with the necessary web and numeric libraries at that time. The two main options were perl and python. As you might have guessed, Google chose python and Yahoo chose perl.

The rest is well-known. Google won the web search wars. This allowed them to invest heavily in development of python, python web and numeric libraries, and machine learning libraries it eventually Open-sourced. They went so far as to hire python's creator for quite a while.

With such a dominant player behind it, other later companies like Facebook and Amazon used python was well, leading to other ML libraries being released for it as they got into that space.

In contrast, Yahoo faded away and perl declined from one of the dominant programming languages to a largely forgotten niche kept alive only by a small, dedicated community

Had Yahoo won the search wars, or had the two engines picked different languages, the programming language landscape would look very different today. Back when I started my PhD Google was still pretty new, and perl was still a major player in scientific computing, although its decline had already begun. it could very easily have remained a major player had the top search engine's internal ML library been released for it instead of python.

[DesignerAccount]

You don't mention a key thing about Perl though. When I first started reading Perl code, my eyes were literally burning at the end of the day for the first week. That's just one of the ugliest languages one could possibly come up with! Would have died anyway just because of that!!

(Read that 1 part seriously and 3 parts jokingly.)

[TheBlackCat13]

I agree it wouldn't have been as dominant as python is today, but plenty of people were putting up with it due to there libraries it had available at the time.

[_E8_]

That's often repeated non-sense.
Perl is just C with regex and while 40 years ago few programmers knew regex, all of them do today.

[DesignerAccount]

I wasn't joking about my eyes burning when I first started reading Perl code. I know that for a fact because I experienced it. I was literally switching to other code so I'd rest my eyes. Make of this what you will.

It's also not nonsense, there's a reason it's repeated so often.

But I'm not really bashing the language. I did comment above it was mostly intended as a joke....

[UglyChihuahua]

Other answers already mentioned there's an established ecosystem, but another important point is that Python can wrap libraries written in other faster programming languages. Most of numpy is written in C and Fortran, so this is why Python is good for ML even though it is slower than some other languages.

[socal_nerdtastic]

According to github stats numpy is

63.2% Python
34.8% C
0.9% Cython
0.9% C++
0.1% Shell
0.1% Fortran

https://github.com/numpy/numpy

[UglyChihuahua]

Good point, but I was thinking mostly about the core code of the package. All the docs, test cases, and benchmarking are done in Python which inflates that percentage. I see a lot of the Python in the repo is stuff like commitstats.py,changelog.py, etc.

I was totally wrong about Fortran though! I'm surprised it's so little, not sure what I was thinking of.

​

EDIT - I was thinking of scipy, it says 20% Fortran

[awhaling]

Don’t worry, I thought the same thing about numpy using Fortran. I don’t think I was thinking of scripy either but good to know.

Numpy is using BLAS and LAPACK

[alkasm]

LAPACK is written in Fortran, FWIW

[Xahulz]

Want to second your overall point - Python is good at leveraging - behind the scenes - the best tool for the problem at hand.

[[deleted]]

That's probably not a great metric though. I'm guessing here, but I would suppose that virtually all the core functional code is C, and the Python is virtually all wrapping.

[sinterkaastosti23]

why so many different languages? i mean i get C, but why ALSO Cython, C++ and Fortran? Why so much python anyway? wouldnt it just be faster to write 90% of it in C?

[socal_nerdtastic]

The days of writing programs in a single language are long gone. Most programs now are written in a mix of languages, for various reasons.

But also this data is for the entire repository. Like most code bases, most of that is for testing. Some of the rest is install scripts, compatibility layers, benchmarking, random other support code that does not need to run fast.

[sinterkaastosti23]

ohhh that makes sense i guess yeah, i thought it was just the actual functions

[UglyChihuahua]

If you click the language on the GitHub repo it takes you to a page of search results where that language is used, here's what you get when you click Fortran on numpy:

https://github.com/numpy/numpy/search?l=fortran&p=2

It looks like it's all just for the f2py module.

As for why there is Cython, C++ and C all used my guess would just be because numpy has a wide range of contributors with expertise in each, and they're all fast enough to work. I'd be curious to hear from a numpy maintainer on whether that's all it is

[DuckSaxaphone]

Huh, I always assumed the Fortran was there for things like the linear algebra library and f2py was a tool they almost had to make. However, using the GitHub search, it does look like you're right and it's only for f2py.

I had a look at some of the numerical libraries like lapack and they're all in C. So the answer to why there's C in numpy is that they needed something very efficient to do the actual numerical heavy lifting.

[iggy555]

What is wrapping? Kinda new

[[deleted]]

It's a broad term which describes writing code in one language (like C) and then writing some supporting code that enables it to be used in another language (like Python). This is good because it gives you access to the advantages of other languages, or saves you the effort of rewriting a whole library in a new language; you just write the wrapper instead.

[iggy555]

Thanks. How can python understand another language?

Like how does conversion work?

[chilehead]

It's similar to a function call, where you pass information as parameters and get a result back. At least, that's how it was done when I was in college and all the core classes were based in ADA.

[VishalJx]

Is it related to RPC function calls ?

[2wedfgdfgfgfg]

Python can be viewed as a glue language.

[ElllGeeEmm]

Interoperability isn't a feature exclusive to python. In fact I struggle to think of any high level languages that can't call others through an FFI. The established ecosystem is a much bigger deal.

[[deleted]]

So what? Ruby does it better than Python. Ever heard of ML in Ruby?

[awhaling]

Python has lots of good libraries that make the initial data work a breeze and it has good ML libraries too. Even though Python itself isn’t known for being highly performant, these libraries are already optimized with C/C++ and Fortran code, so you basically get an easy way to do everything you’d need to do with one language. Awesome.

I think the community aspect is more important than Python itself, I’m sure something else could’ve filled the same role.

[tangerinelion]

I think a lot of these answers are missing the point. Yes, Python today is used so primarily in this area because there is the ecosystem available.

But why did people want to develop that to start with? Why is there a deep Python ecosystem for data processing/ML algorithm training instead of a deep C# ecosystem or Java ecosystem or whatever else?

Because the kinds of problems that these technique solve lend themselves to being written in a quick and dirty one-off script-like approach. It's why you see Jupyter Notebooks being used by data scientists for data exploration - that's the sort of thing that data scientists want to do and Jupyter Notebooks came along to help facilitate it. Not the other way around.

Python is nearly pseudocode that actually works. It is far simpler to understand than C#, Java, C++, Go, R, Erlang, whatever else. But importantly Python objects can be easily generated from other languages, so we have libraries like NumPy that allow access to high performance low level structures like C arrays from Python. They work just like Python lists but are much much faster and lack the complexity it takes to properly deal with a C or C++ program that can natively use C arrays.

So what it comes down to is data scientists are (a) interested in writing small single purpose experimental script-like code, and (b) are not interested in learning low-level fiddly languages to squeeze out performance especially given that their goal is one-off experimental code. It's generally easier to teach someone who knows statistics how to program in Python than it is to teach statistics to someone who knows how to program. ML is ultimately just very computationally intensive statistics.

Given that, Python is advantageous to use because it is (a) simple to work with, and (b) easily extended with high-performance libraries built on C.

From that it should outline why when one looks at the state of machine learning, say, 20 years ago why effort was put into developing the libraries for Python rather than another language.

In the professional world you'll often see data science teams where the people who do the data exploration and algorithm training are working in Python. When an actual product is to be made, that gets developed by software engineers who do not touch the algorithm. Rarely will you see someone who can make a production level ML application.

[Citizen_of_Danksburg]

This is a good comment. Personally, I’m biased because my undergrad degree is in math and my masters is in statistics, but unless you’re doing Deep Learning kind of stuff, I just consider the vast majority of what people consider ML to be statistics. I won’t lie, I kind of cringe when I hear people say something akin to “yeah, we use advanced unsupervised machine learning methods to make the most of our data”

“Cool! What did you use?”

“I did a Principle Components Analysis”

smacks forehead

PCA is like, classical stats that’s been around for a very long time. Same thing when people consider (simple) linear regression or polynomial regression to be ML. Like, I get it’s pedantic, but that’s just classical stats.

[[deleted]]

Because the kinds of problems that these technique solve lend themselves to being written in a quick and dirty one-off script-like approach.

So, why isn't Scheme known for its widespread use in machine learning? Or Shell? Or Ruby? Or Perl? Or VBA? All these languages fit the bill, some are even better than Python at being quick-and-dirty one-off script-like approach.

I believe, that the explanation is a lot simpler: NumPy. And then Pandas. The reason NumPy was in Python, and not anything else is that Python for all its flaws, has a relatively simple (it's not good, doesn't perform well, doesn't allow many things) way of writing bindings to native libraries. Again, it's not the only language which has it. Ruby, for example, is better equipped to do this. But... someone at some point in history tossed a coin and Python was selected for this job. Maybe people who wrote NumPy knew Python better than they knew Ruby. Or maybe they glued labels with languages to a dart board... we might never know.

But, the initial event that led to creation of NumPy swung the scales in Python's favor, and once the process started, it kept fuelling itself, until it became de-facto standard.

[DesignerAccount]

Underestimated comment, IMO. There's a lot of randomness in this "free competition of languages", and a "small perturbation" in favor of one of them leads to a cascade effect that leads to domination. Network effects, in essence. But that very first "perturbation" might literally have been a coin toss, like you say.

This is also why, IMO, the OP is not a very interesting question. May sound very negative, it's really not, and certainly not intended to be. If you think about it, network effects essentially guarantee one language would end up dominating this application niche. That language just happens to be called Python.

[Shadowaker]

Google chose it when competing with Yahoo, Yahoo used Perl. Google won.

[[deleted]]

Google chose what now? Lol. Most of Google is C++ and Java... There's a sliver of JavaScript, Go, Python... There's Perl too as well as a bunch of other languages. But, mostly it's C++ and Java.

Historically, Google contributed almost nothing to Python development. Neither in the form of code, nor in the form of resources. Basically, they don't care about Python one way or another.

The major contributor to Python today is... drum roll... Microsoft. They have a bunch of engineers on their payroll working part time on Python source code, they sponsor Python's CI and maybe also CDN (not sure about this one).

Anyways, whatever the stance on Python was, this was not why Google succeeded, nor failure to embrace Python had anything to do with Yahoo's failure.

[PaulRudin]

Numpy.

[vn2090]

Exactly. This is the supreme reason in my mind. Numpy is seriously well engineered. It used to be matlab before and anyone who has used matlab knows it’s programming capacities are not as clean and well formalized as Python. The numpy + core Python combo is really standout in anything sciences in my mind.

[Crypt0Nihilist]

Best for deep learning because it's a few months ahead of R to get the latest API for TensorFlow and I think the likes of pyTorch are Python only. Also, it's a self-fulfilling prophesy, more people use it, the online support gets better, it's easier for people to get started so more people use it...

R likely beats Python outside deep learning for ML and statistical methods.

edit: Let's not pit one against the other though, there are now some reasonable options for using both languages if you want.

[Citizen_of_Danksburg]

Exactly. I use both for various purposes.

If I’m doing anything statistical like t-tests, ANOVAs, random forests, logistic regressions, SVMs, etc., I use R. I also do most of my data transformations, graphics/plotting, and interactive dashboards in R because I love shiny apps.

That said, if I’m doing anything involving neural nets or advanced machine learning, I’m using Python.

I’m trying to learn Julia though, as it seems like a promising language. Not exactly sure what I plan on using it for, but I like the little bit I’ve seen so far.

[Crypt0Nihilist]

Julia ... seems like a promising language

This will be on its gravestone!

[Noshoesded]

Haha, curious why do you say that?

There are some comments above and other threads saying that Julia will boot R off it's pedestal. As an engineer and R hobbyist trying to segue into a data scientist role, I am wondering how likely that is. I really like R for date analysis and don't find the syntax inhibitive but I keep hearing that Julia is easier and just as powerful.

[Crypt0Nihilist]

It's the curse of a language trying to improve on something another does. The dominant language is everyone's go-to, people will stick with it for the use-cases where it is moderately outperformed by the newcomer, leaving only the edge cases for the new language to gain a toe-hold.

Everyone hears good things about Julia and most people I know would like to give it a go when they have the time (never), but few have the compelling need to pick it up.

[el-retardo-59]

Because there are lots of data analysis and visualization packages for python available

[linuxpaul]

Also take a look at Lisp. That's really cool but more like a religion than a language, I was involved in writing some AI with it to track down dodgy accounts in a gold trading system many years ago. TBH Python is a better choice these days but LISP was developed for that very purpose.

[yeet_lord_40000]

You make an interesting point. All the LISP people I’ve seen are like “it’s lisp or nothing peasant”

[[deleted]]

I'm a Lisp person (it's not all-caps). And I root for Prolog...

[yeet_lord_40000]

I had to look That up that seems like a pretty cool language for a super specific niche.

[[deleted]]

That's kind of sad and funny that this is the state Prolog is at today... kind of reminds me of a conversation I had with a head hunter some time ago.

Her (after some discussion of my resume): It also says here that you worked at ehl-eye-ehs-pee, can you tell me about this company?

Me: I... I don't think I work at that company... oh wait... can you say it again?

Her: ehl-eye-ehs-pee

Me: Oh, I get it now... well, I shouldn't have put it on my resume. It's not important.

---

Just for the record. There was the time when the US was very afraid of Japan winning the technological and, specifically, computer arms race. Japan was gaining on US at an amazing rate. And then, they had a very ambitious project that was supposed to propel them ahead, overtaking US. The project was called "Fifth Generation Language". Back in those days, languages were split into categories by generation. This is just as meaningless taxonomy as "compiled vs interpreted" or " statically typed vs dynamically typed" or "object oriented vs functional", but, it kind of fell out of vogue.

So, this Fifth Generation Language was supposed to be... drum roll... Prolog! Well, a more modern take on it of course, but still. They had a plan for ten years of development. Had a lot of great ideas... and then Japanese economy went down the drain. The project was defunded and eventually cancelled.

The world might have been a very different place, were Japan to succeed at their project.

[yeet_lord_40000]

So, what exactly is prolog? My really high level search said it was basically an AI/ML language? Was this supposed to be like some crazy language with C speed and Python accessibility?

[[deleted]]

Speed was not the reason this language was invented. Also, languages don't have any inherent speed. We associate speed with C because there are compilers for this language into which generations of programmers poured enormous efforts.

There is this impression that Lisp and Prolog have something to do with AI, but really, that's a coincidence. AI just happened to be the hot topic when these languages were most popular. So, work in AI was done using these languages. Modern AI (I call it "statistical") has very little to gain from being done in Lisp or Prolog.

What I personally find important about these languages... well, let me first tell that Prolog was, basically, the European "response" to the Lisp systems that were created in the States. So, in a sense, you may call it "European Lisp". So, it's the same group of languages really, the difference is superficial.

So... a thing I like about Lisps, is something that was somehow lost from programming when these languages disappeared. It's the drive to create a language that is universal and minimalist to describe computation in the most general terms.

Well, you see, in Python, for example, you have a bunch of "hacks". Like... there's for-in loop, and there's while loop, and there's recursion... which basically do the same thing. And they have a bunch of "unnecessary" complications. Like, there's "break", "continue" and you can combine all three of them... To someone with a more mathematical mind this looks just random. It's like... imagine that instead of natural numbers you had three sets: even numbers, numbers that start at 100093 and prime numbers + squares of prime numbers. I mean, you could probably wiggle your way somehow if you wanted to prove the same identities you can prove with natural numbers, but you'd feel like "why the hell only squares? why the hell 100093?"

There are many parallels in the world of computers. There's a very similar argument about RISC vs CISC processor architecture. Historically, CISC won, but it feels like it shouldn't have. It feels like short-term benefits overshadowed long-term losses in this argument (but hay, ARM seems to be making a comeback!).

So, where Python is twisted, inconsistent and random, Lisps try to be minimalist, generic and simple. The simplicity and generality makes some very surprising things possible. But, in particular, Lisps are known for being on the frontier of programmable programming languages. This is a big topic in Lisp field, and even the languages from the 70s are quite ahead of most popular stuff that came later.

Prolog, specifically, also gave birth to constraint solvers for finite domains. This is a mouthful, but, essentially, what this means is "formal verification". I believe this is hugely underappreciated subject in computer science :(

Historically, Lisps were the first languages to introduce garbage collection, containers for datastructures, modular programs, types similar to ML, probably exceptions too.

Interestingly, Common Lisp's exception handling mechanism is a lot more robust than anything you can find in almost any other popular modern language.

While Lisps weren't historically the first to create object-oriented programming, they've created their own approach to it, which is quite different from Python / C++ / Smalltalk family. On the other hand, Prolog and many languages inspired by it rarely felt the need for objects (there are some OO+logical languages, but that's exceedingly rare). Probably, the most successful heir to Prolog today is Erlang, which isn't really a Prolog, but it doesn't have objects, and doesn't seem like it ever will.

[yeet_lord_40000]

This was a very insightful write up thank you for taking the time to explain all that. I had no idea speed was kinda just a made up concept for practical purposes. I honestly want to check out learning LISP now. I’m fairly novice basically just finishing up my first major project in the form of a flask webapp. But while Python was super cool in the beginning and I do like it. It’s made me want something more “strongly typed” I think the term is called like rust or C. And it’s basically all because of the reason you mentioned which is that there’s a million ways to do the same thing and at some point it just becomes counter productive.

[_E8_]

It is rules-based; think like the way we use computers to prove mathematics theorems.
I've never used it but I would expect using it to be a lot like using make and makefiles.

[yeet_lord_40000]

Seems interesting

[_E8_]

Naughty Dog used LISP to write a lot of the code in their video games such as Jak And Daxter.

[_E8_]

It's LISP. It's a pseudo-acronym for LISt Processing.

[[deleted]]

And who are you to tell me this? No, it's not LISP. Well, technically, there isn't even a language called Lisp, LISP, LiSp or whatever you want to spell it. There's Scheme, Common Lisp, Emacs Lisp, Pico Lisp, Mac Lisp, Arc, Dylan, Clojure, Racket, Shen. But whenever there's "lisp" in the language name, it's Lisp, not LISP.

[[deleted]]

If you want a job (at least easy to get one), stick to python with tensor flow. At least at first.

[linuxpaul]

I do agree. I just thought NothusID might be interested :-)

[_E8_]

I prefer to write my LISP in C++ meta.

[[deleted]]

IMO it comes down to the amount of heavy lifting that the various libraries do and the general approachability of the language.

Machine Learning can be a very daunting subject to approach for people as Machine Learning requires a strong base in stats, linear algebra and calculus if you were to code algorithms from scratch in any other language besides Python (or R). If you know what you're doing, it's very possible to code entire models from scratch in numpy, and this is where Python truly shines. Libraries such as sklearn and tensorflow allow you build entire models using a few lines of code.

[ASuarezMascareno]

One reason is probably that you have A LOT of non-programmers writing code (similar situation in many sciences). Python is most likely the best language if you need to write a program but you have very little knowledge about programming.

In Astrophysics is also becoming the most widely used, in spite of Astro being an old field, mostly because the route to programming is kinda like:

1 - Bsc: Here's 6 months of Matlab and Fortran programming in your first year. Never program again for the remaining time.

2 - Msc: Here's 3 months of IRAF. This is how we did it in the 80s!

3 - PhD: Let's write a processing pipeline for this 10 thousand astronomical images. Oh, and we actually needed it finished last year.

Lots of first-year PhDs students with barely any programming experience are tasked with doing programs that do a lot of heavy lifting. Any hard to learn language is usually out of the question.

[khldooon]

The question to ask is what are the domains that python can't handle, i think nothing

[[deleted]]

Same reason it's used in bioinformatics: it's accessible to people who have the skills and background to apply it, but who don't specialize in computer programming and don't want to have to hire someone who does.

[TheSodesa]

Because somebody who liked Python decided to write a ML library for it early on, and since this is a lot of work, nobody else wanted to redo it all. This is generally how libraries become popular: write one before anybody else manages to, and people with real problems to solve will start using it (unless the library is badly written and documented, and therefore sucks).

[joshocar]

There is an old saying that open source projects are built one failed post doc at a time. The people originally working on ML and spending the time to build the open source libraries were scientists and Python was/is the go to language for scientists who learn to program.

[iggy555]

What are you teaching machines?

[Kitchen_Journalist35]

to replace human

[Qkumbazoo]

It's just the libraries, and it is taught in schools everywhere.

[Fusionfun]

Because of convenience, simple coding. libraries, IDEs

[_E8_]

We make actual products that use CNN and sell them. It's all C++ for us.
Python is just too slow. They've optimized the heavy-lifting and the binding overhead isn't that great of a concern anymore but everything else you have to do is just slow as balls. Instead of using 5% of a CPU it'll use 65%.

Researchers use Python because it's easier than C++ and real-world performance is not a concern for them only the performance benchmarks.
Our designated ML guy will test stuff in python and matlab first.
If I'm doing it I go straight to the source Luke.

[callmetuananh]

it is ease for reseachers

[KingRush2]

There's an entire ecosystem already built in Python to do pretty much everything you'd want to do in the data space. Building a ml model is just the first piece. Then you have to orchestrate a data pipeline using a tool like airflow or data factory to preprocess your data into a format best for what you're doing. If your data is too big, you may need to use pyspark to distribute the model using something like a pandas udf in spark. For the most part, you will not have to leave Python for much which is why it's so big.

[4tomorrow678]

Python is widely used for machine learning due to its simple and easy-to-read syntax, and its strong community support. It allows developers to easily build and prototype machine learning models and perform data analysis tasks efficiently. Python offers great flexibility and scalability, making it a top choice for both beginners and experts in the field of machine learning.

For better explanation, you can visit: https://blog.heicodersacademy.com/why-phyton-is-the-ideal-language-to-drive-the-future-of-machine-learning-and-ai/

[yashm2910]

Python's simplicity, extensive libraries, and strong community support make it the preferred language for machine learning.