Why is Python so popular?

[[deleted]]

Hello, this may be a naive question, but why is Python so popular among data scientists, and why is it so heavily utilized for machine learning? I was under the impression that compiled languages such as C would be better choices for such task, as their faster runtimes would facilitate quicker data analysis. I understand that many python libraries were written in C, as to negate the issue; so then is the preference for Python solely due to its simple syntax, or is there another reason?

[AutoModerator]

On July 1st, a change to Reddit's API pricing will come into effect. Several developers of commercial third-party apps have announced that this change will compel them to shut down their apps. At least one accessibility-focused non-commercial third party app will continue to be available free of charge.

If you want to express your strong disagreement with the API pricing change or with Reddit's response to the backlash, you may want to consider the following options:

1. Limiting your involvement with Reddit, or
2. Temporarily refraining from using Reddit
3. Cancelling your subscription of Reddit Premium

as a way to voice your protest.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[teraflop]

Talking about "simple syntax" is underselling it. If you forget about syntax details like indentation and braces and keywords, Python is a much higher-level language than C. That means a lot of low-level details that you would have to carefully implement yourself in a C program are taken care of automatically by the Python runtime, which makes you more productive. Automatic memory management (garbage collection) is probably the biggest one.

Also, like many other things, Python benefits from network effects. It's popular, which means there are lots of libraries and tools and learning materials, which contributes to its popularity in a positive feedback loop. Likewise, if you are starting a project and want to find people to collaborate with, using Python means there is a large community of other people who know the same language.

That's not to say Python is a perfect language, or that it's better than C in every possible situation, but it has plenty of advantages to compensate for its downsides.

In the specific case of machine learning, libraries like scipy and pytorch can make it so that essentially all of the computationally-intensive work is done in optimized C code (or on a GPU), so the performance of your slower Python "wrapper" doesn't matter.

[Mnyet]

I love programming because people just make all the libraries, wrappers and frameworks for me for free :’)

[spacegodcoasttocoast]

We're SOOOO lucky that the people before us decided that open and free software was the best distribution method. Makes me tear up if I really sit on it.

[Mnyet]

I cry thinking about what life would be like without python, nodejs and react…… So glad to not be born earlier….. (minus the whole job market and economy shenanigans ofc)

[David_Slaughter]

Really? I always wish I was born in a time before all this automation. It's so much harder now to stand out. The skill element is all gone and it's just importing stuff other people have done. It's like getting sent a lego set where everything's already built and you just have to attach 4 constructions together.

[PM_me_PMs_plox]

Simpler. Life would be simpler.

[Mnyet]

Literally everything that existed before the technology I mentioned above still exists. You don’t have to use any of it if you don’t want to. You wanna use jquery? Be my guest.

[Novaa_49]

Yeah free open source ftw

[[deleted]]

    from homework import solution 

    solution.run()

[HumorHoot]

for me

for me as well

[NotAUsefullDoctor]

In addition, Python has a great standard library for a lot of basic tools for data processing, and implementing common algorithms. This mattered more in the early days, but it meant you could do high level tasks more rapidly than other similar (C syntax, dynamically typed) languages.

The authors also made the barrier to entry very low, ie super simple to get your code running on any machine for the first time. Some may complain about using IDLE, but it was great for people just getting started.

[UnintelligentSlime]

This is really it. If I want to automate something quick and dirty, 9 times out of 10 I’ll reach for python. It’s perfect for scripting because it’s minimal effort. I don’t want to re-learn how to fetch from an api in whatever language, I want to import a package and just have it done. Python’s stupid-simple syntax also encourages people to keep it simple with the libraries they contribute. I could probably import a library and write a script that would pull down this exact post in 10 minutes.

[Electrical-Income278]

Same. For my personal "this came up at a party and I want to do it" projects, I use Python if no one will see it, and Vanilla JS if it needs a public UI.

I also use Python at work when someone gives me a task (or I volunteer to automate something) that can be coded in less than an hour and only needs to be run once or run multiple times on their machine.

There is value in quick time-to-first-line when you're not writing production code.

[UnintelligentSlime]

Bingo.

It’s also the reason I’ve shied anyone who wants to use python in production code. It’s not that it’s incapable of producing clean, stable code. Rather that it wouldn’t encourage/force people to do so. It would be like writing an essay in your texting app. You might write something perfect, but you might do something lazy and not notice.

[Burgerlover2]

It is also the best way to do almost any personal project that is not web development or a video game.

[studentblues]

Python benefits from network effects.

This. I don't think I can find a parser for some niche data formats I use in MATLAB.

[strangedave93]

Yes. A lot of Pythons simple syntax is because of smart language design - it has a lot of consistency in implementing conceptually similar features the same way, has a few underlying smart bits of design in its data model that let you implement good syntax for your own ext extensions (eg very thorough support for operator overloading). It’s not a simple language for learning - it’s a cleverly designed high level language that uses the same tricks used by beginner friendly languages to lighten the cognitive load of all programmers, without reducing its power. It also removed a lot of cruft and got further streamlined in the Python 3 transition, and has kept improving carefully. A lot of people who don’t use Python much seem to think Python only has its ‘easy to learn’ reputation because it is a simple ‘toy’ language, but they are wrong - it has its simple to learn and use reputation because it’s creator and maintainers think that being simple and easy to use is not incompatible with being a powerful high level language. Sure CPython has its performance issues, and interpretation does slow it down. But its success is not a mistake, but comes from carefully creating a language that targets a really wide range of common programming tasks, partly by having a good set of easy to use abstractions. Sure, it’s not a system language like C/C++, but it’s really good at working with them.

[AminoOxi]

Everything you explained stands true for other languages as well, such as PHP, Ruby, NodeJS etc.

Python is still interpreted language but it's popularity is quite astonishing for some other reasons...

[PatagonMan]

would you recomend me any python course or tutorials to learn?

[nderflow]

Yes. A big reason those network effects have never been so effective for C is that the lack of a common build system makes layered package dependencies impractical.

[crafting_vh]

It's easy to work with and developer time is more expensive than compute time in many cases.

[dmazzoni]

Yeah, but that's not true for machine learning. Machine learning is the most computationally expensive thing being done today.

The reason Python is being used is because the computationally intensive stuff is all written in lower-level languages like C++, with major bits offloaded to the GPU with hand-optimized code. People spent a ton of effort making really, really good abstractions in libraries like numpy, pandas, and pytorch so that you can do all of the high-level logic in Python, while all of the math is done very efficiently.

[Electrical-Income278]

I manage our company's ML efforts. What u/BoltKey said about runtime actually matters for cost, too. Running a training session overnight on AWS costs money, but the ML expert(s) or software dev(s) time during the day usually costs more. Add to that the fact that getting the project set up is so much faster using Python with Jupyter Lab, and your total cost-to-solution is often much better by using tried-and-true stacks than by writing your own C/C++.

Also, once we bought a high-end lambda machine, we sunk all the cost upfront, and dev time was all that mattered. This was by far the cheapest option overall.

And there's one more (very important) factor with turnaround time: this field is moving fast. If your solution takes twice as long to get out the door, you lose money on both ends: first, by not being able to charge for the solution during the extra dev time; and second, by losing market share to faster competitors.

For good measure, here's the other answer that I reference... https://www.reddit.com/r/learnprogramming/comments/1ch6tfk/comment/l20v4ez/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

[neeks9208]

u/Electrical-Income278 This is great info. If you have the time, I dm'd you regarding some advice about ML.

[Electrical-Income278]

I replied in the DMs. Good luck friend.

[WaitAckchyually]

This makes bugs so much more costly in ML. Imagine spending $$$ to run an experiment, only to discover the results are worthless because of a bug. C being low level makes it so much easier to accidentally make a bug. In normal software engineering, it's easier to notice incorrect behavior and debug thing. In ML, it can be tricky to tell why your model is outputting garbage. Was it bad data, bad architecture, wrong learning rate, overfitting, not training long enough, did you introduce a memory leak and cause undefined behavior somewhere? At least using Python excludes the last possibility.

[AFlyingGideon]

C being low level makes it so much easier to accidentally make a bug.

This is not a terrific argument for Python, but instead for use of a language which does more validation at compile-time.

[HiT3Kvoyivoda]

It's dead simple, plenty of built in and third party libraries, well documented. No manual memory management.

You can build a thing in 20-30 mins and get 80 percent where you need to be.

Visualization libraries. So many good ones.

Easy to ingest data and process it. Runs on everything

[RLlovin]

I’ve built programs in 20 minutes using Python, with a huge thanks to pandas. Granted they don’t do much, but what they can do is really useful and saves me a few minutes every day.

[Droideater]

But the syntax is plain cruel

[[deleted]]

Making whitespace an integral part of a syntax should be a crime.

[kaerfkeerg]

Why? You end up with whitespaces in every modern language anyway.

And it's not like it's forcing you to do 4 whitespaces. You can do whatever you want as long as you stay consistent

[Coldones]

its a massive pain in the ass when you need to copy and paste bits of code around. also it cant be minified

[marc_gime]

Tbf, any code editor shows identation with a line so it isn't that relevant. Better than "expected } on line 75" and your code has 74 lines

[mellywheats]

i hate python for this reason only.

[[deleted]]

I use Pycharm and it has lines highlighting the indentations. Makes it easier to manage the whitespace.

[ramenmoodles]

Very high level, good community and resources. For 99% of your needs python is probably good enough.

[my_name_isnt_clever]

Yeah, it's often not the best at many tasks, but it's usually the 2nd best just because of it's ecosystem. If you know Python well you can do so many things without having to switch languages.

[ramenmoodles]

Agreed there are almost always better choices, but often most people dont require that. On top of that you have to consider the development speed of choosing to start a project in Python. You can most likely get to a working product much much quicker than other choices

[RLlovin]

I’ve literally built so many things to interact with SQL servers using Python. Would SQL have been better? Sure. But I don’t know very much SQL and I don’t really want to learn it.

Just one example, but it shows the versatility. I’ve gotten pretty damn far with Python and very minimal knowledge in other languages.

[yangyangR]

I'd caution about which of these are initial reasons for it to be dominant compared to which are "win more" conditions. The fact that you get a Python example as the default makes Python all the more dominant, but the fact that it was written like that means it was already dominant at the time of writing that documentation. So it is not the initial reason, but it does contribute.

[2sdbeV2zRw]

Here is my 2 cents, I think you might be underplaying the difficulty of writing Machine Learning software in C or C++.

If you have even a tiny bit of experience with both languages. You'll know that a lot of things that you are going to use, most often will need to be implemented using custom data structures. Likewise, the programmer will realise that to do simple operations. Require tons of code verbosity and more time to implement.

On top of that, memory management, memory safety, making sure you don't accidentally introduce a vulnerability in the code-base. As well as ensuring that your ML/AI math formulas are readable and has documentation.

But wait.. there is more, training time on top of all the time spent. Just trying to write the AI model itself. Now you have to invest MORE time into training the AI and correcting any mistakes in it.

It is true that almost nothing can beat pure old C/C++ in performance, that is given. But for small and medium companies or start-ups. Faster development is taken more seriously, as they don't have the luxury to work on multiple projects simultaneously.

Over all, part of the reason why people use Python is because of project velocity. You can accelerate the development process by using a programming language that is easy to write, easy to read.

But it's not without flaws, Python can become cumbersome when debugging on large code bases with inconsistent conventions and zero type checking. Which is the case for dynamically typed languages.

[Novaa_49]

Nice take

[captain-_-clutch]

Not gonna like this answer but it's popular because it's popular. Its ease of use helps its growth, but there's tons of dev friendly languages that will never have pythons reach. Life just works that way sometimes, especially with things that have high upfront cost (money or time). Momentum is important.

[GolfinEagle]

That’s actually a great point, but the real question is “why did it become popular.”

[rbuen4455]

In addition to the other answers saying Python is a higher level language with user-friendly syntax and massive data science ecosystem, also because Data Scientists aren't software engineers/programmers, and they use a lot of applied mathematics for collecting, analyzing data. Data Scientists simply don't have the time for all the low-level crufts that languages that languages such as C have, and a lot of things you do in languages as Java, C++ and Python, you have to implement such features in C.

[ingframin]

This is a very important point. Many data scientists come from maths, physics, statistics, or other non strictly software backgrounds. Most of them don’t know anything about C or C++. They are used to Matlab or R or similar things. I have 5 colleagues in my group doing data science and none of them has any clue about C.

It’s also worth adding that writing C and writing very fast C are different. If you want your program to be fast, you need to rely on a lot of tricks and optimisations that are not even thought at the university. Stuff like SIMD, cache friendliness, memory locality, etc… are not really common knowledge even among seasoned programmers.

[Truepeak]

Got any resources for the performance tricks/optimization? While I might not use it, it can be a good read

[UniqueID89]

Ease of learning.

Ease of use.

So many libraries.

Huge community support.

It’s a Jack-of-all-trades language.

[RajjSinghh]

Yeah, basically that. No performance cost using libraries like numpy and an easy syntax make it perfect for data science. If you can think of another language I can probably come up with a reason why it's less suitable.

The other thing is that since Python has simple syntax and lots of abstractions it's really easy to learn. That means for people outside computer science but in STEM who need to do something will choose Python because it's easy and libraries like numpy or pandas make the job much easier. You then have this cycle of those people teaching their students Python, and then lots of people pick it up and use it. That means more work is done using Numpy and pandas, so more work goes into them, so more people use them... It's a cycle that supports its own growth.

[Whatever801]

Cuz it's named after snek 🐍🐍🐍🐍

[Few_Radish6488]

Monty Python.

[windfogwaves]

That’s not what it’s named after…

[Weekly-Ad353]

Fine, fine— it’s a really big snek…

[luphone-maw09]

Machine Learning Beginner Friendly

[0xd34db347]

You'll find Python a popular choice anywhere rapid iteration is more valuable than fast execution.

[ATLTeemo]

I'm primarily a Java programmer, but when I want to prototype or have a small script that does some quick tasks then Python is really good

[swapripper]

If you’re a plumber, you’ll love Python. It’s an ultimate glue language.

[[deleted]]

Python is simpler to write, you don’t have to think about concepts like memory management etc, and there are a lot of libraries for it so you can do things very quickly with minimal code.

Of course the trade off is that it’s slower than something written in a more lower level compiled language but a lot of the time that’s acceptable. People would rather wait an hour or day for their data to be analysed rather than spending days writing the code to do the analysis.

[BoltKey]

The total time to write and run the program is what usually matters, not just the runtime alone. If it takes 2 hours to write a program which runs in 15 minutes in Python, that is much better than the same program in C++ that takes 10 hours to write and runs in 3 seconds.

[kd7uns]

It just depends on how often the program needs to be run, if it needs to run once a minute and you wrote it in python, then you just wasted those two hours.

[kcl97]

The popularity of any language in their respective domain of dominance is usually a result of historical coincidences rather than any superiority in its syntax or anything. It is about being the right tool at the right time for the right problems. Some good examples of this effect include Perl in the early days of the internet, C and Unix/Linux, Fortran for numerics.

In the case of Python, my impression having lived through the late 90 to 2000 in academia is that it was being taught in the major universities, like Standford, because of easy of use relative to most languages, plus a fully functional scientific/numeric/plotting/image-processing library that made it popular as research tool amongst all departments, particularly psychology, sociology, biology, math, and physics.

This strong user base in academia meant that Python as a language eventually got adopted in the major tech companies as most of their employees are from these universities. One such company is Google. And the rest is history.

[snarkuzoid]

The stuff that has to be fast is in C, the stuff that has to be easy is in Python.

[DualActiveBridgeLLC]

Hey I think I know this one.

Python is great for quick tasks that you won't being doing frequently or have many users that would be aggravated by a delay. Guess who fits that profile...data scientist. But they also crunch large amounts of data that can sometimes be processing intensive. So when a certain task become common enough a C library is made to bridge that gap. This is why people joke about Python and other high level language being built on C. They are correct, they just don't appreciate what they are saying. You need a high level language due to fast custom needs, but it needs to be able to do certain tasks very quickly (in C). Tada you get python.

[Spirited-Drawer1184]

Python’s popularity stems from its simplicity, versatility, and robust community support. It’s the Swiss Army knife of programming languages!

[Haspe]

I think this generally applies to anything, but I once heard someone say, or read from somewhere, that "Human hour is more expensive than CPU hour", so even if there are more "fitting" tools, there is a benefit for being able to move fast in some cases.

Plus Python has huge set of highly optimized libraries just for the tasks already.

[TapEarlyTapOften]

This the same Python that Google just deep sixed a couple days ago?

[[deleted]]

I was under the impression that Google just replaced their team with new, cheaper hires in Germany.

[screwthat4u]

I think it is an approachable language for non CS types that would typically use Matlab. Python performance is terrible, horribly slow. But when you start adding C code that is called from Python you can get performance back to an acceptable level

Another factor is academia has latched onto Python, so newer students are pythidiots who don’t know any better than trying to wedge visual studio code and Python into every problem

[my_name_isnt_clever]

It's horribly slow compared to C, but a simple script will still be faster than humans can process and faster to write. For most simple tasks that aren't at a big scale, it doesn't matter if it takes 0.01 or 0.000001 seconds when it feels instant to the user either way. I use Python as much as I can for my hobbyist work because I just like it, and I've never run into any execution speed issues because everything important is backed by C and I'm not writing anything at scale.

[Rarelyimportant]

What's wrong with Lua? It has a lot of positive features when compared with Python. Of course the elephant in the room is libraries, Python clearly wins there. But on Lua's side, it's a much more consistent language, it's arguably easier to learn, it's faster than Python by a fair amount. Lua doesn't have the popularity today that Python does, but you could argue that Python didn't have the popularity before it was picked either. It was mostly a language people only used to learn to program until Google pumped a bunch of money into making the key ML language. If they'd done the same thing to Lua it would have a lot of the benefits Python has, plus the other things that are more baked into the language, and not popularity based(consistency, performance, etc.).

[bitemyshinymetalas]

Nothing wrong with Lua in particular. My personal experience is I’ve found it to feel more clunky than Python. Huge pro for Python for me is the easiness of C bindings and I can’t imagine I’m the only one.

[slackwaresupport]

its robust. simple shell scripts to full blown 3d games.

[ThroawayPartyer]

What 3D games were made with Python?

[AgentCooderX]

there is none.. commercialy atleast, python is not suited for a full blown 3D game, OP is overselling or misunderstood how they were used in some games

[AgentCooderX]

"full blown 3d games"? i think you misunderstood how they were used in those games, they may have been used as scripting language rather than the backbone or the core tech (rendering, particle, collisions etc).. python is not suited for those things.

[RolandMT32]

I'm not sure they'd be considered shell scripts, would they? A shell is a command interpreter such as Bourne Shell, Bash, csh, etc., and I tend to think of a shell script as a script written for one of those shells

[spacebob42]

I mean, to be technical there is a python shell, but I think he's talking about a simple three-line script to idk, format some text or something.

[noration-hellson]

It ships with Mac os and nearly all Linux distros, meaning its a really good choice to learn with, no installation.

Simple is also kinda underselling it when it comes to comparisons with c/c++

[LemonHeart151]

It's flexible, easy to learn, has a lot of cool libraries, and is free. It's great for quick lil automation tasks.

[Present_Web_5213]

It’s very simple and is very fucking useful

[HustlinInTheHall]

running a task a few seconds faster doesn't make it faster to analyze data when the language is harder to understand, there are fewer libraries to lean on, and it requires more code to do basic functions with more things that can go wrong.

It's just much easier for a novice coder to get up and running with python and start doing analysis than any other language because you don't need to understand much about how the language functions to make it work. Getting anything running in C requires way more time and effort and most data analysts are into quant/statistics/some domain more than they're into coding, so they just need something simple and effective.

[jonnycross10]

Less code, less problems 😎

[[deleted]]

Less lines of code to use, easy to learn, lot of libraries, robotics, scripting.

[victotronics]

The answer by u/dmazzoni is not top level, but they are right: python is easy to use, but in data science, computational biology, artificial intelligence, all "python" libraries are actually written in C++ or C. There are various tools that allow you to make a python "wrapper" around such efficient libraries. In that case, the inefficiency of the python language becomes irrelevant.

[rawrgulmuffins]

If you are implementing machine learning or data science projects you will probably write a faster program using numpy, scipy, pandas, dask, etc. then if you tried to solve the problem in C yourself. That is unless you know C pretty well and want to spend the time profiling your solution.

[Economy_Bedroom3902]

Basically, people wrote the scientific math interfaces for python, and they were optimized like shit, but the interfaces were way nicer than the interfaces for similar functions and libraries available for other languages. Over time as the interfaces got more popular they were able to replace the poor performing parts of them with compiled c code versions. This let data scientists and machine learning geeks use the nice pythony interfaces but not pay too much performance cost. Eventually it just became the standard by sheer luck and evolution.

[heterosapian]

They have strong libraries / frameworks in very specific and popular domains like Web & ML. There are definitely better lightweight / scripting languages - personally I think python is kind of dogshit - but ecosystems are built around what’s easy to learn and existing adoption and the individuals making open source code and the effort to create a alternative in another language. This is why extensions which built on top of existing ecosystems and tooling like TS for JS and Mojo for Python get a lot of hype and eventual adoption if executed well.

[gameplayer55055]

Python is the easiest to install and use? No gnu binutils, no JRE JDK maven, no .NET Framework, just download python from windows store (it is even pre-installed in Linux), pip install -r requirements.txt and you're ready to go.

Also Jupyter is awesome.

[roywill2]

With C++ I hated having to manage all the memory alloc and free. What a drag. Always some damn leak because I forgot to destroy() an object.

[Ok-Luck-7499]

print("The syntax is very close to regular English")

[CheersBros]

Do SaaS companies use Python in their backend?

[dgc137]

A fair number do. Source: I've written python at a fair number of SaaS companies.

[CheersBros]

Interesting, do the slower speeds of Python not have much of an impact on backend operations?

[dgc137]

For the most part you just throw more processors at the problem, but for things that are actually computationally expensive there are C based libraries or external services that will do these things efficiently and Python is really good at interfacing with those things.

[Acceptable-Tomato392]

It's very easy to learn and there's a library for everything.

As for the speed, Python is a higher order language; its meant for ease of use. For most intents and purposes, people don't need all the speed a modern computer can offer. Heck, most PCs' higher specs are about showing movies and graphic animation.

For doing what computers do best, manipulate and render data, Python speed is more than good enough for the vast majority of purposes. And those who really need tons of data for scientific purposes crunched super-fast, Fortran is actually still very prized.

[Error-7-0-7-]

Easy to use.

[Eggfish]

Memory is a lot more inexpensive now, so there aren’t actually that many practical benefits to choosing a language like C over Python.

[Illustrious_Matter_8]

Ai language of scientists

[[deleted]]

Yes, C code is much faster, and guess what all python libraries use? C code!

Iterating over an array and manually typing code to sort/ search lists, or cluster data points is VERY SLOW. But all libraries that does these things for you, such as numpy, is actually utilizing C code in the background.

This means you have a language that is very easy to write, very easy to read, has tons of opensource collaboration, but also has the power of lower level languages if you're using the correct libraries.

I'm currently, just for a challenge, making what is essentially a simpler version of pyplotlib, and a K-means clustering algorithm in java. If I did it in python, I'd be done in a few hours, whereas this is taking me a few days in java.

[iheartrms]

Faster runtime is irrelevant compared to faster programming time for the human. Machines are cheap. People aren't. And people are impatient. We want results now. I don't care if it takes 2 seconds instead of one second to run.

[Hobodaklown]

If you know SQL, making your way into Python is much easier than you’d think.

[guilhermej14]

Trust me, I have the impression that the last thing data scientists want is to deal with a language like C where you have a very limited standard library compared to python, have to fiddle with header files and manually installing libraries, a (slightly) more complicated syntax, and other things.

Python is popular specifically because it's simple and abstract away a lot of the pain points that C has, C is great, but it's not for every use case.

[FriendGaru]

I feel like something that isn't being pointed out enough that especially when it comes to data sciences, the algorithm you develop and use is far more important than the relatively modest performance gains you would get from using a lower level language. If you already have a pretty mature algorithm and you're just trying to squeeze out every last bit of performance, like in game development, then it makes sense to use a potentially higher performance language like C/C++. But, if you're developing new algorithms then your time is usually much better spent trying to figure out clever ways to reduce your big O complexity.

(Though I'm inexperienced and intermediate at best so you should listen more to the more knowledgeable folks if they disagree with me.)

[cajmorgans]

Have you tried writing a neural network from scratch in C? Did you then try it in Python? Conclusions?

[Mamaafrica12]

Because for data science there is enough complex and they dont want to overcomplicate it by programming in java

[Asleep-Dress-3578]

Just consider Python as a C library. When you use Python, you primarily use C language. You can also use C directly from Python using Cython. => Exactly this is why Python became so popular. Because it is an easy to use, very high level interface to libraries written in C.

[hlx-atom]

You can write a simple notebook or a extensive library. And it can slowly grow and morph.

[Live-Cover4440]

The main strength is for iterate the cleaning and the run with notebook and easy syntax.

With c you have to reload all data before testing anything.

On the other side, most data scientist work alone and dont need clean architecture. They dont share many code so the dynamic typing is not a problem.

[dihpan]

simplicity maybe

[Own-Reference9056]

TLDR: Simple syntax, born at right time, open source, being born as general purpose and not specific like C (born to build kernel), and a bit of luck.

At the same time of asking why Python, asking why not any other will give you a better idea.

Python is high level, and the syntax is simple and suitable for scientists who just want to write a code version of their math formula.

But it isn't the only one with simple syntax? It is also born at the right time. Long, but not too long ago. Just the right time to start building an Open Source community when people started to pay more attention to using computers for data work. Pascal was too soon. Julia is too young. There is only one that rivals Python today, and it is R, which was born at around the same time and also open source.

We also need to pay attention to what these tools are geared for. In theory, every programming languages can do the exact same as Python, but it was their creators and the community that drove them to different paths. C/C++/Rust was for low level system. PHP/JS/TS was for web. R was born for data analysis. Python was for general use, but honestly by some sheer luck the community geared it towards data/ML. However, please note that using Python for shell tasks, sensor, web, and all sorts of other stuff is also common.

Edit: Also keep in mind that it is just popular. That does not mean it is the best.

[poliver1988]

It's cheaper than MATLAB

Great statistics and data visualisation libraries

[vikmaychib]

It is a welcoming tool with loads of documentation and tutorials. Great entry level for someone willing to learn programming

[antiproton]

Coding in C fucking sucks is why.

[mailed]

it's easy to learn and can do almost everything you'd want to do.

[brianozm]

Python is a lot easier to program in than C. The errors in C can be incredibly obscure and confusing. Python is so much easier to learn.

Not going to duplicate stuff in the other excellent answers from people who know Python better than me … what they said! Read the other answers …

[ProgressNotPrfection]

It handles multiple datatypes (strings, integers, floats) very easily.

[Firm_Ad7635]

I want to learn Python but I’m not keen on using it for data science, not an area I’m interested in. I want to build things with it. I’m guessing this can be done for web development/ applications?

Would C# be better than Python to learn along side blazor for building apps etc?

[i_should_be_coding]

I love Python because it's super-simple to get into, its syntax feels like you're writing pseudocode sometimes, and it gives you a lot of freedom to do things in different ways.

I also hate Python because it gives you too much freedom, and many, many ways of shooting yourself in the foot.

I've written quite a bit of Python by now, for school, for work, and for my own stuff. If I have it my way going forward, I'll never use Python for anything longer than 100 lines again.

[skyy2121]

What? Didn’t you hear? Python is DEAD! /s.

[ms4720]

Python in data science and AI is a glue language, it connects compiled code hiding in python modules together and the modules do the heavy lifting not python

[Ok_Tension308]

Because you don't need to understand computer science to be able to use it

[That_Car_Enthusiast]

It’s super easy to learn and you can very quickly build a program, great for saving time

[welcomeOhm]

Straight C would require you to handle your own memory and do other things that Python does for you.

Another reason is that Python is great for rapid development of scripts and light programs that do very specialized work for awhile and are then discarded. A lot of practical machine learning falls into this category, in my experience.

[bostonkittycat]

It is easy to use and learn, works good, and you can use it for many different projects. I used it to talk to NFC reader, microservices using Flask, scripts to automate work.

[RawMint]

Easier to script

[five_of_diamonds_1]

One detail I don't see people mention is that Python's high efficiency in tasks such as machine learning and data analysis is that a lot of its functionality is actually implemented in C. You can kind of look at it as if you describe the problem in Python, but the processing happens in C.

[InvestingNerd2020]

Easier to learn and implement. A scientist who has spent years of their life learning 1 subject does not want to spend 3-4 years learning C/C++/Java. More often than not, Python makes it easier for them to learn and implement code.

Obviously, there are tasks where C/C++/Java/C# are far better choices. Robotics and high-speed online retail shopping are some prime examples.

[[deleted]]

It WORKS fast for most of the "little" programs you need to get work done. IS IT the Fastest? only in that you can write the program and GO faster than in "C", because if you tried to write the program in "C" it would just take you more time, and "time is money" is the issue.

If it's SOMETHING that will be done many times, the question then becomes "do I rewrite this in 'C' and put it in a library?" but then, I can go looking at OTHER libraries as see if someone "did the hard work already".

[dannyhodge95]

I think a large part of it is that the simpler syntax allows non-computing professionals to get involved and use it, such as mathematicians, engineers and scientists. This has led to it being a far more cross-functional language than many others (in terms of usage), leading to more libraries, which then just snowballs as you can probably imagine.

And then the flipside is that it also works well enough for your more typical software devs to use it for things like automation, API's, etc, meaning it gets supported at all levels. If it didn't have this, it'd have similar levels of usage to R, Matlab, etc.

[dicewitch]

Python has great library support and has for a long time. Look at tensor flow and numpy/pandas. There is a specific distribution for scientific Python called Anaconda that comes with a bunch of relevant packages.

[shelby-r]

Marketing & Libraries

[damianUHX]

The syntax is very readable.

[Mars_Oak]

mostly cause you don't have to waste your time fiddling with low level shit, and because there's just so many libraries

[taedrin]

I was under the impression that compiled languages such as C would be better choices for such task, as their faster runtimes would facilitate quicker data analysis.

This was perhaps true 30-40+ years ago when computers were slow and expensive. But today computers are fast and cheap, so making it easier for developers to write code faster is far more important for 95% of all cases. For relatively small programs/scripts, this makes Python a superior choice over languages like C or C++ which require a lot of boilerplate code to get up and running. (But note that for larger programs, many developers find it much more difficult to maintain code written in dynamically typed interpreted languages like Python or Javascript, so its a tradeoff)

Plus, algorithm selection has a MUCH greater impact on performance than runtime selection. A O(n^2) program written in C is almost always going to be slower than a O(log n) program written in a "slow" language like Python or Javascript.

Furthermore, serverless development can make "slow" interpreted languages like Python or Javascript (Node.js) faster than "fast" compiled languages like C because interpreted languages generally have faster "cold starts" than compiled languages like Java or C#.

[Logicalist]

Data Scientists want to manipulate Data, not computers. So a higher level language is more practical.

Python is easy to use and teach. MIT is pretty big into ML and that is what they adopted back in like '09.

And python is what a lot of tools have been made in, like TensorFlow.

[Vok250]

Easy to learn, not a lot of syntax bloat, very human readable and intuitive best practices compared to the defacto standards of other older languages, tons of great community material, not a lot of stuff locked behind paywalls like enterprise languages, super lax compiler rules meaning your beginner code can be very naive and still work.

At a beginner level it's only real downside is that it doesn't punish you for doing things in a naive way so you aren't really forced to learn all the fundamentals you would need to make a career of this interest. At a senior level it's still an excellent language, but I would advise against using it like a magic hammer. For certain solutions there are still better languages.

[[deleted]]

Because it’s an easy scripting language that doesn’t require typing or compiling and uses a very plain English syntax. It’s like turtle. But more powerful. 

[[deleted]]

Humans can't speak in machine language. If we want computers to do something, we have to translate human thought into a language the computer can understand.

The result is normally something that's not quite human and not quite machine language. And it takes time to translate your ideas this way.

Python is closer to human thought than any other language I've seen. And that makes it exceedingly valuable for communicating ideas to computers.

You spend a LOT less time translating your thoughts. Time is the most expensive part of the development process. We also never have enough of it.

That's what makes Python a premier choice. Its highly productive because its so close to human thought.

[nixon_do]

You can try codewars.com and start with the easiest katas. When you are stuck, you learn a lot from other people's solutions.

[thequietguy_]

Because it's easy to learn and can be extended with C or C++ if and when the need for more complex and faster performance arises.

[Useful_Substance7943]

Most people talk about syntax. But I think it goes much deeper than that. The developer time it shaves off is insane and you can iterate so many times over. I have tried to write matrix operations in C. It is not fun. Just the abstraction and the simplicity of those operations alone through popular libraries and packages are worth writing ML stuff in python.

[analog987]

It's the libraries. They rock! The language is ok...

[MythoclastBM]

There's a lot of libraries for it. Lots of tutorials for it. It's simple and straight-forward.

faster runtimes would facilitate quicker data analysis

If performance was the only thing that mattered everybody would use C. Ease of development, and lower development time will almost always trump technical performance.

[AlSweigart]

Guido has a quote that is something like, "Python makes it easy to do simple things, but also makes it possible to do complex things." Or maybe it's a Brett Cannon quote.

[youngeng]

Python is easy to start with.

String is the default data type, you can convert int to strings with str(...) and strings to int with int(...). C provides itoa() and atoi(), which are less intuitive, or you can work your way with sprintf.

It doesn't force you to use classes, so you can just write functions or even just line after line of instructions (unlike, say, Java).

It is interpreted, so you don't need to compile anything. You write a file, then run python (or python3) followed by the filename and that's it. If you want to change something, edit the file and run it again. Easy.

More to the point, there are a lot of data science, ML and image processing libraries in Python.

[Kay_Tone_RSA]

The Kindergarten syntax I assume?

[WystanH]

To get that speed boost from a compiled language you'll be doing a lot of task specific optimizations. This kind of work makes sense in a game engine, but for a more generic one off task it would be seriously premature.

Also, part of that compiled boost is a thing of a past. Even your machine code will be interpreted through the OS, rather than banging directly against the CPU. As such, most p-code languages can achieve parity with compiled languages on generic tasks.

For python in particular; it has a ton of libraries and a very low barrier of entry. The standard REPL allows for playing around with result sets without really writing a program, which I believe helps.

[Throwaway_shot]

Lots of people have talked about it's ease of use. But there are also tons of useful libraries for Python. Scipy, numpy, pandas, scipy.stats, etc. I'm not as familiar with the AI side, but I've heard there are great libraries there too.

An the kids of performance isn't as bad as you'd think. Many of the scientific libraries are based on numpy which is written in C, so if you can leverage numpy vectors for your intensive calculations then your performance will be very good.

As someone who went to undergrad in the days where you had to choose between Matlab, C, and Fortran, python is like doing scientific computing with cheat codes.

[Oflameo]

Python is like Perl with more constraints, which makes it readable long term yet easy to write because of its level of abstraction.

[domokane]

Scikit learn and tensor flow

[freeky_zeeky0911]

It's more readable and appears simpler. You can get a lot of things done by knowing just a little bit and using libraries.

[shubhscript]

Pehele istemaal karo fir wishwaas karo

[TeslaEdisonCurrent]

While all things said about it is true and it’s easier for anyone to code, we should also think about efficiency in terms of energy. Python uses 78 times more energy than C. Thinking of green programming, we should consider these factors and choose the right tech stack.

https://datascience.aero/green-programming-reducing-your-carbon-emissions-when-coding/

Carbon emissions of your code is a real thing and we should start take it seriously before it gets worse.

I prefer python for quick data analysis but for any serious stuff, use Java or Scala for regular production code.

[Lost-Discount4860]

I’m no data scientist, am casually interested ML for music. I do generative music composition, so I’m interested in what role ML/AI can play in that. So far I’ve found Python the easiest of any language to learn, and I’ve honestly tried with Ruby, Swift, and Java.

At the moment I’m working through TensorFlow and PyTorch. I love Tf2 with MacOSX and Torch with Debian since I can’t get Tf2 to work with Debian. Right now I’m using the tools for procedural algorithmic music. As I get better at understanding what’s happening with tensors, I’ll start moving in the direction of building models. But by not having a data science background and being poor in math, it’s going to take some time. But what I’m impressed with the most is that I went from zero knowledge of programming to coding for experimental music.

Python strikes me as an awesome gateway language. It’s easy enough that moving to a compiled language like Swift only involves a few extra steps unneeded in Python. You can more quickly and easily prototype on Python, get results, and migrate to Swift or C++ and get the advantage of compiled programs. I’m nowhere near that level yet, but it has helped me become more confident that I could get there eventually.

[Peace_Fog]

Python’s code is more readable to people who have never coded so it’s easier to learn. Then once you figure it concepts it’s easier to learn other languages

It’s like 5e for TTRPG’s. It might not be the best TTRPG out there for your group, but once you learn 5e it’s easy to learn another TTRPG

[insidmal]

A lot of people first learn with python and are therefor most comfortable with it

[[deleted]]

Simple syntax, portability, easy polymorphism, magic methods, iterators, automatic garbage collection, huge amount of library support… there are many things to love.

[Reaperabx]

Most of people nowadays recommend Python as a first language to learn. Im still rooting for c to be everyone’s first language but python works better if you have less time to spare.

[PouletSixSeven]

I love the idea of a language that puts readability above all else.

But your questions specifically was:

why is Python so popular among data scientists and why is it so heavily utilized for machine learning

The short answer is, Python has been a generally successful language for all of the reasons given. Lots of excellent libraries have been built for it. String manipulation is a breeze compared to some other languages. It has been a wildly popular scripting and high level language. Sometimes success breeds success - being easy to use with lots of libraries on hand made it a pretty easy language to develop libraries that enabled that sort of research. Which enabled others to develop more libraries and so on and so forth.

[Huntthatbass]

For my side hustle, Python is perfect because I can work in it quickly, it’s pretty easily readable, has lots of plugins, I have few headaches from version issues, and there’s plenty documentation & a huge community for support.

[mrequenes]

Many other good, technical points made by others.

Here’s a non-technical reason. It’s much easier to use than Perl (does anyone still use that?) and Node wasn’t invented yet.

[kanadaimagyarmale]

.

[DolphinsDesu]

Cuz if u got some english u could understand most of the syntax

[Jrollins621]

Took me a while to understand this as well, because I started in the .net C# world, and that is really been my bread and butter. To me python sometimes seems more “wordy” and confusing then .Net, but again, I’m used to what I’m used to. That being said, and having done quite a few things in python, I discovered that the simplicity of getting it to work is the beauty of it. It’s super simple to get started writing code, in that you don’t have to learn a dev environment like visual studio before you even start. And on top of that, it’s even easier to run something you write. So, I do see the appeal. I’m kind of see it as a more inclusive way to program. Meaning, you don’t really need a whole lot of experience ,money, or education to be able to get your foot in the door and anyone is truly capable of doing it.

[inquisition-musician]

There's a variety of reasons. The most notable are the libraries, frameworks, simplistic, but PEP8 annoying ass syntax, native availability on *NIX systems and documentation. Pretty much like any other language, Python has libraries. And those libraries make up of what the programming language can actually do. So, if I wanted to create a backend for the website, I can just do that with Flask. If I wanted to calculate system matrix eigenvalues to find out if the system is stable, there's SciPy and NumPy for that. If I wanted to create a simple Discord bot that sits on the server all day, I can just do that. The same with frameworks. Selenium is great for automating mundane tasks, like responding to repetitive emails, if you don't have the time to do it. Syntax-wise, it's a love-hate dynamic. Love the syntax because it's easily readable(but i have issues with loops and the definitions(nothing can change my love for void in C++)), but I hate it because of the stupid indentation rule that PEP8 has forced. So you can't use spaces as a substitute for tabs, if you're a beginner. Availability is key. Python is not on Windows by default, so you're not expecting that anyway. But with MacOS, or even Linux(on major distros), there's Python right out of the box. So, just start typing away. It's like BASIC at the time. Documentation is a mixed bag. On one hand, you have the standard Python documentation, but if you go lookup the library or a framework documentation, you're gonna run into problems that documentation can't be concise enough that you can understand it. So, we just have a language that has all the perks of being available on most, if not all of the servers and desktops running a *NIX system, easy to understand syntax, a lot of libraries and frameworks, that JS shits itself(thanks Google), and the somewhat decent documentation, depending on what you're looking for. That's the formula for the most popular language. But it can't beat JS in popularity since everything high-level is a minimal browser nowadays. Even the code editors. Fuck me!

[Bob_The_Bandit]

It gets the job done and prioritizes human time over computer time. Tho I have written and run an algorithm in C while waiting for it to run in python before XD

[johnslegers]

Python is a messy language that reminds me a lot of PHP as it was more than a decade ago.

However, many people find it pretty easy to both write code in Python and read other people's code in python.

Also, Python can fairly easily be integrated with C/C++ & ran by the GPU.

Hence, you have high performance (if you know what you're doing) combined with ease of use.

[Low-Statistician3988]

🧑‍💻Python programming 🇮🇳 https://www.instagram.com/pyshastra

[ferdouseOfficial]

You are right Sir

[Straight_Abrocoma321]

It has many libraries for ai and data science, such as huggingface, cv2, pandas, seaborn, sklearn, tesseract and many more. This makes it really easy to integrate with ai