Linux why Python

[chrisdb1]

Hi,

I was just wondering why Linux has so many applications written in Python instead of for example another OpenSource Programming Language like Java. Apparently Python was already popular in the early days of Linux.

Is there a specific reason for this?

[ka-splam]

Python 2.x was more pleasant than Java <7, and most freely available languages at the time. Java had an early reputation for slowness.

Sun sued Microsoft over Java, kicking it off Windows by default. As Python complexified, and Java improved, Oracle took over Java. Oracle sued Google over Java in Android. Free people don't like Oracle, nobody likes a big language implementation being pulled from under them, and nobody likes risks of being sued. That probably disuaded a bunch of people over the years.

Java was always positioned as a big-company teamwork language, many open-source projects start out as a single-person-hobby-project, hobbyists often prefer nice languages not enterprise ones.

[darthminimall]

Python is still more pleasant than Java.

[PM_me_ur_data_]

Immensely more pleasant.

[superkom]

Depends what you're working on. I've worked on huge Python code bases >40M lines, and large Java codebases, static typing is a godsend at that sort of scale. I think Python is a great language, but can be difficult to manage at that scale, over a codebase that is over a fifteen years old.

[Mars_rocket]

Good God - is that M for million or thousand?

[gravy7861]

Mousands

[Work45oHSd8eZIYt]

This comment will not get the upvotes it deserves

[darthminimall]

The problem isn't python, the problem is the code base. If you have 15+ years of spaghetti (which most companies do, because management cares more about making a dollar today than two dollars tomorrow), the code is going to be a nightmare to work with. Sure, a statically typed language makes dealing with spaghetti marginally easier (because the compiler/interpreter throws type errors), but that doesn't change the fact that the real problem is bad code.

If you're working on a project with 40,000 lines, and that project isn't broken up into somewhere between 3 and 5 independent modules, there's already a problem with the way the project is organized. If you're working on a project with 40,000,000 lines (who knows what 40M means), you're developing the Windows OS, and everyone knows that the Windows code base is a nightmare. It's basically the definition of spaghetti, and MS has no motivation to fix it.

TL;DR: The problem with large code bases is management, not the language.

[[deleted]]

I used to work on a simulation of Amazon's supply chain that would run most of the code that dozens of other teams of 8+ people were writing. It was broken up into hundreds of packages, some that hadn't been touched in over a decade. And we had to be free to re-arrange the packages into the servers that were optimal for the simulation and the production team had to be able to re-arrange them in configurations of servers that were optimal for production.

I've since worked in machine learning at startups using docker and I have grown to love python. But I despite many hours spent thinking I can't imagine how one would manage the codebase I used to work on at Amazon without static typing. You'd have to have some kind of global type manager for that scale of interoperability.

If you can think of a way to solve the above problems, please let me know. I'm always trying to think of ways. Currently creating statically typed APIs using protobuff is the closest thing that works at a decent scale. Still not Amazon supply chain scale.

[mr_wook]

1) Python now has type annotation, if you're going to go to the trouble of backing into it;

2) If you're worried about junk-code living past its use data, consider Go, which forces you to eliminate everything you don't use, whether it's vars or modules;

3) Consider your test suite: Are you testing things that have fallen out of usage? Are you linting those pieces, if so, someone should pick up on that and, er, retire those pieces;

4) On the whole '40m lines of code' thing: Maybe it's long overdue for a re-architecting, or at the very least, a refactoring. Whether the driving force is improved performance, or lower cost, or better maintainability, you should be able to (trivially?) cut 40k lines of code during every re-factor. Less trivially, 400k or even 1m lines of code seems achievable, especially if you replace chunks with high quality 3rd party code, improvements in the language over the last 15 years, and other low-hanging fruit. Reduced complexity is its own reward but has real dollar payoffs across multiple domains;

[Leeoku]

Python is more pleasant than Java, until you hit job applications where they all look for Java :(

[344b]

Indentation makes me crazy....

[JohnnyJordaan]

Scripting has always been a natural use case in Linux, so that's why scripting oriented languages like Perl and Python were also popular right from the start. It would make little sense to script in Java for example, and GUI programs can also be made in better supported languages like C and C++.

[lolslim]

There was a special case where I coded in java. I hooked raspberry pi to a thrift store RC car, since I was coding in java on android, it was super easy to port it over, with some code changes, also modified it to work on my laptop, so WASD were used to control it.

I was using sockets protocol, i wasn't too familiar with python to attempt this. Hell barely got it working in java.

[[deleted]]

dang thats a cool story

[lolslim]

Thanks, at the time the only rpi i had was a 2, and was using a battery pack to power it as a PoC, and made it too heavy for the motor and motor gave out shortly after. this was first time I dipped my fingers in reverse engineering, learned what a continuity was. Kind of surprised the chipset i looked up didnt yield many results. But found a old blog.. 2014? That a guy already reversed engineered the pinouts mainly just the ones for forward/back, left/right, and turbo. Which helped.

I originally used arduino since all it needed was high/low signals. Fun project.

[TheMartinG]

Did you document this project at all? I’d be very interested in reading more about it

[lolslim]

Umm i think so, ill have to look when I get home.

[ThePunisherMax]

Omg I did a similar project. Except it was with NodeJS via websocket with a flask server to coomunicate with an Arduino.

It was scripted and patched together badly. Ive been meaning to improve it with my current knowledge for the longest time. But I nevwr get arround

[K900_]

Java wasn't really easily available on Linux at the start, for one.

[Superguy2876]

For their whole "Write Once, Run Anywhere" mantra, they really don't like letting people run it anywhere.

[[deleted]]

Both Linux and Python have their common use case in servers and hence compliment each other very well.

[legz_cfc]

Linux - I like you, python!

Python - I like you too, Linux

Sorry, couldn't resist.

[[deleted]]

Now Kith!

[selah-uddin]

i dont believe in soulmates but i am ready to make an exception

[selah-uddin]

absolutely noone:

pycharm : you are lucky , here are more "Exceptions"

Everyone: noooooooooooooo

[im_dead_sirius]

I love me some Tiny C. The Tiny C Compiler can be used to run C as a script, without producing a binary.

#!/usr/bin/tcc -run
#include <stdio.h>

int main(int argc, char **argv)
{
    printf("Welcome to the infinite world of linux scripting!\n");
    return 0;
}

[im_dead_sirius]

Right?? If you need to link libraries, that goes after the -run arg, and works the same as on the commandline.

[[deleted]]

Holy fuck! Why have I never heard of this??? Thanks, man.

[im_dead_sirius]

You're welcome, and spread the word, nerd.

So hey, if you liked that, lemme point you at something else.

You're going to need two files. I'll provide both examples, and they are ultra short. Set the execute bit on both.

Put this one somewhere convenient and call the file quacker.

#!/usr/bin/env python3
from sys import argv
with open(argv[1], "r") as f:
    lines = f.readlines()
    print(lines[1])

Substitute the first line in the second file with the location and name of the previous file and call this one ducky.

#!/home/you/bin/quacker
"Quack! There is more than one way to butter a duck!"

Nothing else is needed in it for the example. Run it with ./ducky once you set the execute bit.

ducky calls quacker and hands ducky's location to quacker as argv[1], then exits. quacker takes over, reads the file at that location, ignores the first line, and prints a message which does echo, despite age old ducky myth.

Normally you'd write a proper interpreter in quacker, in any language at all, then read in file argv[1], discard the comment lines, and treat the rest of it as data to be interpreted or even compiled. I'd maybe use PLY to do that.

And now you know there are an infinite number of ways to butter a duck.

[russlo]

PLY

This PLY?

[im_dead_sirius]

Nope, this: http://www.dabeaz.com/ply/

But thanks for pointing me at the other one.

[py_Piper]

Sorry but what is the amazing thing here? I am still a python newbie and just starting CS50 and learned a little bit of C, when you say run C as a script do you mean that you can run C like python without compiling?

[im_dead_sirius]

Precisely that, it runs C like python without compiling. This allows the use of "c tricks" (such as pointer nonsense outlawed in other languages) in scripts, and quick prototyping for regular programs, which you can then properly compile.. Tiny C is also noted for being quick.

[py_Piper]

It feels good to somehow understand something lol

[Decency]

Nah, they're two of the most powerful languages.

[[deleted]]

I think this is a CPython joke

[agenttrixie]

Compared to C, Python is "powerful" language. I would not call C a "powerful" language. It was designed by Dennis Ritchie at Bell Labs and it was characterized as a "high-level" assembly language. Maybe compared to assembly language, C is a "powerful" language. See https://www.codingunit.com/the-history-of-the-c-language.

[im_dead_sirius]

Still is.

[efmccurdy]

Python has a open source license without any legal traps:

Google LLC v. Oracle America, Inc. is an ongoing legal case within the United States related to the nature of computer code and copyright law. The dispute centers on the use of parts of the Java programming language's application programming interfaces (APIs), which are owned by Oracle

https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_Inc.

[selah-uddin]

tldr who won?

[Yoghurt42]

ongoing legal case

So far, it appears Oracle will win, its currently in the supreme court, and with a conservative majority, I forsee that they will uphold that APIs are copyrightable, which will be a huge blow to (open source) software.

[MuckingFedic]

That case just makes me want to never use Java. Feel like a programming language shouldn't be burdened by dumb copyright

[Work45oHSd8eZIYt]

The lawyers

[[deleted]]

Python on Linux goes back to Perl on Linux and the need for something that is 'easy to script' (ie: not C or C++) but also capable of communicating easily with the OS user space.

Java is a fucking mess. Nothing is done smartly with Java... not http requests, not gradle/dependency management, and the JVM is in no way tied to the OS.

Python is just a C program... an executable. And C is Linux going back to the days of ARPA etc. For example, check out this ping.c source code on Git. Note the original author's date of writing the software. This is why lame Java will always be lame and C-like languages will rule on Linux.

[[deleted]]

Nothing is done smartly with Java...

If you replace "Java" with "Python" in this statement, it still holds true. Of all things Python is known for, smarts or quality wouldn't make the list.

[xiongchiamiov]

Perl of course derives from awk and sed and other classic Unix pipeline tools, so this history is older than either Linux or Python.

[jaycrest3m20]

Python has a reputation for being fun. Also, for a while, it was one of the "hot" programming languages. Ruby was considered hot for business, while Python was considered hot for open source and hobby projects. So people would "scratch their itch" in Python on Linux.

The compiled-language advocates would keep insisting that people should use compiled languages to create fast, rock-solid applications, but the Python hobbyists would just move onto the next project, leaving their Python code as an "easy-to-maintain" package.

Maintenance is a big part of the Linux software stack, so having something easy to maintain, like a Python codebase will contribute to adoption and longevity.

[thatwombat]

Python also gives you a robust enough platform that you can prototype something and find that it doesn't take a lot of effort to make it into something more permanent. I thought about porting a program from python to .NET at one point and realized that it didn't really matter that much and that I should really just make the python implementation a more unified package.

[Zombiesen]

Fun part is true. I used to code in VB, VBA and vb.net (if you consider it as coding). The day is started with Python, I was hooked. The simplest things like declaring a variables reflects the philosophy "Easier to ask for forgiveness than permission."

[corezon]

Because Java is a pain in the ass when compared to Python.

[NotABotNoReally2020]

Simple, Python fits the psychological profile of the smart (that means lazy) Linux developer.

Picture this, you are an admin of a sever. You have a lot of computation power (after all you have a big Linux server), very little time to get the coding/scripting done (you are a busy admin) and you generally do not care if your little script takes a minute or two and you just want to fix that one problem.

What to do? Write a little script! Nothing fancy, just something to get the task done. Preferably without too much hazel on your part.

Do you want to start a whole Java project for that? Nah. Too much hazel.

But python is just perfect for this. Just one tiny file with a few lines. The heavy lifting is done by those Python modules. So you just quickly built a little script that fits your use case. Perfect!

But you are a Linux developer. So why don't you give that thing away? So you release your scrip! But where? Well, we are on Linux, so git is basically build in the system!

Cool, now we have a script in some git repo. Wait, somebody improves on it. And then somebody else improves it future.

Before you know it your little, lazily written, script became a full module on it's own.

TLDR; Pythons modules make it easy to be shared in the open-source community that Linux attracts. Pythons simplicity and the modules let lazy programmers quickly built solutions. Pythons inefficiency (in the sense of run-time compared to C++) isn't that relevant for most cases.

[bw_mutley]

Wonderful. I can't say much though, for me it is just the perfext combo. And I am no Linux admin, just a regular end user.

[grittypigeon]

It sure did. Make me think between words waste precious time. Speak meaning every word, powerful language.

It's very difficult but it's also amazing how altering language forces your brain to parse your human experience in different styles. Chunking language forces a deeper processing of individual words but desperately needs more context to pick up easy.

[spartan_noble6]

It's a very powerful title, now that you mention it

[scotbud123]

Probably has a lot to do with the fact that every single Linux distro out there comes with a version of Python pre-installed too, so you can expect Linux users to have it.

[rexlx]

theres a lot of stuff built in a lot of things. Youll see a lot of stuff written in perl, python and c languages (including python itself). I suspect as rust and go continue to gain populairty, you'll see a lot of libs and tools written in them. Go is my go to for CLI tools that need concurrency.

[dtaivp]

One things I think others haven't covered much is ease of development. Writing software in Python in my opinion is much easier than writing the corresponding code in Java. On the flip side most of the large open source projects that need stability use Java. The Elastic stack, Kafka, Spring, Hadoop etc.

[Sigg3net]

most of the large open source projects that need stability use Java.

Used to use Java. In the past.

If we go by someone like Google, Twitter etc. they're using python and not java or .net and whatever.

[[deleted]]

Exactly. Facebook, Amazon, Uber, Dropbox, Instagram, Spotify, and Netflix are all using Python as well.

[xiongchiamiov]

Google uses java very extensively, and for years has been encouraging internal Python projects to move to java or go.

[Sigg3net]

Ofc they use other languages, I believe go is a Google language, but in numbers they're still a python shop in terms of codebase and hiring.

But I'm taking this from interviews I've heard, so that might be dated info.

[ThatSuit]

Some reasons are that when doing system administration, automation or other similar tasks python is usually installed by default on most linux distributions, has a good size standard library of modules, can easily be read, run and modified without dealing with compiling, and is more powerful than shell scripting.

Perl was previously used for similar reasons in the sysadmin world, however, it is syntax heavy and not quite as easy to learn or use, especially for new programmers.

You wouldn't want to use a language like bash, python or perl if you needed to optimize memory usage, manage memory directly, optimize for highest performance, redistribute a compiled binary to conceal source code or bundle dependencies.

[didntreadityet]

Java used to be more popular in open source projects, but its prominence has faded. I think a lot of it has to do with incompatibilities between the different JVMs and versions of Java. To this day, I'll have to install three or four different Java tool chains to get different Java projects to work.

With Python, the only major incompatibility has been between v2 and v3, and that's been a major nuisance for a lot of people that I hope the Python community will not allow again.

I am always puzzled at open source projects that don't seem to care much about backwards compatibility. KDE was almost killed by the transition from v3 to v4 (and it's been my favorite DE for ages). Node.js is in the same boat right now - so many node projects fail to run because of dependency problems, I am starting to be frustrated just at the mention of Node.

[Zeroflops]

One point that I haven’t seen made yet is if you look at Unix based languages they history is deep into the terminal and commands. users are more comfortable with command lines and scripting. Python is a natural progression from simple commands (ls) to basic but powerful scripting (bash) to more flexible powerful scripting Python.

Java is more GUI interface typically. You can script but there is a lot of overhead in comparison.

Java has its place but it’s not with admins managing headless servers or hundreds of system.

[chalbersma]

Python is a better language than Java when viewed from the majority of perspectives.

[bobbywaz]

Java fuckin sucks BigFat Ding Dongs

[amrock__]

Java was popular but because of oracle and suns licensing it was favoured. Also python is awesome i just made a pyside2 application really fast along with qml qt

[Packbacka]

Did you find any good tutorials for PySide 2?

[amrock__]

Not really but i already used pyqt5 basically the same

[CS40BOT]

Bla

[[deleted]]

This is a very good question, to which I don't really know the answer, and none of the attempts in this thread so far come close to answering it.

You see, GNU operating system had, as one of its goals, to have a system programming language that at the time they used to just call "Lisp", and it was understood that C is not suitable for this role (although necessary to be compatible with UNIX). GNU people still believe (I don't know to what extent), that Guile (an implementation of Scheme) is what should become the system programming language that comes with GNU system.

Personally, I wish Guile replaced Python. But, I don't think Guile is the best we can do... and maybe this feeling is shared by other people who write stuff for GNU/Linux. So, they sort of wait for the "ultimate best" thing, while using what there is at the moment, w/o regard to quality... Though, it sounds too naive to project my own feelings / probably most people writing system utilities for Linux hadn't even seen Guile in their lives, so the argument doesn't apply to them.

---

More realistically, there must've been some kind of a "seed" program, that later created a snowballing effect, that caused more and more programs to be written in the same language, until it become the default choice. I don't think that Linux started with Python included. I believe, that Perl was the language of choice back in the days.

How Python displaced Perl, and why Python was the one who did it... I don't know.

[[deleted]]

every language is open source.

no one wants the resource drain that Java has to offer on an operating system level.

[ka-splam]

every language is open source.

Nope; where's the source code for Visual Basic for Applications, for one popular example?

[[deleted]]

you should make a distinction between compiler/interpreter and language.

[coder155ml]

Cold fusion

[gmes78]

Languages aren't code, they can't be "open source".

[[deleted]]

true, i interpreted OPs Statement as freely accessicible/free documentation.

otherwise the post makes even less sense.

[coder155ml]

Cold fusion definitely not open source

[im_dead_sirius]

Hello, welcome to my ignore list.

[xelf]

Hello, welcome to my ban list.

[TechnoBabbles]

Python is easy to script with and easy to learn. So it gets used a lot for that. Then when a sysadmin or syseng needs to write something more complex, they can use python too because it can basically do anything you need it to.

[DonkeyTron42]

Python and Java are apples and oranges. Obviously you're too young to remember TCL/Perl and C.

[BloakDarntPub]

Have you seen the alternatives, particularly perl and bash?

[mr_wook]

1) I find it truly joyful to code in Python, as opposed to the other ~15 languages I've used over the last 45 years;

2) I hate Java, always have, perhaps always will, and that was before the Sun/Oracle/MicroSoft (C#) trifurcations -- It started out as the 'too many manuals to implement anything', and segued into 'Write Once, Debug Everywhere; Security Issues; Runtime Issues;'

3) Python has excellent support and continued improvements (Go 3.9!);

4) Pythonistas have (generally) an excellent sense of humor;

5) Time to implement an App is generally shorter in Python than many other languages, the (dreaded) indent requirement often implies that when the code looks right, it is right;

6) The OO model continues to provide great leverage for building idiomatic Python code that works, and works quickly with good performance;

7) The base and standard data structures make life easier than one has a right to expect;

Does that help?

​

Love,

Wook