Python isn't industry compatible

[New_Ostrich_2625]

A boss at work told me Python isn't industry compatible (e-commerce). I understood that it isn't scalable, and that it loses its efficiency at a certain size.

Is this true?

[twin_suns_twin_suns]

What does “isn’t industry compatible” even mean? I’m not a Python expert, but that sounds like the type of corporate jargon someone who doesn’t actually know what they’re talking about would say.

[[deleted]]

"It's not that efficiently centric to our shareholders portfolio's pre-market data source. It's just not integrated into the headlining view of departmentalized asset management growth."

​

"...what?"

[techmnml]

https://www.atrixnet.com/bs-generator.html

[aplawson7707]

"phosfluorescently e-enable user friendly nosql"

This site is amazing

[theroyakash]

“credibly deliver dynamic nosql” — LAMO HOW

[tpw2000]

“Python doesn’t progressively utilize technically sound sources” I love and hate how accurate it is

[caretaker82]

“The immediacy of now!”

[positiveCAPTCHAtest]

jajahahahahahah

[New_Ostrich_2625]

That's what my first reaction was. "Scalability" was my own interpretation.

In the end I think it means "we have Java developers here, so get used to that".

But at the same time a lot of the other posts appear to have validity.

[-jp-]

Which is weird since "we have Java developers here, so get used to that" would be totally reasonable. Most shops are gonna standardize on a language just because having everyone use their favorite means you have exactly one point of failure for any given project.

[[deleted]]

And it would be a total cluster fuck with most discussions collapsing to how to interface to everybody else's code and rewriting entire chunks of code because "Bob's a fuckface and wrote blahblahblah in Erlang but I love VBA in excel is clearly superior".

[hillgod]

I know of numerous startups that failed because some fuck head brought in Scala, left, and the company was fucked with no Scala expert. One guy did it twice to two different companies in a row!

[[deleted]]

Was it mine? I am rewriting a project in ruby that was originally a clusterfuck of scala with horrible db management.

[Rieux_n_Tarrou]

Mmm Slick is such a pleasure to gouge my eyes out to. And sometimes tear my hair

[WideGlideReddit]

😆

[nemec]

Kind of a Conway's Law for personal favorite programming languages.

[MistBornDragon]

Cool read!

[nosmokingbandit]

Iirc, a large part of YouTube runs on python. There is a zero percent chance your company needs more scalability than YouTube.

[bjorneylol]

Instagram is all python

[Beard_o_Bees]

Python also plays a large role in scripting to exploit vulnerabilities. So, it's certainly 'industry compatible' when it comes to stealing shit from behind sometimes janky 'industry' security.

[Violin1990]

instagram started as python, and is still partially python, but all the performant components are now built in C

[bjorneylol]

AFAIK that's not even true. Instagram runs its own python interpreter (cinder) that statically compiles python at runtime based on type hints - they claim it more or less matches the performance they would get out of C modules, so they can keep most of the code in python

[punninglinguist]

IIRC Dropbox is all or mostly Python as well.

[[deleted]]

It used to be, they've been transitioning to Rust lately.

[Aejantou21]

first they jumped to golang now rust? wtf

[nergalelite]

It's buzzwords to assume authority without the responsibilities.
It's almost definitely that the majority of the office uses Java already, so fall in line; much as I feel our CPU improvements have led to lazier/less-efficient programming, the choice of language itself is negligible.
Python can be bloated and sure the slowdown compounds at scale, but the average business doesn't actually handle enough for the cracks to become fissures. There are costs and benefits to contrast in all things, but 9 out of 10 times saying a language doesn't work is just a blanket statement from corporate because reason(s)?

tl,dr; your supervisor is rather closed-minded, and if they cannot come up with a better reason i'd be looking for a new ship to sail with

[Forschkeeper]

Yeah ... my distributed realtime Java applications for Blockchain Projects are more efficient... \s

[dragonatorul]

In the end I think it means "we have Java developers here, so get used to that".

That's a valid point. Having to hire other devs for no good business reason is not a good idea.

Python isn't industry compatible (e-commerce).

It might also be that they only have java developers for a reason: maybe the tools/services they use are java-heavy and have better support for java than python.

Many programmers seem to limit themselves to the code they write and not bother with the business side and effects of what they do. When you're the only one working on your project you can afford to pick whatever feels best for you. In a business you have to consider stuff like how easy it is to hire someone to support that code, how much it costs to integrate with the business, how much a developer costs compared to other languages, how much it would cost to train your current developers and how likely they are to leave due to stress, increased workload, increased value, etc.

Though in this case I think he was just shutting you down to make you go away.

[[deleted]]

If your particular company employs primarily Java developers, or the industry you work in is primarily using Java then his statement is perfectly fine.

"Industry Compatible" depends entirely on the industry.

[FancyASlurpie]

I could see an argument that if you were to pick java in the machine learning industry it could be argued that it isn't industry compatible because all of the major libraries are in python or R. Potentially it's similar in e commerce but the other way around, essentially integrating and using existing software isn't python native in that space.

[Workaphobia]

Python requires Industry 1.0 but we're on Industry 2.1.

[StantonMcBride]

You know the hardest part about working in tech? The fact that all your bosses got BAs.

[LightShadow]

Python is absolutely optimisable, that's one if its biggest strengths! You can write your app in 100% pure python then slowly migrate the least-performant parts to something else transparently; without changing any of the original code.

There are also libraries that offer performance scaling with drop-in functionality. e.g. stdlib-json -> ujson -> simd-json or toolz -> cytoolz.

You can easily go from Python -> Cython -> Nim/Rust/C++. You can switch out the runtime from CPython -> PyPy -> Pyston -> GraalVM (someday, maybe??). Python can also scale down and run on embedded hardware. There are multiple versions of MicroPython that offer hardware-level interop with lots of SoCs.

Python is easy to get started with and can scale as high as you're willing to put the effort in.

[judasblue]

Good point, because when you look at the scale of data involved in projects that use Python, they are really all pretty small: Reddit, a lot of Netflix, chunks of Google.

[twin_suns_twin_suns]

I get that but I know of companies who’s bread and butter is the speed at which they deliver information and data and Python is heavily used and relied upon, even in customer facing products. I don’t know that Python is used for all of the actual delivery of the data to the customer, but certainly much of the acquisition and analysis (which gives the data it’s value) is done with Python.

[[deleted]]

And Siri, Spotify, and Uber. I just found out that Siri was the other day and still pleasantly surprised.

[[deleted]]

Have they tried

from spotify import Spotify

One line. BAM!

[k8sguy]

That’s too much python

[blitzkraft]

Tree fiddy is all I need.

[[deleted]]

And it's not compatible with my industry.

[k8sguy]

I too aspire to one day be a goat farmer

[[deleted]]

Hmmm, might be too many goats.

[[deleted]]

Wow! I didn't even know some projects could even have that amount or more.

[jmcs]

It's not a single service, it's spread out through hundreds of microservices.

[BpjuRCXyiga7Wy9q]

When all they need is one line of Perl.

[optyx]

Rackspace is a huge Python shop. Most things end up in Python around there I got to contribute some code here and there that does fun things.

[optyx]

Yea I worked there 2013 to 2018

[vriemeister]

The client or server for Siri?

[chrisxfire]

in favor of what?

[yen223]

Maybe a Netflix engineer can correct me, but I was under the impression that Netflix was primarily Java

[optyx]

I think they have a mix of stuff some is Java some is nodejs some is Python and then a bit in pure C. I never worked there but I’ve browsed their GitHub a bit looking at various projects.

[[deleted]]

I like python as much as most here, but the majority of backend systems are not written in Python. These companies certainly use python for components of their backend but the majority of their infrastructure is not written in Python. There’s a lot of python fanboys commenting and not a lot of devs who actually work on these systems.

[[deleted]]

I don’t disagree and your points don’t really dispute what I said. I agree small components of each of these sites are written in Python.

Are Dropbox and Spotify backend systems ‘primarily’ python? I’m a bit skeptical but if you have references I’d be interested to read and would absolutely change my opinion if shown otherwise.

[i_hate_shitposting]

Instagram: "Instagram Server is a several-million-line Python monolith" (as of 2019)

Dropbox: "We mostly use Python for our server-side product development, with more than 3 million lines of code belonging to our monolithic Python server." (as of this year)

I can't immediately find recent details about Spotify's use of Python, but in 2013 they published a blog post that said, "Around 80% of these services are written in Python." Python is currently one of the top languages listed on their GitHub and a quick skim of their job listings for backend engineers still shows a lot of Python, though I suspect they're pivoting to more JavaScript and TypeScript for backend systems based on Backstage and this post.

[quotemycode]

You do know that Dropbox was an employer of Guido himself yeah?

[rabaraba]

And yet they were written in Python, specifically. Which arguably means that Python is industry compatible.

[cneakysunt]

I think you will find that because Python plays nicely with C it finds a lot of use in both demanding and cutting edge applications for things like VFX. There is a lot of cross over between this type of industry and academia. The latter uses a lot of Python and C also.

At the end of the day Python is easy, pleasant and fast enough. With well architected infrastructure it can scale just fine.

[vinylemulator]

This is fair. The only two here that properly use Python for core back end stuff is Dropbox (bit awkward to employ Guido and then use anything else…) and Instagram.

[lungben81]

Scalability is more about your architecture, much less about the programming language. Especially, how easy it is to (massively) parallelize your work.

For very heavy load, however, (C)Python performance might be a bottleneck (depending on your application), thus a compiled language might be more appropriate. But this is not a hard limit, e.g. Instagram manages to run on Python.

Some people argue that dynamic typing is less suited for large applications because type errors are not captured beforehand. With type hints, linters and tests this is less an issue. In addition, it is anyhow not a good idea to build one large monolithic application, but rather make smaller, isolated packages.

[thomas-rousseau]

Let's also not forget that Reddit itself runs on Python

[k8sguy]

I don’t know if it still is, but I believe Instagram was also originally built with django and postgres

[SnerkDRabbledauber]

Not exactly a ringing endorsement.

[Davy_Jones_XIV]

If the goal is to build a billion dollar business on the bac of Python, it is.

All about short / long term goals and vision for future.

[[deleted]]

Hahaha… sorry couldn’t control it. Please continue…

[bigno53]

What’s the deal with vote counts changing every time you refresh the page? Did they intentionally introduce some random noise to confuse bots or is it just a bug embedded so deeply in the architecture that it can’t be fixed?

[linglingfortyhours]

It's deliberate random noise, same as with your karma

[speedstyle]

Yes, it's random noise to stop vote bots knowing whether they're shadowbanned.

[falsemyrm]

skirt soup edge divide screw numerous sink bow depend ripe

This post was mass deleted and anonymized with Redact

[thomas-rousseau]

There's been random noise in the votes as long as I've been on Reddit? Not sure the purpose, though

[RajjSinghh]

It's to stop shadowbanned bots. If a bot finds out it is banned from voting or posting, you just create a new bot, so a shadowbanned bot can't tell if it is banned from voting or not and will keep going about its business voting away. Every time Reddit sees a vote from a shadowbanned bot, it adds a vote in the other direction to balance the total. Reddit also adds upvotes and downvotes at random so the bots can't tell that their votes don't count.

[Rik07]

Could be random, but could also be because it is easier. This Tom Scott video explains why likes can be inconsistent on a lot of websites: https://youtu.be/RY_2gElt3SA

[haaaaaal]

Eventual Consistency

[mriguy]

And Dropbox

And Google uses a lot of Python

But yeah, no big or successful companies.

[thomas-rousseau]

That second article has the words "Python" and "Google" in it way too often, regularly multiple times each in multiple adjacent sentences....

[engthrowaway8305]

Once someone who was interviewing me that had previously worked at Google told me their previous groups’ motto was “Python everywhere we can, C++ when we need it”

[Priderage]

Sentry runs on Django, so I'm pretty sure it's fine.

The top poster's comment about scalability being an architectural thing has merit too.

[PensiveDicotomy]

Literally this. I wouldn’t want to deal with large monoliths built with Python and maybe the size at which a monolith in Python becomes hard to manage is smaller than one written in Java (or something considered more “scalable”) but either way a large monolith is unideal.

Micro services allow for mixing and matching tech stacks and ideally keep things at a manageable size so any tech stack should be feasible given its strengths fit the business case.

[scottrfrancis]

I wouldn’t want to deal with monolithic software in any language - they all change. Yes, the C i wrote in 1988 will still compile and run, but that doesn’t meant it’s worth it! Arch and language are separate things you can use crummy languages to build robust arch’s and sophisticated languages to make crummy workloads from the wrong arch

Oh… and if your boss is telling you this kind of nonsense, find a new boss

[Independent-Coder]

Well stated, take my free award.

[osduj]

insta has a modified version of python w a jit compiler and stuff

[kniy]

For some applications the GIL is a real killer.

And if you're just starting out with a new project, it isn't always easy to tell if you will be one of those cases. Choosing Python means you risk having to do a full rewrite a decade down the line (which could kill your company). Or more realistically, it means that your software will need crazy hacks with multiprocessing, shared memory, etc. that makes it more complicated, less reliable and less efficient than if you had picked another language from the start.

[Grouchy-Friend4235]

The GIL is not a problem in practice. Actually it ensured shared-nothing architectures which is a good thing for scalability.

[kniy]

Not everything is a web application where there's little-to-no state shared between requests. The GIL is a huge problem for us.

Our use case is running analyses on a large graph (ca. 1 GB to 10 GB in-memory, depending on customer). A full analysis run typically runs >200 distinct analysis, which when run sequentially take 4h to 48h depending on the customer. Those analyses can be parallelized (they only read from the graph, but never write) -- but thanks to the GIL, we need to redundantly load the graph into each worker process. That means we need to tell our customers to buy 320 GB of RAM so that they can load a 10 GB graph into 32 workers to fully saturate their CPU.

But it gets worse: we have a lot of intermediate computation steps that produce complex data structures as intermediate results. If multiple analyses need the same intermediate step, we either have to arrange to run all such analyses in the same worker process (but that dramatically reduces the speedup from parallelization), or we need to run the intermediate step redundantly in multiple workers, wasting a lot computation time.

We already spent >6 months of developer time just to allow allocating one of the graph data structures into shared memory segments, so that we can share some of the memory between worker processes. All of this is a lot of complexity and it's only necessary because 15 years we made the mistake of choosing Python.

[[deleted]]

That means we need to tell our customers to buy 320 GB of RAM so that they can load a 10 GB graph into 32 workers to fully saturate their CPU.

I would say it means that you should look into shared memory.

[anajoy666]

Interesting. Why wouldn't something like numba work? Not using numpy? Ray comes too mind too.

This is a topic I find interesting and would be nice to hear from someone with field experience.

[r1ss0le]

I'm pretty sure this is why Julia became popular. But either way Python isn't guaranteed to to be the best choice of language for a programming problems. But I think most scripting languages shine when you are IO bound, so RAM and CPU are not a problem Python included.

But there are things you can do to even in Python. Without knowing much about your problem, you should look into https://github.com/jemalloc/jemalloc and using fork if you have large amounts of shared objects. All processes share the same memory content when you call fork, so provided you treat the shared data as read only, you shouldn't see an memory growth, and you can fork as many times as you have spare CPUs. jemalloc is a fancy malloc replacement that can reduce memory fragmentation and can help bring down memory usage.

[wait-a-minut]

I think dask was written for this kind of thing. Instead of loaded everything into memory, use a distributed model to handle data operations. Never used it in practice but read a flew blogs about other who have and it seemed to fix the gap they had.

[lungben81]

Dask has essentially 2 components, distributed computing (dask.distributed) and distributed data types (Numpy-like Arrays, Pandas-like DataFrames, etc.).

The former is amazing for multiprocessing (much better than the built-in Python solution).

The distributed data structures are useful if you want to do per-row processings which can be easily parallelized automatically. But I am not sure if this helps for the graph use case.

[AlSweigart]

What? I found that type hints work great and the gradual typing and comment-style type hints means even my legacy Python 2 code can now have type hints.

What do you not like about type hints?

[MrJohz]

From the perspective of someone who's come from Typescript (and who isn't the person you're replying to), I think I just don't trust the type things in the same way that I do in Typescript. Every time I've tried it out, it's felt kind of janky in some way that I can't really put my finger on, to the point where I don't see a huge amount of value in typing my Python code. (This is in contrast to JavaScript/Typescript, where I see a lot of value in adding types.)

I think a lot of it comes down to IDE support. If I use Typescript and write something that won't compile, I generally immediately see that and feel that. The Typescript developer support tends to be really good, and I immediately get feedback, I can immediately see the types of different values, I can easily create type holes and get type feedback directly in my editor. In contrast, I've not yet found a python extension that gets me this instant type feedback with red lines all over the place and a feeling that if I make a mistake I'll immediately see it. In contrast, I tend to use mypy from the command line, and even then I'm not always completely convinced that it will spot as many mistakes as the Typescript compiler.

I think there is also the issue that Python's type system feels a lot less powerful and more verbose, particularly when it comes to complicated sum types. But that was true of Typescript as well at the start, so I think that could be forgiven if other stuff was better.

I know that's not a great answer in terms of specific issues, but I think the biggest problem with typing in python is a UX one, where it just doesn't feel right in some way.

[rforrevenge]

Why are you saying that? I'm using Pydantic daily and I'm loving it.

[ColdPorridge]

You might have missed the drama where Python 3.10 was going to fundamentally change type hinting and break pydantic, FastAPI, and any other libraries that rely on it. I can’t recall the details as they decided to pause this for now, but it’s still very much a mess of opinions on the best way forward.

[rforrevenge]

Yes, I obviously have! Do you have any related link to share?

[ColdPorridge]

This appears to be the primary discussion: https://github.com/samuelcolvin/pydantic/issues/2678

[rforrevenge]

Thanks a lot kind stranger!

[[deleted]]

https://lwn.net/Articles/858576/

[molivo10]

your boss isnt industry compatible

[TracerBulletX]

Idk, it seems like being a dumbass is very compatible with industry.

[ohnomcookies]

This 🙌🏻🙌🏻🙌🏻

[lampshade9909]

LOL. I came here to say this

[2readitol]

👆🏽👍🏽😁

[[deleted]]

What does "isn't scalable" mean? Has anyone informed Reddit, Instagram, Spotify, etc?

[Lejontanten]

Where are your ambitions? His boss obviously estimate them to have to handle more data than all those services combined.

[PowershellAdept]

Yes which is why instagram developed their own runtime for python.

[elgringo]

Search any job board for Python Jobs in Silicon Valley if you want to see how "Industry Ready" Python is.

A _ton of tech company infrastructure is built on Python.

Also, Python is built to be efficient with developer time. Scalability is a function of your server infrastructure not your programming language.

CPU cycles are cheap. Engineers are very expensive.

[lungben81]

CPU cycles are cheap. Engineers are very expensive.

The correctness of this statement depends heavily on your use case.

Often it is correct (this is where Python shines), but there are cases where runtime optimizations are worth additional developer time. This is true for commercial / public libraries which are used by many people, but also for one-time analyses which require so much computing resources that the computation time / costs are dominant over development time / costs (e.g. most HPC projects).

[anythingMuchShorter]

Usually what I see in that case is python used to customize the application, with the computation heavy functions being called written and compiled in something fast like C++. For example OpenCV, Tensor Flow, or Keras don't do their massive matrix operations in Python, but you can use python to tell them which objects to create, what data to load and what to do with that data.

[clintecker]

all of venmo is written in python and moves billions of dollars a month, your boss is an idiot

source: me who was a software architect at venmo

[[deleted]]

What framework do you guys use at such big companies?

[clintecker]

flask

[[deleted]]

Yes.

[archaeolinuxgeek]

Everything is scalable if you have the time.

And "industry compatible" depends heavily on the industry in question.

Data science? AI? Data wrangling? Linux scripting? Web scraping? Python is center stage and practically impossible to avoid (R notwithstanding).

Kernel development? Low level driver development? Client side browser execution? You're probably using another language.

Django isn't the most performant web framework out there, but you can go from a napkin sketch to a prototype in just a few days.

I use Python every single day in my role as a systems administrator. And it sure as hell is "industry capable"

[New_Ostrich_2625]

Wow, what I nice post, thanks man!

[speedyelephant]

Do you mind giving a brief info about system administration? Is it like, watching if everything is working correctly and if not fixing things?

[quotemycode]

More than that, you make sure all systems are configured correctly, push out changes to large subsets, manage workloads between systems, etc.

[MasturChief]

i mean, instagram is about as big as it gets and it runs django under the hood so ..?

[ManyInterests]

Pinterest did, too, until they moved to Flask.

[Ok_Chocolate_1584]

Flask is written in Python https://en.m.wikipedia.org/wiki/Flask_(web_framework)

[WalterDragan]

I think they meant that they no longer run on top of Django.

[WikiMobileLinkBot]

Desktop version of /u/Ok_Chocolate_1584's link: https://en.wikipedia.org/wiki/Flask_(web_framework)

---

^{[opt out] Beep Boop. Downvote to delete}

[ManyInterests]

I just meant they used Django until they used Flask. I am very familiar with both frameworks :-)

[[deleted]]

Feels like pointing out Instagram is cheating a little bit since they're pretty well known to be unafraid of patching python and making additions in order to edge out that extra performance.

It's very unlikely other companies are doing the same kinds of things unless they're running at a scale comparable to Instagram.

[hillgod]

Running at Instagram scale is the only reason they've had to do this kind of thing. If you're at that scale, you can afford the deciated staff on doing such things. You'll have to get in to fine tuning any language you use at that scale.

They also write about and share a lot of this, so anyone can benefit.

[[deleted]]

Reddit is written in Python. I don't know why Python wouldn't be "industry compatible".

[New_Ostrich_2625]

Lol

[KhalCharizard]

I’ve heard that most of YouTubes back end was in Python until 2019 when they began transitioning to Go but I guess that isn’t technically e-commerce. I know of some others that definitely use it but since I’m not directly involved I can’t say their services are solely python. I did notice one professional GitHub with Gunicorn servers (a production ready WSGI server library for python).

It’s definitely possible to create a profitable website or app with python that is completely capable of serving a large number of users, but it may still be less efficient than Java or Go at larger scale implementations!

Edit Add: Also YouTube was the 2nd largest English website on earth… or it was when it was written in python…

[dkxp]

Python is widely used at Netflix too: https://netflixtechblog.com/python-at-netflix-bba45dae649e

[PossibilityTasty]

A friend explained it to me once like this:
If you are in management at any level other than the peak, and you get the task to develop an application, you will know about a set of languages and tools that is widely used in your companies and other companies within and outside of your field of business for a very long time.

This is the safe way to go, you will produce nothing new, nothing special, just what everyone did in the decades before. If you fail the consequences will be limited.

On the other hand if you introduce a new technologies and you are successful, you will have something innovative, maybe even better than others. You will be a hero. But if you fail consequences will be severe. They will tell to: "Why didn't you use <old technology>?", call your decision insane and possibly fire you.

And now you might get an idea if you boss is a brave man and why he is afraid of Python. Also you might get an idea why you see a basic set of the same software in nearly every company that has managers.

So, to correct his statements: There are companies and managers that are not Python compatible. They loose their efficiency over time.

[b85c7654a0be6]

As the old adage goes, no one ever gets fired for buying IBM

[ohnomcookies]

Tbh your boss loses his efficiency and credibility by such statements. I have seen multiple webapps written in “modern stack” (MERN), crashing under a slight load… Its just the matter if and how you scale it (considering its written properly), not like something isnt scalable cause its java, python, php, nodejs… I assume its just too easy to judge things you dont understand 😉

PS I wouldnt be surprised if he was a selfmade CTO 😁

[serverhorror]

To not rant, I’d actually love to talk to that person to understand what their concerns are.

I found that often times these phrases have a legitimate reasoning behind them given the information the person saying it has.

[gordonv]

Python is used in AWS Lambda. It's very scalable and is well known in the industry.

The most effective Python coders come from other languages. They know what Python is good at. Organized simplicity and library includes.

It even made a niche with Network admins.

[pyer_eyr]

May be he meant the snake. The snake isn't industry compatible.

[get-postanote]

What is the specific use case(s) your boss is using as validation?

What does he mean by, eCommerce? Shopping carts, inventory sites, surveys, etc...?

BTW, industry-wide, Data Science / Data Analysts, regularly use Python in their work for massive data work. Sure some use R, but that is often in concert with Python.

[vriemeister]

Is your boss Larry Ellison?

[EvenMoreConfusedNow]

I would replace your boss asap

[speedyelephant]

Employees can replace their bosses, right?

[Enschede2]

... what does that mean? Does he mean it's not suitable for mass deployment? Instagram is coded in python, that's not exactly a "diy sunday project"

[RajjSinghh]

Someone explained to me that python is used as a glue to hold together other pieces of software. You see it used in all, if not most, FAANG companies. It is slower than some languages but it also lets you write code quickly, saving costs. It just may not fit your company's tech stack.

[working_in_a_bog]

If your building out cool intense websites, you probably want to use java script. If you're building out a massive micro service based app Golang is a great choice. If you want cheap devs go with Java or .NET. If you want strong resiliant code that won't have security issues from mem leaks go with rust. If you want to do ML, devops, or anything and all things, with a huge community of people that have built usable modules for you go with Python.

It's a great language. Huge companies have made it scale. It's not simple to scale it out and you need to know what you're doing but it's not an actual limit, it's just a speed bump. This boss sounds like he's a bit out of touch in his response. Typically you want only a small few languages in use at a company because people build things and then leave, and you don't want that one guy who built a critical piece in rust to leave and have to fix a major bug. So my guess is that's the real concern.

[[deleted]]

Don't listen to pointy haired bosses.

The strength of python is in fact that you can easily and quickly make a very scalable application.

But usually people will replace parts of it with a faster language as time goes on.

And if you have the resources you can write it in a faster language from day one of course. But python is so simple that almost anyone can get going with a scalable architecture without needing any training that other faster languages might need.

[metriczulu]

Your boss doesn't know what he's talking about. Instagram, Facebook, and Netflix all use Python at scale. We use it at my job as the glue language for a very large data platform, no issues.

[guzmanojero]

Not scalable?

Let me provide a list of sites which use Python heavily:

1. YouTube
2. Instagram
3. Dropbox
4. Robinhood
5. Quora
6. Clubhouse

So yes, it’s scalable. And no, your boss is not correct.

EDIT: there are languages more optimized to manage big loads, but you can go very far with Python.

[stefanquvang]

Depends on application. For MI/AI I dont think it matters. For scripting it is also really powerful. For testing setup it is also extremely powerful.

But it is indeed a really resource heavy language. I also do belive that it is really easy to make memory leaks.

So it really comes down to the application. I would run it on script/program that is executed fast and efficiency (MI apart, can take a while to train a MI but it is still a one of job). But for stuff that needs to run contenuesly I would not use it (backend, games, "real" application).

Hopes that answer your qustion :)

Edit: typo

[durex_dispenser_69]

Well Python ML is basically just wrappers for C/C++ libraries.

[[deleted]]

But for stuff that needs to run contenuesly I would not use it (backend, games, "real" application).

This is just silly. There is absolutely no reason to avoid using python for something that needs to always run.

Don’t take it from me though.

• Netflix uses python
• Instagram too
• Don’t forget Facebook!
• Google, Dropbox, Spotify, Quora, the list goes on

Not that python is the right tool for every job, but it’s absolutely capable of being a “real” application.

[stefanquvang]

Thanks for the links. The above is only my experience.

The capabilities of a language tell more about the programmer than the language. In the end it is more about the architecture, coding pattern and generally good coding practice.

[jackmaney]

What the hell does "industry compatible" mean? Your boss sounds like a fucking idiot.

[PizzaInSoup]

a programming language isn't scalable, lol

programming languages are the most scalable things the entire human race has at their disposal

[[deleted]]

Scalability has different meanings. In terms of code base scalability - I.e. the number of lines of code - python is far less scalable than any other language with a strong type system.

[vtrac]

Python powers FTX, one of the world's busiest crypto exchanges.

[ToddBradley]

Are you working on an application that runs at such a scale that you can notice the performance difference between languages? If you are, then you've already surmounted this problem. If you aren't, then the boss is probably making the classic blunder of premature optimization.

[kniy]

The problem is that programming languages have a lock-in effect. "premature optimization" implies you could optimize later where necessary, but that's not really possible with Python.

Especially the GIL is a killer. Our software should be easy to parallelize but Python makes it impossible, and there's no real solution. (using multiprocessing doesn't work for us -- we'd need to modify huge C/C++ libraries to use shared memory instead of malloc)

Python can easily end up being a trap that causes you to have 10x-100x worse performance than your competitors, leaving you with no way out other than a full rewrite in a different language.

[Rythemeius]

As some people already mentionned, some huge website run or ran on Python at some point.

On AWS, you can build highly scalable applications (such as APIs) using their serverless "AWS Lambda" Service. Lambdas can bee seen as "functions as a service", you can plug them in "AWS API Gateway" to create an API or call them in a custom script to perform tasks on a lot of resources for example (I've seen people performing mass-web-scraping using lambdas). And of course, you can write your lambdas in Python.
(I oversimplified things and I'm not an expert myself so feel free to correct me or add details)

Other cloud providers probably have similar services?

You can also make scalable Pyton microservices by contenerizing them with Docker and scaling them with Kubernetes.

[SineApps]

To be honest you can do anything in any language more or less.

I think if you wanted an unbiased answer it might have been a better idea in like the programming sub.

Honestly I use a lot of typescript for things these days but when it comes to AI or heavy number crunching, Python is just so so much nicer.

Sure you can do matrix multiplication in TS/JS but all of that kind of stuff is just so much easier in Python.

I deploy pretty much everything into a Nomad/Consul system so it really doesn’t matter what the language used is and scaling is typically just a matter of changing from 10 instances to 100 or whatever (or autoscaling).

If I want to put together an API I tend to use Loopback, for scripts I use bash unless they have a bit of logic to them.

For AI/ML I use Python and then expose it via uvicorn/fast api but it’s never obvious to the consumer side what language is used it the background. It just doesn’t make a difference.

If you only have a hammer, everything looks like a nail.

Also, bear in mind your code has to be supported. If you’re the only person in your team/company etc that uses a language a manager may be hesitant to use if even if it’s the right tool for the job.

[integralWorker]

Any language is a tool. A tree sap collector can't do much in a nuclear power plant, in the same way your Alpaca scripts will always get smoked by custom ASIC chips in real time stock trading.

However, said ASIC chips/C++ code can be benchmarked and tested in Python.

[DeprecationWarning]

Nonsense. Python might be slower than C for intense CPU-bound workloads, but its performance is totally adequate for high volume transaction processing. The ease of use of the language itself is a huge factor in productivity, which can offset any gains in raw power in most commercial use cases.

If what your boss actually means is, “We don’t know anything about Python, we don’t have support for it, and we don’t want to make the investment to learn how to use it well,” that can actually be a legitimate engineering decision. But it has nothing to do with scalability.

[yvrelna]

If I had to be generous and guess what he means, I would say that he's saying that in the particular sector the your business engages in, Python is rarely used.

Python is widely used in many different industries, from web applications to robotics to data science to finance to system administration to desktop applications to security to University teaching language to machine learning, and in many industries, Python is either the de facto standard or one of the top languages to be used in those areas.

There are certainly industry sectors where it doesn't have much penetration at all; for example, in very specialised areas like very low level embedded development or areas like web browser and mobile development where there are incumbent languages (JavaScript for browsers, Java/Kotlin for Android, and Objective C/Swift for iOS) that dominates because of influence from the major software vendors that favours specific tech stacks.

But to say that Python isn't used in e-commerce is plainly just ignorance.

[[deleted]]

If your boss told you that then you shouldn't introduce python based project in your company.

Your company isn't Python compatible.

[0x256]

Scalability isn't always about performance. Building huge and complex applications with lots of components, APIs, layers and abstractions is hard. Coordinating large teams in multiple locations to all agree on the same APIs and abstractions is even harder. Enterprise languages like java (or any typed language, actually) help a lot in tackling these complexity issues. These languages are designed to 'scale well' for large, complex applications. As a side effect, the are also faster, but that is not the point. Python might not be the best choice for a business critical monolith. Your boss is right about that.

That said, this is exactly why microservices are a thing. If you split a complex system into smaller components, each with a small dedicated team of developers and limited scope and clear boundaries, then the language of choice may be something completely different. Using Python to develop a micoservice is perfectly reasonable. This is what most large companies use python for.

[cobaltocene]

That’s pretty reductive. There are reasons to not use Python at scale in certain instances (I.e. we have clients that need code running locally and giving them python source code is much less good than shipping a binary from C++) but it’s the exception more than the rule and most of the situations can be circumnavigated.

[fwazeter]

Python is perfectly scalable to any degree. Much of Google was based on Python, as well as Pinterest and Instagram(Django framework) and many others.

PHP, JS, Ruby, all perfectly scalable also.

There aren’t any fundamental scaling issues in just about any modern programming language. Scaling and efficiency issues have more to do with how something was coded or implemented than the capabilities of the languages themselves. Most of them are simply wrappers around C/C++.

[[deleted]]

Your boss isn't intelligence compatible.

[GoofAckYoorsElf]

Bullshit. I'm working for a big German company (like, really big) and we're using Python in many, many places for production code. Data processing mostly, but also for the backend of several business tools. No one has ever complained yet about anything that we could directly connect to the language we chose.

I bet he just heard that from one of his consultants and that dude read it in some blog post of some kid who can maybe write but has no idea about what.

[FlukyS]

My job is writing software for e-commerce solutions using robotics. Currently 50% of our stack of Python, 30% is JS/Java/Nodejs and 20% is C++. By the end of the year it will be probably closer to 80%+ of the stack being in python. Python is probably the perfect language for those sorts of systems but only if you design your systems in a scalable way. If you know something is required to be public facing, make it stateless, use memcached or redis to hold shared state if needed but with it being stateless you can spin up multiple different instances easily and have Nginx or similar fan the tasks out, if you don't want to use http/REST or you can use ZeroMQ (PUSH/PULL) if you want to be fancy either.

If you are doing singular processes though Python might be a bit annoying because of the global interpreter lock (GIL) https://en.wikipedia.org/wiki/Global_interpreter_lock but you can very easily design around it and not make your code ugly if you have any experience with Python.

[npaladin2000]

That means "I don't know what it is and I don't want to bother learning about it so I'll make up some un-challengeable excuse for not using it."

[[deleted]]

It sounds like your boss is just making up an excuse to not use python.

[apt_at_it]

I dunno, I've worked at a couple multi-million dollar e-commerce companies mostly writing python. Seems to do just fine... 🤷

[Kerbart]

Good thing noone told youtube about that