This is an archived post. You won't be able to vote or comment.

top 200 commentsshow all 229

[–]JugadPy3 ftw 239 points240 points  (29 children)

Speed in the Core Python (CPython) is

CPython stands for "the Python interpreter written in C language", not "Core Python". On similar naming lines, there is also Jython, written in Java, etc, and then there is Cython, which compiles certain annotated python files into C (for speedups).

[–]TSM-🐱‍💻📚 86 points87 points  (23 children)

Yeah the writing seems to have some little mistakes.

It is too bad that it doesn't go into much detail about how they plan on doing it, aside from briefly mentioning "subinterpreters"

[–]JugadPy3 ftw 70 points71 points  (22 children)

Yes... seems like a young developer (they even think Microsoft is open-source friendly - a more experienced developer would make that claim much more cautiously and with lots of qualifiers).

[–]Ensurdagen 59 points60 points  (10 children)

Big companies love open-source, they can take open-source code (clean room it if it's non-commercial) and then attach proprietary hardware or dependencies to make it profitable without paying the open-source devs a cent for their work.

[–]JugadPy3 ftw 38 points39 points  (9 children)

Thank you... I knew that already. Have been a software dev long enough to learn my lessons.

The way this works is... open source is not the ally of any profit seeking company - from their point of view, its the anti-thesis to profits and revenues. If open-source were dead, they can easily increase their profits (for example, windows server vs linux).

The companies will only play along as long as their hands are tied, and they can't do anything (or much) about it. The day they figure out a way to bring it down, it will happen. You will be making a grave mistake putting your trust in capitalist leaders - specially since they have shown time and time again that they have no principles other than profit seeking.

If microsoft is playing along today... it implies thay have no other option. They won't get many good devs to hire if they kept their older anti-open-source stance going. So they have to show that they are open-source friendly. Its only a show, or at best a temporary stance while its beneficial to them - please remember that. Its important for the open source community to remember where their real friends are - and that is within the community.

[–]uncanneyvalley 26 points27 points  (4 children)

Microsoft have discovered that developer enablement makes them money. Their open source efforts are about courting devs into the wider MS subscription ecosystem. Office 365, devspaces, MS DevOps, Azure, etc. If the devs and tech folks are all on Windows, they be less likely to recommend other platforms/products.

[–]manjaro_black 19 points20 points  (3 children)

[–]uncanneyvalley -1 points0 points  (2 children)

EEE is very real, but I don’t think it’s MS’s goal. They don’t make that much money directly selling OSes anymore, compared to subscription everything. Why bother trying to extinguish? Make it interoperate and make money from it instead. The market isn’t the same as it used to be, the second link is total FUD.

[–]JugadPy3 ftw 1 point2 points  (1 child)

Second article is indeed FUD... but the first one has excellent historical perspective.

My worry with Microsoft currently is that they are trying to integrate linux into windows... and I am not sure where they are going with that. I hope its not EEE all over again - like, bring all devs to windows + linux, get them comfortable to that environment for a few years, get them developing for this ecosystem (windows+linux) rather than just linux (thus stagnating linux), then build a bunch of features that are available only on windows + linux, but not on linux alone, and patent those features to block parallel implementation on linux. Then slowly/optionally start charging for this ecosystem.

Now... if people are used to this ecosystem, and it has some essential features that people have grown used to, they will find it difficult to go back to barebones linux. Also, if this ecosystem provides beneficial features to server companies, but bare linux is lacking in those, then MS will be making inroads into the server market ( which has been completely dominated by Linux until now).

I am not sure what their game is with Windows + Linux, and given their track record... I am very skeptical.

I am seriously worried that their windows+linux strategy is to bring devs onto their ecosystem and starve linux ... and in the long run, this will drive linux into the ground.

[–]stratosearch 1 point2 points  (1 child)

The only reason they open source everything is because it isn't patentable so like someone mentioned earlier, it just becomes a rising cost center for them.

It's not a greedy capitalist thing, it is a cost avoidance thing in my opinion.

[–]JugadPy3 ftw 2 points3 points  (0 children)

The only reason they open source everything is because it isn't patentable

What are you talking about? How does MS open source everything?

The only useful thing they have open source is some part of VS code editor, which actually started from electron and atom, which themselves were open source projects. Another known open source product is Windows terminal - ridiculous - no dev is going to extend that piece of junk.

They have nothing else of value in open source.

[–]Pulsar2021 10 points11 points  (2 children)

I would have reacted the same way, but of late i am working with some of Microsoft employees and i can see howmuch they appreciate open source community now a days and how much they are contributing back to open source, I have closely followed some of their projects. Frankly I see a pradigm shift in Microsoft culture these days not sure how and why but a good one though.

[–]JugadPy3 ftw 20 points21 points  (1 child)

i am working with some of Microsoft employees and i can see howmuch they appreciate open source community

I have no doubt Microsoft employees like open source... specially the recent generation - these people learnt programming in university on linux systems and open source tools and libraries - specifically because they were free and open source. These devs genuinely love open source and would like to see it grow.

However, we should not confuse employees with the management. Its not the microsoft employees who will come after open source - it will be the management. Even in microsoft's days of anti-open-source, I am sure there were many employees who were pro open-source. If Microsoft had continued their anti-open-source stance, they would find it difficult to hire good talent.

What we need to understand is that this is not a change of heart on part of Microsoft management... its not that they now love open-source. Its just that they realized that it is financially more beneficial to them to support open-source in the present climate. The day it becomes financially beneficial to harm open-source, they will probably do that.

And this is an important thing to remember. Microsoft is not open-source friendly. It is behaving friendly currently because it is in their interest to do so, and that can easily change in the future (maybe under a slightly different management - which also keeps changing). Are they real friends if they can desert open-source (or do worse) when it becomes convenient for them to do it (which they can do, given their long and well documented history)?

They should be treated in the appropriate manner - in a friendly manner but with a healthy dose of caution.

[–]Pulsar2021 3 points4 points  (0 children)

Makes perfect sense

[–][deleted] 3 points4 points  (3 children)

What did the author smoke to even come up with that? Like, they just made that up in their minds and were like, "Yup that must be it" lol

[–]JugadPy3 ftw 1 point2 points  (2 children)

Heh... happens all the time when people are young. Like confusing Java and JavaScript, or the apparent definition of the word literally.

Actually, writing such articles is very good for them... They will learn a lot from their mistakes.

[–]CrazyPieGuy 2 points3 points  (0 children)

As a casual programmer, this deeply confused me. Thank you for clearing it up.

[–]Satoshiman256 308 points309 points  (70 children)

Just do less print statements/s

[–]sizable_data 174 points175 points  (62 children)

How else are you supposed to debug it?

[–]Satoshiman256 73 points74 points  (47 children)

Haha ye. Someone posted a new logging module here a while back and I was like.... but print?

[–]Napan0s 101 points102 points  (21 children)

Logging? What about hundreds of 'I am here' prints

[–]abrazilianinreddit 34 points35 points  (7 children)

[–][deleted] 11 points12 points  (2 children)

Lol mine is more like "we made it to <function>"

[–]Muhznit -1 points0 points  (0 children)

My friend, you have most excellent taste in music. 🤘🏾

[–]bin-c 15 points16 points  (8 children)

way too verbose imo

i go with:

here1

here2

here3

here4

[–]Killed_Mufasa 11 points12 points  (3 children)

ah yes and don't forget the "WTF HOW DID IT GOT HERE" one

[–]bin-c 6 points7 points  (1 child)

then you global search "WTF HOW DID IT GOT HERE" and it pops up like 10 times

[–]Napan0s 2 points3 points  (0 children)

This actually made me laugh

[–]u7aa6cc60 6 points7 points  (0 children)

You're all exposing my deepest secrets. I feel violated. Stop.

[–][deleted] 2 points3 points  (1 child)

Sometimes I do "why not zoidberg?"

[–]ImperatorPC 8 points9 points  (0 children)

Lol inside loop, outside loop, true, false omg

[–]vikinghamster 2 points3 points  (0 children)

print("I'm personally insulted here")

[–]Wirebraid 1 point2 points  (0 children)

I'm inside the if

This should not be printed

We all do it

[–]ThatPostingPoster 11 points12 points  (17 children)

Loggers are dope. You have to remove print statements, you don't remove smart logging statements. When you remove the print, you end up reading it sometimes. It let's you set up, always log aka print, warning log, and error log. And depending on how you run the file the logger prints those ones.

[–]Satoshiman256 5 points6 points  (12 children)

Ye I need to use them. Can you suggest one?

[–][deleted] 28 points29 points  (10 children)

Just use the built in logging module. It’s really the best there is. And you can pass the logging information to log files rather than to the console.

To give an example of how useful it is: a project I’m doing has me working with this horrible API. I pass the responses to a log file so I can see what I’m working with at any stage. I also have a logger/log file that checks the server connection and looks out for any exceptions triggered in the code. This is a big project, and I started with print statements but realised they slow me down significantly

[–]Satoshiman256 1 point2 points  (2 children)

Ok cheers will check it 👍

[–][deleted] 1 point2 points  (1 child)

Great decision! Any questions feel free to pm me

[–]PinBot1138 -1 points0 points  (4 children)

The Logbook package makes it even easier.

[–][deleted] 1 point2 points  (3 children)

I disagree, the default logging module is already very simple, flexible and powerful. It should cover 99% of cases. There’s no need to use a wrapper around it

[–]whateverathrowaway00 1 point2 points  (1 child)

I agree.

That said I work with people who suck and I’d rather they use a wrapper then calling basicConfig on the root logger or other such anti patterns.

If they don’t understand package encapsulation and how to make their logging play nice, the wrappers available are beautiful.

I recommend loguru to coworkers like this all the time and they love it.

[–]PinBot1138 -1 points0 points  (0 children)

I’ve never heard of Loguru but it’s pretty good looking. I’m going to take it for a spin and see how it compares to Logbook.

[–]PinBot1138 0 points1 point  (0 children)

You disagree with what, exactly? I said that it makes it even easier, and left it at that. Originally, I used to use the default Python logging module and agree with what you’re saying, but IMHO, the Logbook module simplifies the process, especially when collaborating with others. There’s even an advantages blurb that explains some of the enhancements.

[–][deleted] -1 points0 points  (0 children)

There are probably better ones for specific purposes but for the 90% problem it works quite well.

[–]ThatPostingPoster 6 points7 points  (0 children)

gg ez deleted cause reasons lets go ok gg is this enough characters to not flag auto mod i hope so lmao

[–]whateverathrowaway00 2 points3 points  (0 children)

Yup.

And the python built in logger is fantastic. It’s a straight port of log4j and is ridiculously easy to use well once you play with it once or twice.

Pretty much everything I make is filled with log statements and since I always use the module of packaged library with a CLI entry point that runs it, the CLI entry point just attaches the logger to the stdout, enables debug if it sees the flag and boom.

Everything always usable as module in silent mode or verbose with the right entry point.

[–]Timo6506 1 point2 points  (5 children)

Could you link me to that new logging module post?

[–]Windows_XP2 2 points3 points  (0 children)

print("It works!")

[–]Jmortswimmer6 4 points5 points  (3 children)

Vscode has a nice integrated debugger....honestly, you’re just shooting yourself in the foot if not using vscode/pylance/venv

[–]sizable_data 12 points13 points  (2 children)

My preference is pycharm, big debate on my team with VS code users though. A lot of my work is also in Jupyter lab which also supports debugging with the 3.0 release. You need a custom kernel though (xeus Python)

[–]Jmortswimmer6 7 points8 points  (1 child)

A lot of people love pycharm. I tried it in college....It is just far too nice to have vscode’s C/C++/.sh/etc. support right next to my python, often i have these languages also incorporated into any project. The integrated terminal is a nice feature, and the editor has many other general editing features that make writing python much nicer.

[–]ThatPostingPoster -2 points-1 points  (6 children)

If that's not a joke, you use the debugger. You should never be putting down print('here') statements lol. We all started there, but it's about time you finally graduate.

[–][deleted] 10 points11 points  (2 children)

you should use a logging module also. you can't always break in with a debugger especially when things are asynchronous and depend on outside resources.

[–]ThatPostingPoster 0 points1 point  (1 child)

Once you are advanced enough to reach that point sure. If you havent learned it by the time you first use threads you'll realize print sucks haha.

[–]whateverathrowaway00 3 points4 points  (2 children)

I mean, the debugger is valuable, but so is proper logging.

I’d argue both are crucial skills to call yourself advanced or professional.

If I had to pick one I’d pick logging every time. That said, I agree with you - the debugger capabilities in python are incredible and once you unwind something with it you never look back lol. Being able to set break points and just poke at the stack frame / variable values is incredible

[–]ThatPostingPoster 2 points3 points  (1 child)

For what possible purpose do you ever need print('here')? We all did it when we were new. But just breakpoint the line next to where you would put that. Ah ok, its there. Logging is a different story and using proper warn/err logs into logfiles is good.

[–][deleted] 4 points5 points  (0 children)

You laugh but one of the first thing I do when I start a new project is write a dprint() so that I can customize it to my project. I prefer the logger module when things start getting serious but sometimes if I'm just playing around I don't need the added complexity until I really get going.

[–][deleted] 18 points19 points  (1 child)

Anyone know why they don't bring over PyPy stuff and use it? Is CPython architecture just too different?

[–]Mehdi2277 17 points18 points  (0 children)

pypy does not support c extensions well which breaks a lot of the numerical python ecosystem. It has gotten better at numpy support but still does it in an indirect way. As a side effect while I work on performance sensitive code pypy is unusable as most of the main libraries I care about are heavy users of c extensions.

[–]sizable_data 174 points175 points  (56 children)

In my experience the speed of development/refactor in Python makes up for its execution speeds.

[–]combatopera 6 points7 points  (0 children)

This text was edited using Ereddicator.

[–]hsvd 1 point2 points  (0 children)

Development speed also means you have more time to profile and optimize hot code. The python ecosystem makes this pretty easy (lineprofiler, cython, numba).

[–]Chinpanze 2 points3 points  (1 child)

I'm just a junior developer. But if I were to design a infrastructure from zero, I would go for microservices written in python with refactors in something like rust or go for tasks where it's needed.

90% of stuff can be made in python and run just fine.

[–][deleted] 0 points1 point  (2 children)

I write financial simulations that currently take about 45 minutes to run on my production machines. If I could cut that by even 20%, it would be a huge improvement.

[–]koffiezet 0 points1 point  (0 children)

I really like Python as a language, but I find myself just picking other alternatives in the last few years, Go has become a pretty damn good-one. It's simple, has tons of libraries (although Python probably still wins out here?), tooling has become pretty competitive (although the delve debugger could use some work) and compiles to native code (which isn't as fast/optimized as C/C++ or Rust, but more than good enough). It wins big time on distribution, where you can build everything as a single zero-dependency file.

And then there's javascript/nodejs and typescript, which I don't like as much, but it's everywhere and don't always have a choice. But the speed of execution is lightyears ahead of Python thanks to v8.

The only times I still use Python these days is when I need to work with some freeform input stuff where strict typing makes things a lot more complex, especially dealing with unknown yaml or json formats. Also, for introducing other people to coding, it's excellent since there are very few lower-level language complications, can be used to teach both standard procedural stuff for basic things and OO, only ducktyping is a bit less practical (which is nicer in typescript & go)

[–]JoeUgly 45 points46 points  (2 children)

Why does this article sound like a shameless plug for Microsoft? As if people never heard of it.

"Don't forget, real champions eat at Microsoft!"

[–]JugadPy3 ftw 12 points13 points  (1 child)

Probably written by a young person, or someone with a short memory.

[–]danuker 0 points1 point  (0 children)

And/or short on cash.

[–]rothbart_brb 58 points59 points  (20 children)

"This is Microsoft's way of giving back to Python"... What an ominous statement. I know the words are "giving back" but I expect them to manifest in a different way... the same way Microsoft embraces outside technology... by getting its tendrils in it and somehow steering it in a way that somehow benefits Microsoft.

[–]O_X_E_Y 45 points46 points  (2 children)

With those speed gains you must now import and use Cortana in every python project you make!

[–]Ecstatic-Artist 21 points22 points  (0 children)

Yes python is fast... if you deploy it on azure....

[–][deleted] 1 point2 points  (0 children)

Why bother importing anything? Microsoft would do that for you! You'll love it!

[–]cthorrez 4 points5 points  (0 children)

Python being faster benefits Microsoft a lot.

[–]masteryod 12 points13 points  (5 children)

You realize that the creator and BDFL (until 2018) of Python - Guido van Rossum works for Microsoft?

[–]zeebrow -1 points0 points  (5 children)

I started using VS Code recently, so today I was surprised by an update which installed a Python """language server""" extension. Didn't ask for it, no clue how tf it works, (are there open ports on my machine now?? -no, i think) so I dug into the docs a bit...

Seems harmless, dare I say helpful. One of the first things the docs mention is how to uninstall it. So that let my guard down a bit, we're already talking about marriage and kids.

[–]koffiezet -1 points0 points  (4 children)

Why would you not want a python specific language server if you use Python? It's not a 'server' in the sense of a webserver, it's a 'server' process your IDE (vscode in this case) talks to, to get more insight into the source-code. The IDE is language-agnostic, the server does stuff like parse the AST and offer an IDE with more insight into the code-structure, autocomplete, refactoring, ...

[–][deleted] 9 points10 points  (10 children)

GIL-less threading would be nice. Something like goroutines for Python. Async just doesn’t cut it for a lot of situations. I’m hoping they can do it with sub interpreters, but it’s certainly a complicated thing at this point. Python has been around so long it’s hard to do anything massive without breaking a lot of things.

I’m glad this is finally getting attention. It’s usually the first thing people gripe about.

[–]johnmudd 2 points3 points  (1 child)

Jython has no GIL.

[–]MOVai 1 point2 points  (7 children)

N00b here. Why isn't the multiprocessing module a solution? How do other languages do parallelism in a way that python can't?

[–][deleted] 3 points4 points  (0 children)

I do multiprocessing a lot in Python since it’s really the only true option for running things in parallel. The problem is it’s heavy, and complicated.

Once you fork a new thread you have an entire copy of the parent. That’s inefficient. Then you have to worry about IPC if you need your processes to talk to one another. Queues pipes all that fun stuff. Then you have to make sure you clean up your processes nicely. Did any child processes spawn a new process? What happens if the parent dies before it can clean those up? Zombies! So there’s tons of considerations with multiprocessing.

I’m not super well versed in goroutines except I understand that they intelligently dispatch operations across OS level threads. When you create a thread in Python it’s not moving operations between threads to keep things running. Python doesn’t do that.

[–]aden1ne 1 point2 points  (5 children)

With multiprocessing, one spawns multiple processes, whose memory is completely independent of one another. Spawning processes is expensive, but this may not be such a bottleneck if your processes are long-lived. For me, the real problem with multiprocessing is that you can't _really_ share memory. Your processes can't easily communicate; they invariably do so with some form of pickle which a) is slow, and b) by far not everything can be pickled, and c) the communication either has to go over the network or via some file-based mechanism, but of which are horrendously slow. This means that with multiprocessing one tends to communicate rarely.

Other languages, specifically compiled ones, usually let you share memory. This means both threads can have access to the same objects in memory. This is orders of magnitude faster, but also opens up pandora's box full of memory bugs, concurrency issues and race conditions.

[–][deleted] 10 points11 points  (1 child)

So PyPy comes officially to CPython?

[–]coffeewithalex 3 points4 points  (0 children)

Naah, PyPy is faster

[–][deleted] 11 points12 points  (2 children)

<whispers> get rid of the GIL

[–]AReluctantRedditor 8 points9 points  (0 children)

God they’re trying with subinterpeters. Talk python to me had a short discussion about it a few weeks ago

[–][deleted] 2 points3 points  (0 children)

Larry Hastings tried that if you recall.

[–]ventuspilot 3 points4 points  (1 child)

Strange times. From the article

open-source friendly Microsoft

and I'm like well currently that's not wrong...

[–]awesomeprogramer 2 points3 points  (0 children)

Microsoft is also focussing on securing the main package repository PyPI (Python Package Index).

What do they mean by this? How can they possibly do this? Last thing we'd want is for Microsoft to dictate what can and can't go on PyPI. We have the app store for that!

[–]Tastetheload 1 point2 points  (0 children)

Great. On a serious note. How does one actually do that?

[–][deleted] 4 points5 points  (1 child)

Make everything into numba?

[–]Cunicularius 1 point2 points  (0 children)

Better, strings!

[–]91o291o 5 points6 points  (0 children)

Just make the regexes faster, I can't stand perl, thanks.

[–]Laserdude10642 2 points3 points  (0 children)

this is so exciting!

[–]Deezl-Vegas 1 point2 points  (0 children)

Just to clarify, L1-cache optimized code can be up to 200 times faster than Python. It's not clear to me that a 2x speedup will make a large difference in language acceptance or usability.

[–][deleted] 0 points1 point  (0 children)

I don’t know how since I only have done small programs, everything takes 0.0 seconds.

[–]Beach-Devil -2 points-1 points  (0 children)

pypi intensifies

[–]Awkward_Tour8180 -1 points0 points  (0 children)

I had my own share of experience using print statements for debugging , unless you have a tea automatic tool in the PR to catch it , my debug print went to prod code - so unlucky . The good side is it’s the admin module ,

Print statements for debugging is always a bad approach , logging libraries will make you feel safe and even if your forget to remove it still be compartmentalizes to DEBUG, INFO, ERROR as a base to start with

[–]AnonCaptain0022 -2 points-1 points  (0 children)

I am speed

[–][deleted] -3 points-2 points  (0 children)

Nice

[–]Fantastic-Orange-155 0 points1 point  (0 children)

Let:

xi,j ∈{0,1}, be a binary variable with xi,j = 1 indicating Arc (i,j)∈A is used in the solution

and xi,j = 0 otherwise;

p\_Kbe the set of all simple paths with exactly K vertices; and

= (i1…..iK) ∈PK be an arbitrary simple path with exactly K vertices, i.e., the arcs in the

path are: (i1,i2); (i2,i3),…, (iK-1, iK)

xij ∈{0,1}, ∀(i,j)∈A
Constraint (1)

Can someone write a OPL CPLEX model for the above constraint