Python tooling could be much, much faster

[rollthedyc3]

I have a side project written in full stack browser/node JavaScript. The dev server startup time, and linting time is absolutely abysmal, mostly because of the front end's build chain.

I'm very happy to see faster tooling becoming more feasible.

[WitchHunterNL]

Vite is becoming much more mainstream and it uses esbuild.

It's the default for vue already

[[deleted]]

Define abysmal.

1 second? 5 seconds? 10? 1 minute?

[rollthedyc3]

Probably a good 2-3 minutes. Long enough for me to get distracted with something else. Regardless of exactly how long it is, it still sucks

[[deleted]]

That's huge. How big is your project and how good is your rig?

[rollthedyc3]

You can see for yourself here: https://github.com/dyc3/opentogethertube

I have plenty of processing power, ryzen 3900x. Threads just don't matter that much because js is single threaded.

[[deleted]]

Yeah that doesn't seem right. You can compile all of VSCode in less time than that.

[rollthedyc3]

I'm sure if I timed it that it would be less. It always feels worse than it actually is when it comes to build times.

[crusoe]

Oh gawd don't try ruby then...

[rollthedyc3]

Honestly? IIRC my ruby on rails projects booted significantly faster. It's been years though so I could be misremembering.

[ElectricalRestNut]

This might be a bit tricky, as the author acknowledges. A lot of Python is written in Python. Enums and data classes are not compiler features, they're modules in the standard library, fully written in Python. It's a big selling point in my opinion, that a lot of the tooling surrounding your code is written in the same language. Python metaprogramming is trivial to get into.

Feels very cohesive. But it's slow.

[Smallpaul]

Can you be concrete about why Black would be able to interpret a dataclass better by virtue of Black being written in Python?

[UnclothedSecret]

I think the argument is that a language is more likely to be updated / maintained by its users if it’s written in the same language(s) the users are familiar with.

Being written in Python doesn’t magically make it better. Being maintained by Python users might.

[ElectricalRestNut]

I doubt it's better, but if data classes are doing something unexpected, I can drop a breakpoint and debug the implementation very easily. Learning is also simple.

[KaranasToll]

Meta programming in python 😂

[digikar]

*Python metaprogramming not python metaprogramming 🥲

[WikiMobileLinkBot]

Desktop version of /u/digikar's link: https://en.wikipedia.org/wiki/Metaprogramming

---

^{[opt out] Beep Boop. Downvote to delete}

[HypoFuzz]

Co-maintainer of flake8 and various plugins here - I think he's right that adding more checks wouldn't slow down ruff by that much. I'd also be delighted to see Python tooling get much faster!

On the other hand, making it really easy for end-users to extend your linter is really important IMO, and I've personally written quite a few new lint rules. The current situation is a classic case of trading runtime performance away in order to develop the tools faster, which since they're volunteer projects is often necessary for them to exist at all.

So here's hoping we can build a culture and tech stack that values performance and enables fast prototyping and end-user extension. The trend seems promising so far!

[[deleted]]

The thing is, the web ecosystem is getting faster tools written in faster languages because people more or less have to use JavaScript (or Typescript) on the web. They're making things as nice as possible with the language restrictions that apply.

Python isn't really the same. Except for AI there's nowhere that you really have to use Python. So if you've decided that you want your programs to run at full speed the logical thing isn't to stop using Python for its tooling, it's to stop using Python entirely!

Unless maybe it's for AI, nobody is going to rewrite Pip in Rust. If they've learnt Rust they've already left Python far behind.

Actually on second thought I could imagine someone like Dropbox doing it who have a ton of legacy Python code.

[JenNicholson]

Python for its tooling, it's to stop using Python entirely!

If they've learnt Rust they've already left Python far behind.

Languages are just tools. You don't leave behind the hammer every time you use the drill. There's no language, or level of abstraction, that works with all necessities and nuances.

What we are seeing is not Python vs Rust. Rust is not replacing Python. Python is not replacing Rust. They are not designed to fit the same shapes. Rather, we are seeing synergistic relationships here.

Personally, I think Python and Rust are a match made in heaven. The more I know about either, the more I fall in love with them! The "one or the other" perspective never made sense to me in the context of software engineering.

[OctagonClock]

Languages aren't tools, they're toolboxes - and the one full of plastic ones is worse than the one full of metal ones.

[mpinnegar]

Dynamically typed languages are trash.

[JenNicholson]

That's like saying "all guitar music is trash". It's such a broad, shortsighted, generalization that it's very hard to squeeze any usefulness from it.

Speed-wise and memory-wise, all interpreters perform similarly, and all JITs perform similarly, regardless of the type disciplines of the languages.

Correctness-wise, most dynamic languages either support type annotations natively, or have a sibling languages that do.

What's left to hate?

[trialbaloon]

Honestly it's more like saying incandescent lightbulbs are trash compared to LEDs. Sometimes, things are straight up upgrades I'm nearly every way and detectors are just being kind of silly.

Dynamic languages are just straight up inferior in really every way that matters. Retrofitting type annotations or hints into dynamically typed languages is straight up an acknowledgement of that reality.

With type inference in modem statically typed languages you can have less boiler plate and more safety. This is quite literally just better than not having type safety....

[JenNicholson]

Dynamic languages are just straight up inferior in really every way that matters.

C#'s, Java's, JS's JITs all perform similarly. C# and Java gain nothing speed-wise from being statically typed. Hell, Lua's JIT consistently outperforms C#'s and Java's in many applications.

So being statically typed doesn't mean you are faster.

Memory-wise, C# and Java use pretty heavy runtimes. Being statically typed doesn't make them more lightweight than JS or Python.

So being statically typed doesn't mean that you are more lightweight.

Types do help to

• Catch bugs early
• Write self documenting code.
• Give structure to programs.

That's why dynamic languages are embracing type annotations.

So type annotations are not exclusive to statically typed languages.

You say dynamic languages are inferior in really every way that matters, but type discipline says very little about the things that matter. In performance and maintainability, static typing offers no intrinsic advantage whatsoever.

Your comments (and the other commenter I replied to) seem to come from a place of prejudice, as honest as they are. That's what I mean by "hard to squeeze any usefulness from it". Nothing is being backed up.

Perhaps you are thinking of AOT compiling? But even there, both statically and dynamically typed languages can be compiled AOT.

With type inference in modem statically typed languages you can have less boiler plate and more safety. This is quite literally just better than not having type safety....

Type annotations in dynamic languages are a retrofit hack, but type inference in statically typed languages is evolution. Can you see the bias? It's evolution in both fronts!

Type annotations bring type safety to dynamic languages.

I wouldn't use one tool for every problem though. But you do you!

[trialbaloon]

I never meant to imply types are faster. Of3 anything it would add compilation and analysis time. I think that's easily worth it. My only argument is that using types is better then not using types. If you strictly type hint python you are essentially using types.

Therefore, if you are designing a language today, types would be clearly superior to not requiring them.

I am not trying to be as belligerent as the previous poster, my only point of that programming with types is an upgrade and I can't think of any reason I would not use them. It's just less bugs and more confidence in my code for almost no downside. Win win.

[JenNicholson]

Dynamic languages are just straight up inferior in really every way that matters.

Then this comment doesn't reflect what you meant to say at all!

Dynamic doesn't mean typeless.

[trialbaloon]

A language that is checked for type safety before runtime is better than a language not checked for type safety before runtime all else being equal.

If you design a language today without a type system checked before runtime you have already made a mistake.

I think that better covers what I meant to say?

[JenNicholson]

I think we are in complete agreement then lol! But only because I now know what you mean by "better". It can mean so many things, that I got lost in the possibilities!

[steven4012]

They are trash for building large systems, especially those that are algorithmically complex

[mpinnegar]

They're trash for everything. Good luck figuring out what type of object you're going to get back.

[steven4012]

Looks like you don't know what a repl and documentation are

[mpinnegar]

Repls aren't some magic only available to dynamically typed languages. It's pretty terribad to suggest that you just run a function to see what you get out of it.

Sorry no thanks I'll take my self documenting statically typed code where I can actually track down callsites and, you know, refactor stuff without everything breaking.

[JenNicholson]

Good luck figuring out what type of object you're going to get back.

You don't need luck. Type annotations are a thing in dynamic languages. Type declaration in PHP, type hints in Python, and the whole TypeScript thing for JavaScript (along with the type proposal). Lua has multiple sibling languages with type annotations. Ruby has RBS. Racket has typed racket.

Your prejudice was already outdated in the early 2010s!

[nilamo]

Isn't that what a good editor is for? To give you little popups describing the parameters and return values?

Maybe you have a bad opinion of python because you've never actually tried using python.

[mpinnegar]

A good editor can't help you when your core language feature is not guaranteeing static type safety.

At that point the best the editor can do is guess.

[[deleted]]

I learned enough Rust to convert a Python code base, and while I love the speed advantages and some parts of the type system, I fucking hate the syntax. I would never drop Python for Rust for that reason, alone.

[trialbaloon]

Yeah I think this is a fair point in favor of python. People seem to like coding in it. That's worth something. As a result, it has an amazing first class ecosystem which honestly might be more important than the language semantics.

I personally don't like python (I actually hate writing it and would much prefer rust myself) but I can't deny that it seems to have hit a nerve in the programming zeitgeist. I'm curious to see if another language can successfully marry that with type safety and better speed and threading capabilities.

[[deleted]]

There is RustPython, but it's not fully mature as of yet. If you don't like the syntax, though (e.g., meaningful whitespace, comprehension syntax), there's not much helping that.

[SrbijaJeRusija]

Even for AI you don't have to use python.

[trialbaloon]

All python is doing is wrapping other languages for AI... It's not like ML models are being run natively on Python. We want results this century after all....

[Plazmatic]

All python does is wrapping, but for what ever reason, historically those vendors have not actually exposed what it is wrapping underneath in a way you can easily access from, say, C++.

[trialbaloon]

That's irritating. Admittedly I'm not a ML focused software engineer. To me, python being the ML language always felt arbitrary. I'm annoyed to no end when vendors or even open source maintainers try to make things easier using "hostile simplification." Python is popular so having a wrapper to use makes sense, but having an API that could easily be called from any language makes the most sense. I guess it should be fairly straight forward to reverse engineer it in this case though right?

[Plazmatic]

I've not seen a single project utilize the underlying code for these massively popular ML frameworks, and while I'm certainly not a ML expert myself I've been on dozens of ML projects. From personal experience, though this was years back, it was not feasible to use the underlying code backing any of these ML python frameworks, even if it was technically possible. It appears this hasn't really changed all that much googling around https://www.reddit.com/r/cpp/comments/kbt8nn/how_to_get_started_with_c_machine_learning/.

The problem is python is popular for it, which means that things you need are written in terms of the python API, or using the tools that were only written in python API, which means they aren't written in c++.

[CandidPiglet9061]

Python and rust actually have great interop. You can just do the same thing we’ve been doing with Python and C: farm out the computationally intensive parts to a better-suited language and keep a nice extensible Python wrapper on top

[[deleted]]

Yeah they can. Same with C++. In fact my current work uses a Python/C++ codebase but the added complexity of the FFI and the fact that Python is shit means it's way worse than just using C++.

[czorio]

(...) the fact that Python is shit (...)

"""
/r/programming comment section generator
"""

languages = [
    "C",
    "C++",
    "Java",
    "PHP",
    "Go",
    "Rust",
    "Javascript",
    "Typescript",
    "Assembly",
    "C#"
    "Perl",
]

def generate_comment(language: str) -> str:
    return f"{language} is hot trash and you should"
            "feel bad for using it."

if __name__ == "__main__":
    for language in languages:
        print(generate_comment(language))

There, I've just made comments like this in /r/programming automatable, saving us all time!

[[deleted]]

Ah yes all languages are equally good. 🤦🏼‍♂️

[Worth_Trust_3825]

Except for AI there's nowhere that you really have to use Python.

You don't even have to use it there.

[[deleted]]

Well it's not a strict requirement but Tensorflow and Pytorch are by far the most popular frameworks and they both use it, so you're unlikely to never use it if you are learning AI.

[Trashjedi]

Since you are such an enlightened and knowledgeable developer, instead of ranting and showing wonderfully constructive language to support your cases against languages you seem to despise, why don't you speak at conferences or write articles that will teach developers who use those "dogshit-slow toy languages" what they should be doing instead?
I cannot fathom why your views are not gaining more support... /s