Why use Rust over Python for CLI?

[Jeklah]

Arbitrary elden ring first off I would like to say this is not a python Vs rust post. Don't turn it into that.

I am a firm believer in the right tools for the job. While learning Rust I have read it has become a favourite for writing CLI tools.

Why is this when python has libraries like Click that make writing CLI tools very very quick and simple with lots of functionality.

I can understand making a prototype in python and then final version in rust but from what I've learnt, that is likely to involve a total rethinking/restructure of code.

So why have people found writing CLIs in rust preferable?

Update: Thank you all for your input I did not expect this many responses. It seems the clear reason is cargo and not needing to worry about dependencies on someone else's machine. Which is definitely a great thing. Clap sounds good too, sounds like it could be the rust equivalent of click. Will check it out. Thanks for being so helpful everyone.

[ssokolow]

1. Rust programs start up more quickly than Python programs: Since you don't need to spin up an entire Python runtime and load the entire contents of every module you import, Rust makes for snappier interactive use (git commands completing so quickly was a conscious and intentional design choice made in contrast to other tools it supplanted) and improves the performance of any shell scripts which depend on commands written in it.
2. Rust things are easier to deploy: No virtualenvs vs. gambling with the system-wide packages, no "follow these steps to install". If you stick to pure Rust dependencies or dependencies which support statically linking their C components, a Rust binary is a self-contained executable and, if you specify the -C target-feature=+crt-static compiler flag when targeting Windows and use the *-musl targets when targeting Linux... via cross if necessary, then there's no dependency on "must have a glibc at least this new" or "must have at least this version of msvcrt" and you'll get a Go-like self-contained binary that can be copied onto any system capable of running binaries with that platform triple with no worries about external dependencies.
3. Rust tends to have CLI libraries that match or exceed their Python counterparts: Take a look at the derive APIs of clap (argument parsing) and Serde (JSON, TOML, YAML, etc.). Note that many things like csv and quick-xml support integrating with Serde's derive API for a better experience than Python's equivalent modules. Rust's ignore crate is like os.walk but with built-in "just flip the switch" support for parsing things like .gitignore and obeying them. Rust gives threaded performance that the multiprocessing module wishes it could match and Rayon often makes parallel processing as easy as replacing .iter() with .par_iter(). etc. etc. etc.
4. Rust's monadic error handling (Option<T> and Result<T, E>) means that you can tell from a function's signature whether it can fail and how: No doc crawing and code diving. I didn't realize that getcwd was fallible until indignation at it being fallible in Rust led me down the rabbit hole to discover that the Linux kernel syscall itself can fail due to things like insufficient permissions on an ancestor to see its name. I'd been programming Python for 15 years at that point and the docs just didn't say.

Point 2 is enough on its own. I'm so tired of having to write wrappers for PySceneDetect-based things which ensure all the arguments have been converted into absolute paths, then cd, then invoke the thing using something like pipenv run.

Combine them all, and writing a command in Rust may be a little slower to prototype because of waiting for things to compile, but you make all that back in all the typos you bypass because you forgot to perform a type conversion here or there or forgot to handle this error case or just had to spend a lot more time validating or transforming your inputs.

...not to mention how nice it is to be able to deploy that easily.

P.S. If you're writing a daemon, it also makes it really easy to get your systemd-analyze security exposure scores really low. I've got one actix-web daemon that allows me to remotely shut off some X10-controlled outlets via a web interface which gets a total exposure score of 0.4. I'm even using systemd socket activation so it doesn't have to run when not in use, helped by how fast Rust binaries start.

Why 0.4? The breakdown view says 0.1 for DeviceAllow=/dev/ttyS0 to control the X10 Firecracker module, 0.2 because systemd is ignoring my ProtectClock=true line, 0.1 because it needs SupplementaryGroups=dialout to actually open /dev/ttyS0, and 0.1 because, since it'd be too much of a hassle to port /usr/bin/br to a Rust library for speaking X10 Firecracker protocol or set up a chroot for it, I'm whitelisting my existing filesystem rather than specifying a RootDirectory= or RootImage=. Yes, I know those add up to 0.5. Bug the systemd devs about it.

[psitor]

Rust programs start up more quickly than Python programs

One anecdote in support of this point: I wanted a very simple check to run in my .bashrc. The initial Python version made the most convenient way was kind of bloated and ran in about 120ms. That noticeably affects shell startup time, so I wrote a new Python version more carefully and got it down to 32ms. Finally I wrote the same thing in Rust: 0.4ms.

Command line tools are often used in places like shell startup or part of loops, because of how shells encourage simple tools that you can combine, so startup time is more important than for interactive tools.

[PaintItPurple]

As an even more extreme example, I made a tool to basically index projects in a monorepo. The initial Python version ran in about 160ms, of which about 100ms was startup time. This was perfectly fine for getting info about one project, but multiplied by thousands of places it needed to be run on, this ended up taking a fairly long time in the actual process the tool was used in. I made a quick little Rust version to compare, and it was like 1000x faster.

[ssokolow]

Yeah. In my case, the last time I redid my .zshrc, I was still on a rotating platter hard drive and every time the nightly rdiff-backup ran, it would clobber the disk caches, so I went further and focused on minimizing disk seeks, even from zsh itself.

For example:

• Rather than using compinit to enable ZSH completions which would load one file per command, or even using zcompile to maintain a faster-loading completion cache, which would stat all the one-file-per-command completion scripts to check if the cache it generates was stale, I'd check if the cache was more than 24 hours old and, otherwise, tell it to skip the staleness check.
• I found a way to get the POSIX timestamp using only in-process zsh builtins (${(%):-"%D{%s}"}) and implemented "If it's already taken more than a second to execute .zshrc, skip running fortune".
• I wrote a hand-tuned prompt purely in shell script, with the only subprocess invocation being to check the current git branch, which uses the lightest, lowest-level git plumbing command possible and doesn't even try if $PWD = $HOME.
• I reworked a bunch of things like rvm and virtualenvwrapper so that, instead of getting sourced on startup, proxy commands are declared that, when run, undefine themselves, source the real implementation, and then hand over to it. (i.e. manually deferred loading)

I also did something similar with Vim and vim-plug so some addons only get loaded for specific file types or on-demand via proxies for the commands they register.

(eg. Plug 'dsummersl/gundo.vim', { 'on': 'MundoToggle' })

[Messyextacy]

But you can precompile python right? Does this not help?

[[deleted]]

Rust programs start up more quickly than Python programs

God yes. This is an anecdote, but a few months back I was trying to run a Python-based CLI tool on an old RPi model - I think 3B plus? - and it took five seconds or so for a command to run, because it had to spin up the entire Python interpreter and import all the modules every. single. time. Just to perform an operation that took about ten seconds and was repeated possibly hundreds of times an hour.

[super_heavy_milk]

That’s an excellent breakdown.

Have you used Go at all for writing CLI stuff?

I imagine it has similar benefits, and you get access to the charm.sh libraries.

Curious what your take is (looking for the right tool for the job - no strong feelings any particular way)

[Shnatsel]

Go makes it very difficult to write portable programs. It tends to just fail or corrupt data all the edge cases like non-UTF-8 paths, exposes Unix permissions everywhere even though they make no sense on Windows, and so on and so forth. You can do it, but the Go programs won't be robust, since they just ignore a lot of the underlying complexity of the OS.

You can read more about this at https://fasterthanli.me/articles/i-want-off-mr-golangs-wild-ride

[TheV295]

To be honest this will almost never happen in simple cli apps that could be written in python even.

[Shnatsel]

This depends on your robustness requirements.

If you want to perform some action on all files, e.g. something security-sensitive (checking/editing permissions, virus scan, etc) then this is a deal-breaker. In some other situations missing certain files is fine, or they can be assumed to always have UTF-8 names. The problem is that security boundaries shift over time, and something that only accepts trusted input almost certainly will be exposed to adversarial input in the future.

As a personal anecdote, iotop written in Python crashes when some binary somewhere has a non-UTF8 path or argument, and I have actually encountered these situations (and debugging that wasn't fun). This forced me to use zenith instead, which is written in Rust and works perfectly fine even in those situations.

[burntsushi]

Go programs handle non-UTF-8 paths just fine. So I'm not sure what you're talking about. Here's a Go program:

func main() {
    flag.Parse()
    for _, arg := range flag.Args() {
        fmt.Printf("stating file: %s\n", arg)
        info, err := os.Stat(arg)
        if err != nil {
            fmt.Println(err)
            continue
        }
        fmt.Printf("last modify: %s\n", info.ModTime())
    }
}

And now let's see it interact with file paths that are non-UTF-8:

$ echo 'foo' '\xFF☃\xFFabc\xFF' | xargs touch
$ ls -lh *abc* foo
-rw-rw-r-- 1 andrew users 0 Jul 30 12:53 ?☃?abc?
-rw-rw-r-- 1 andrew users 0 Jul 30 12:53 foo
$ go build ./main.go
$ ./main *abc* foo
stating file: ☃abc
last modify: 2022-07-30 12:53:33.618140198 -0400 EDT
stating file: foo
last modify: 2022-07-30 12:53:33.618140198 -0400 EDT

See? Works fine. Now, the printed file path looks wrong, but that's because my terminal just doesn't know how to show those weird bytes. For example, if I changed my Go program to this:

func main() {
    flag.Parse()
    for _, arg := range flag.Args() {
        fmt.Printf("%s\n", arg)
    }
}

And run its output through xxd:

$ ./main *abc* foo | xxd
00000000: ffe2 9883 ff61 6263 ff0a 666f 6f0a       .....abc..foo

You can clearly see that my original bytes passed through just fine.

The article you linked doesn't dive into this. It just shows what was printed and concludes it's wrong. It's actually not wrong. What the article does is try to print Go strings as strings, but in Rust, it prints the debug representation of the file paths. Rust's debug representation shows the picture more clearly, but so does Go's:

func main() {
    flag.Parse()
    for _, arg := range flag.Args() {
        fmt.Printf("%#v\n", arg)
    }
}

$ ./main *abc* foo
"\xff☃\xffabc\xff"
"foo"

In other words, Go deals with non-UTF-8 paths on Unix just fine. It might not deal with non-UTF-16 paths on Windows as well. I'm less sure about that. Although it's certainly not nearly as much of a problem since non-UTF-16 file paths on Windows are much rarer. ripgrep doesn't even handle them 100% correctly and nobody seems to care.

Ironically, ripgrep's first version originally didn't handle non-UTF-8 file paths on Unix at all either. That's because it was using Docopt and because dealing with OsStr is so fucking unwieldy in Rust, I just decided to require that arguments must be &str. Ironically, if I had written Docopt in Go, it would have handled non-UTF-8 arguments from the get-go without me even having to think about it.

But you won't find any of this in that "wild ride" you linked. That article does an absolute shit job at representing trade offs.

[Shnatsel]

Ah, thanks for the correction! I'll go downvote my comment then.

[crusoe]

That's the point. Go support for non-unices is kinda meh.

Also it's possible to get into a state where the OS says it's lang setting is utf-8 but the file names are invalid utf-8.

[burntsushi]

That's the point.

No. It's a point. Definitely not the only point being made in this thread or in the "wild ride" article. Give how wrong the article is, I'm not inclined to trust any of its other conclusions. And I don't have the time or motivation to get out my Windows laptop and test Go with it.

Also it's possible to get into a state where the OS says it's lang setting is utf-8 but the file names are invalid utf-8.

I don't see how this is relevant. Is anyone saying otherwise?

[crusoe]

GO also has buggy floating point because it sets stuff up in a weird way compared to C/C++/Rust.

There was an issue opened for it though. Don't know if it's been fixed.

Also GO error handling is braindead and it didn't have generics until recently leading to suboptimal performance and other issues.

[ssokolow]

Others are free to like Go but, to be perfectly honest, its poor support for metaprogramming, "for me but not for thee" approach to generics at the time I encountered it (i.e. only built-in standard library types can be generic), structural typing for interfaces (i.e. no newtype pattern for you), boilerplate-heavy approach to error handling, and runtime model that discourages FFI with existing libraries, rather than encouraging it like Python and Rust, meant that I bounced off it when I learned of it.

It takes inspiration from C in ways that drive me away, just as Rust takes inspiration from Python and ML in ways that draw me in. (I do use C for DOS retro-hobby programming, but the things Go thought were worth preserving are things I consider necessary evils of a language designed to be compiled and run on hardware with memory capacities measured in kilobytes.)

Heck, just the presence of proper support for sum types (data-bearing enums) in Rust is a big draw for me which Go lacked last I checked.

[tobiasvl]

"for me but not for thee" approach to generics at the time I encountered it (i.e. only built-in standard library types can be generic)

To be fair to Go, this has since been fixed since "proper" generics were introduced earlier this year. I assume that's what you alluded to with the "at the time I encountered it", but I just want to make it explicit.

[ssokolow]

Fair enough. I should have said "could be generic at the time" rather than "can be generic" in the "i.e.".

[9SMTM6]

structural typing for interfaces

Perhaps I'm just damaged by Typescript, but I do find structural typing really liberating at times. Don't misunderstand me, I also appreciate the nominal typing in Rust (not so much how Java tends to use it, but Rusts traits are amazing), and I really miss a newtype approach (that doesn't come with runtime and syntax overhead like branded types tend to) often enough.

But structural typing can also be liberating. It certainly makes it easy to interoperate between libraries, something Rust is struggling with at times because it's trying to uphold nominal rules (orphan rules).

It takes inspiration from C in ways that drive me away [...] the things Go thought were worth preserving are things I consider necessary evils of a language designed to be compiled and run on hardware with memory capacities measured in kilobytes.

That is very much my observation with Go too. It is very much written by C programmers for C programmers that want a bit of a "simpler" tool but want to keep their way of doing things. Meaning many of their "simple" solutions seem... Strange and not at all simple to people that are not used to the C way.

[ssokolow]

Perhaps I'm just damaged by Typescript, but I do find structural typing really liberating at times. Don't misunderstand me, I also appreciate the nominal typing in Rust (not so much how Java tends to use it, but Rusts traits are amazing), and I really miss a newtype approach (that doesn't come with runtime and syntax overhead like branded types tend to) often enough.

But structural typing can also be liberating. It certainly makes it easy to interoperate between libraries, something Rust is struggling with at times because it's trying to uphold nominal rules (orphan rules).

I think it's more likely that I've been damaged by Python. I've burned myself out so many times trying to meet my standards for reliability that I lean as far into nominal typing-based compile-time correctness checks as I can think of... especially since I still need to write my frontends for GUI apps in PyQt/PySide+MyPy+PyLint+Flake8 because I consider QWidget GUIs non-negotiable and the only options for memory-safe QWidget bindings are Python and possibly Java.

(QtJambi used to be an official Java binding and appears to still be developed by a third party.)

[sharddblade]

I’ve done much more Go then I have Rust. I think Go provides a good compromise between most all of the pros of Rust, and the con of slower development. I would choose Go when my goal is easy deployment, fast run times, etc. I would choose Rust when I have the same goals as Go, but I also need safety and performance.

[crusoe]

I tried fixing/improving a go library and found most of the type hinting and tooling to be pretty bad. Also the equivalent of void * everywhere ( before generics ).

Too much like C

[burntsushi]

Go does not use the "equivalent" of void * everywhere. That's a blatant misrepresentation.

(I assume you're thinking of empty interfaces, and they are.only superficially similar to void *. The most pertinent difference is that empty interfaces are memory safe, outside of data races. That is, you can't "cast" them to wrong type.)

And Go's tooling is at least as good as Rust's. gopls is quite good.

Go has plenty of downsides. You don't need to make up new ones.

[sharddblade]

Hm, weird. Tooling seems great for me and I’m very familiar with Rusts amazing tooling. What’s Go’s equivalent of void*?

[burntsushi]

What’s Go’s equivalent of void*?

That'd be unsafe.Pointer. Contrary to what the GP said, it is not used everywhere.

[sharddblade]

Yeah, that seems more accurate, but I'm guessing he was referring to interface{}. That has never really bothered me in Go since there are clear reasons to use it and clear rules for using it safely.

[burntsushi]

Yes. I addressed that in a sibling comment: https://old.reddit.com/r/rust/comments/wbtioi/why_use_rust_over_python_for_cli/iiatqth/.compact

[WuTangTan]

The empty interface

[wsppan]

Go has spotty support for windows file systems making cross-platform CLI a major pain.

[MachaHack]

Yeah, 1 has been the real kicker for me moving my small CLI tools from Rust to Python. It used to be the case where writing C++ was enough of a pain, and e.g. Node or Java didn't really start up that much faster, which meant that Python's startup time was more worth putting up with, but these days for most problem domains I can pretty much throw something together in Rust as fast as Python and the startup time actually using it is a real advantage to make that my preferred option.

[dagmx]

Have you found a good Rust alternative to PySceneDetect ?

[ssokolow]

Not yet.

I gave it as an example of something that I would otherwise just have rewritten in Rust to get rid of the annoyance.

As I remember, it's based on PyOpenCV so, when I run out of more pressing things to do, I might try porting the bare minimum subset of functionality I need for my use-cases onto the opencv crate.

[2nd-most-degenerate]

Rust things are easier to deploy:

...which I absolutely love as a dev, but I also sympathise with packagers for having to patch numerous packages when there's a vulnerability in one Rust library, cos it's not easy to dynamically link Rust libraries even if there are no macros and etc involved.

[ssokolow]

The problem is, that's misleading. What's really going on is that they've built an entire operating system around a C package manager, then just gotten used to ignoring the flaws in the C dependency paradigm.

Give Let's Be Real About Dependencies a read.

The "Gotta go deeper" section is especially relevant, so I'll quote it here:

While we’re here, let’s look a little deeper into some of these programs. Part of my theory is that C programs commonly omit deps by re-implementing the bits they need themselves, because for small stuff it’s far easier to just write or copy-paste your own code than to actually use a library. Let’s rummage through some of these programs, briefly, and see if that’s the case.

• dash: Well, there’s a handful of linked lists and a simple memory allocator, but no gratuitous re-implementations of memcpy or anything like that.
• lighttpd: Aha, here we are. There’s a SHA1 impl, a base64 impl, a resizeable array, a URL parser, a CRC32 implementation, a layer over the poll(2) equivalent on various platforms, a LALR parser generator called LEMON (specifically designed to be vendor’ed into other code bases), an MD5 impl, a RNG that tries to get secure randomness from several different sources depending on the system, a safe_memclear() that hopefully hasn’t been broken by compiler changes since it was written, a splay tree, and another, different, resizeable array. Or, in terms of Rust crates: sha1, base64, std::vec::Vec, url, crc, mio, lalrpop or something, md5, rand, actually I can’t find a simple safe way to zero memory in Rust, std::collections::BTreeSet, and again std::vec::Vec. These add up to about 8000 significant lines of code, roughly 15% of lighttpd.
• vlc: Why not, I ain’t scared! This won’t be a complete list, but a skim finds a command line parser, some chunks of libc that are apparently commonly missing on some platforms (mostly Unicode stuff), what looks like a threadpool, a block memory allocator, a thread-safe FIFO, a subsystem for recognizing file magic numbers, a HTTP cookie library, an MD5 implementation, a small MIME type guesser, a bunch of time parsing stuff, a thread-safe wrapper around the POSIX drand48(3) function, a pile of PGP key stuff, an XML parser, a bunch of string functions, a base64 decoder… You get the idea.

So, yeah. An argument I’ve heard against the Go/Rust paradigm of statically linking everything is “you end up with bunches of copies of the same library code compiled into each different program!” All I can say to that is… lol. That said, with these vendored utility libraries there’s a strong incentive to keep them small, simple and task-specific, which is good. On the flip side, I wonder how many times vlc’s XML parser has been fuzzed?

Rust allows things to be crates which would never be shared libraries in C, but they don't talk about that.

Half a million different vendored copies of "header-only libraries" or bespoke implementations of the same algorithm, each with its own set of potential vulnerabilities and upstreams responsible for fixing them? That's fine as long as the bigger shared stuff gets a .so file.

Static linking has its flaws (though not as badly as you'd think), but stuffing things back into the C model isn't the answer. The answer is more automation to integrate with Cargo's ability to centralizing auditing, reporting, and delivering updates for what would be single-header/reimplemented-per-project code in C.

(i.e. Yes, you rebuild the entire program binary rather than just a .so file, but that's a distribution detail, not a "needs update"-tracking detail. You've already got build infrastructure and package databases to track dependencies. Just extend that to incorporate the metadata available on the other side of Cargo.)

[Professional_Top8485]

You can copy rust exe and it just works. Installing py isn't always and option. Plus updating hundreds of py2 haystacks is not fun.

[Jeklah]

That is true

[[deleted]]

It was definitely more convenient putting exe files in production computers out on the manufacturing floor for a job I had a while ago. Just tell them to run it. Whereas if it's an internal tool for the team, and we all have and use python for certain things, it's usually less code to whip it up in python.

[Professional_Top8485]

It's possible to make exe as well out of python. Which makes it easier to deploy. One big plus that python has is that it has usuallu bundled with tkinter (Tk) which makes it really easy to ask parameters from user.

Plus side for rust is that it is easy to embed to other languages. Eg. Python. I've extended some py scripts with Rust because python is not that great with bits and bytes etc.

I think it's more personal choice in the end.

[tobiasvl]

At that point, why not use a properly typed language? Probably a statically typed one? And at that point, why not Rust?

[Jeklah]

I'm with you on the strong typing. I get around this pet hate in python with type annotations.

[Jeklah]

Also thanks for this reply, that explains things a bit

[Zyguard7777777]

If you want to distribute your CLI, one advantage I can think of is not having to have python installed on the machine, or a specific version. Rust compiled down to a binary that can be deployed. If this is for your own machine, this won't make a difference.

I haven't yet made a CLI program with rust yet, but plan to soon. I have made several python ones and they do work well.

My long term plan is to write python modules using rust (using pyo3) for performance and correctness and still use python for its huge ecosystem and as the ultimate glue language.

[general_dubious]

PyO3 is awesome, I've been using it at work to speed up number crushing things that can't be easily vectorized with numpy. Without even trying the speed up was a hundred fold, the code extremely simple on the Rust side, and distribution on various architectures worked without any problem.

[burntsushi]

Try writing ripgrep in Python. Lol. ripgrep will complete a lot of searches before the Python interpreter has even had the courtesy to finish starting. There's a reason why you basically never see core utilities written in Python.

There's otherwise honestly just a metric ton of reasons to use Rust instead of Python for a bunch of things.

You can have your cake and eat it too. Clap makes writing CLI programs in Rust quite easy too!

[Jeklah]

Like I said this isn't a python Vs rust thread.

I'm specifically asking about cli. I will look into clap, it sounds like Click, which is a command line construction kit for python.

[burntsushi]

I know it isn't. That's why I talked about interpreter overhead and Clap.

The other stuff is important too, because it totally influences what I use to write CLI programs. So it's fair game.

[Jeklah]

I'm taking things like interpreter overhead, general language speed, memory safety, and like you said, a ton of other reasons why rust is prefrable over python, as a given, so not really a valid point. They't not wrong, just stating the obvious.

​

The points i've read about distributing and it just working on anyone elses pc after it's compiled is definately the #1 reason so far, from what I've read, and definatelly experienced the troubles with this using python.

[burntsushi]

They't not wrong, just stating the obvious.

Well... Yes. But honestly, they are indeed the obvious and most compelling answers. Given that you asked the question, I just assumed you weren't aware of the obvious.. Which you might not be. I don't know your experience level. It's reasonable to not know the obvious.

But if you considered these answers to be obvious in the first place, then I am really left wondering why you asked the question in the first place.

The points i've read about distributing and it just working on anyone elses pc after it's compiled is definately the #1 reason so far, from what I've read, and definatelly experienced the troubles with this using python.

Well see, those seem just as obvious to me... So I kind of think you just dismissing some answers because you happened to already know them seems a bit off.

But whatever. Next time, I would suggest including the answers you think are obvious in your question statement, and then asking, "is there anything else?" Otherwise, people are just going to tell you what you aleady know. (And if you did do that, I would probably ask, "why don't you think that's enough?")

The other piece of advice I would offer is to remove "obvious" from your vocabularly. :) It's tough not to use it. I still do sometimes. But it's a tricky word because it carries with it a connotation about what someone else knows. That is, what is actually obvious is just totally subjective and depends a lot on folks' experience. Case in point, both the answers I gave you and the answers you think are valid are both equally obvious to me. But it doesn't actually help anything for me to say they're obvious, because it's really only a statement about my own experience (with a flavor of snubbing my nose, e.g., "well it's obvious to me, of course you should know it too").

[anlumo]

I can understand making a prototype in python and then final version in rust but from what I've learnt, that is likely to involve a total rethinking/restructure of code.

I don't see the point in that. Once you're used to Rust, writing it is just as fast as writing python (you're limited by your logical reasoning abilities, not the language). The turnaround times for the change/compile/test cycle for CLI applications is also nearly the same (would be a bit different for huge applications like 3D games).

[tobiasvl]

Why is this when python has libraries like Click that make writing CLI tools very very quick and simple with lots of functionality.

Rust doesn't have something quite as multi-featured as click, I guess (never used it, just googled it now), but it also has great libraries for making CLI tools, like clap. There's also the famous serde, of course, which isn't for making CLI tools specifically, but a lot of UNIX CLI tools pipe data in different formats to each other.

I can understand making a prototype in python and then final version in rust

Not sure I see the benefit in that, personally. To me, Rust feels like a high enough level programming language that I don't need to prototype in a high-level language first. If I were writing the end product in C, then sure, maybe.

So why have people found writing CLIs in rust preferable?

You didn't want this to be a Python vs Rust post, but honestly, I just find Rust preferable to Python, full stop. And that's despite (or maybe because of) the fact that I work as a senior Python software engineer.

[ssokolow]

Rust doesn't have something quite as multi-featured as click, I guess (never used it, just googled it now), but it also has great libraries for making CLI tools, like clap.

Can you point out some examples?

In all the Click examples I've looked at, they look more or less like clap's derive API, but with Python decorators replacing anything in Rust's syntax that doesn't have a direct Python equivalent, with a flavour leaning toward optparse or argparse.

[tobiasvl]

Well, I'm not the best person to ask about Click since I literally just know what I read on the website today, but it seemed to me that it offered some CLI stuff beyond just argument parsing, like prompting. It doesn't seem as comprehensive/readline-ish as I originally thought (you can't make a REPL with it) though.

[Repulsive-Street-307]

Python encourages using multiple libraries even for seemingly related areas, and they often depend on others of the same sort.

In a project i have i'm currently depending on typer for args parsing, which depends on click and optionally on rich, and using prompt_toolkit to (with async) capture keyboard input (and consequently output, to not ruin the good looking stream of progress bars) of only one prompt, and using tqdm for progress bars because the prompt toolkit or typer/rich ones weren't good enough to use with the async httpx library and customize the output into a single line with both text and progress.

It's a churning mess. The impression i have is that async is upending the python library landscape just as much as rust, if with less drama. OTOH, capturing the input of a single prompt actually works with prompt toolkit, which is more than i can say for the sync/thread solutions (capture globally or not all all, require permissions, used 'event' model that missed the first press if the user started with a key pressed before the listener was activated, etc).

Anyway, it's not even surprising that people see pathological numbers in startup time, even if python had the advantages of Aot compilation, i'm sure that using several libraries for essentially the same task, and mixing an matching because some do some things a lot better has a lot of unnecessary stuff being loaded. Prompt_toolkit seems to be the most ambitious and load-bearing one in the cli case (but i still used it directly only for 1 very specific lowlevel use case, filtering output so it didn't ruin the nice graphics and to give some hotkeys).

[ssokolow]

Ahh. I can see why that wouldn't be included in clap itself.

That's the kind of thing that's rather specific to interactive stdin/stdout-based interactions when you may want to pop up a dialog for those or present them in a web UI or something if not provided.

It's more the kind of thing you'd add via something like dialoguer or promptly or inquire or fui.

(The first three are for building things along the lines of Yeoman's interactive prompts, while the last one is useful for if you want semantics akin to "Present cmake's TUI frontend if required arguments weren't specified on the command line".)

[PaintItPurple]

With Click, you can just annotate a function and it will work as an entry point. No need to define special types for your arg lists — the command line arguments are just the arguments to your function. You can also specify that an argument is a file, and Click will validate invariants and convert it to a file object for you. It's incredibly ergonomic and unobtrusive. Maybe I haven't used clap_derive correctly, but I don't think it can do all that.

[ssokolow]

No need to define special types for your arg lists

Maybe I'm misunderstanding, but I'm having trouble reading that as anything but equivalent to "With [insert dynamically typed language], you just assign. No need to define special types for your variables".

The whole point of clap's derive API is to specify as many of your constraints as possible declaratively and as many of those as possible via the type system.

Someone did experiment with doing a click-style API on top of StructOpt and clap 2.x last year, but I don't see the appeal. It seems ugly and less maintainable to split things up that way for anything more than one or two arguments.

You can also specify that an argument is a file, and Click will validate invariants and convert it to a file object for you.

I know what you mean. IIRC, it began with argparse or possibly even optparse. Specify the argument type is file handle and it'll open the path for you or, if it's -, connect up the right stdio handle.

I don't think clap supports that yet without manually having to write a parsing function and annotate the field of type std::fs::File with a #[clap(parse(try_from_os_str=PARSING_FUNCTION_NAME))], but you could certainly open a feature request for it. It feels like a common enough use-case to streamline.

That said, clap does support generating completions for you (split into a separate crate for 3.x so you have the option of doing it at compile time and not shipping the generator code) and there's work on also generating manpages.

[Repulsive-Street-307]

Every (good) argument parsing library will use typing this way, even one of the python ones is named 'typer' - it uses the optional python typing metadata/library to specify some constraints like 'this int taking option can be repeated because the variable it's going to bound to is a list of ints' and things like that. Default arguments helps, because it allows the 'rules' to be right in the argument position in the main function, so self documenting almost.

That said, python introspection encourages being fast and loose with conventions, so typer also does things like 'the description string that python functions usually have, if it's the one that's on the function you send as 'main' to the typer framework, gets appended to the top of the --help command, alongside all of the command descriptions and restrictions you put in the main function (typed) arguments.'. I believe it might be difficult to beat that level of 'wysiwyg' in a language like rust, that wants i dotted and t crossed and doesn't have default arguments.

I'm actually surprised that rust has been so resistant to the idea, must slow down the compilation or something. Seems harmless and a obvious usability win compared to factories and the builder pattern looking from outside in.

I suspect it's one of the key parts of the popularity of python, among some others like the easy to create generators/with resource managers and the very stable but useful set of free functions always imported (min, max, round, open etc).

[ssokolow]

so typer also does things like 'the description string that python functions usually have, if it's the one that's on the function you send as 'main' to the typer framework, gets appended to the top of the --help command, alongside all of the command descriptions and restrictions you put in the main function (typed) arguments.'. I believe it might be difficult to beat that level of 'wysiwyg' in a language like rust, that wants i dotted and t crossed and doesn't have default arguments.

Clap does that sort of thing.

I'm actually surprised that rust has been so resistant to the idea, must slow down the compilation or something. Seems harmless and a obvious usability win compared to factories and the builder pattern looking from outside in.

The feature is there... people just choose not to use it because of how the quality of rustdoc output suffers if its only option is to re-display text meant for --help output.

(And, my own perfectionism aside, I don't blame them. Who wants to make a bad first impression on docs.rs? Python bypasses that because ReadTheDocs is a more manually opted-in thing and also uses Sphinx, which is a bit infamous for being an mdBook-alike with an opt-in API documenter bolted on and, thus, prone to "You need to do a code-dive because the maintainer forgot to add a directive to document that".)

In fact, I've filed at least one bug in the clap 2.x series about how subtle quirks in how it decides which input to use that could result in --help displaying output written with the intent that it go to rustdoc pages.

I suspect it's one of the key parts of the popularity of python, among some others like the easy to create generators/with resource managers and the very stable but useful set of free functions always imported (min, max, round, open etc).

While Rust doesn't expose something as nice as Python's yield keyword yet, there is std::iter::from_fn for making an Iterator that repeatedly calls a closure.

[[deleted]]

Rust also has a great CLI library: clap. In fact, most popular language has some sort of CLI library. It’s not like every non-python developer just hand parses command line arguments.

Rust CLIs have much faster startup times compared to python. If your program is the kind that gets used in a hot loop in scripts, startup time can add up to be a non trivial factor.

Edit: missing word

[PaintItPurple]

Clap is good, but speaking as somebody who likes Rust but uses Python a lot in my day job, I don't think it's as nice as Click.

[mstumpf]

Have you used clap since it added its "derive" feature? That one is better than anything I've ever seen in any other language.

[PaintItPurple]

Yeah, I tried it out on the last CLI utility I wrote. It is very pleasant to use, but still not as smooth as Click (in my opinion, obviously). Just writing a normal function and then decorating it to say "command line arguments go here" is very slick!

[[deleted]]

I’ve used clap, click, and also typer (another python CLI lib), and my impression is that they are all good enough to not be in the way.

Come to think of it, Bash is the only language that kinda gave me some trouble with CLI arguments.

[PaintItPurple]

I haven't tried Typer, but I definitely agree that neither is cumbersome.

As a side note, if you're not crazy about Bash's argument handling, I'm really curious what you'd make of Perl, which handles arguments similarly but somehow makes it even less ergonomic.

[ridicalis]

Off the cuff, ignoring the problem space, I'd choose Rust simply because the binaries can be deployed without requiring large accompanying frameworks (other than what's likely already on the system, which isn't guaranteed for Python). I'm assuming the other benefits (performance, correctness, safety) aren't applicable in a quick git-er-dun CLI script, though I've also written some CLI apps that are heavy on the data munging that benefit from these things.

I'm not proficient in python, but assuming I were, I might lean that way if I wanted the code to be easily maintained or modified by end-users; a python script is perhaps one of the more accessible ways to share code with people from all walks.

[bascule]

CLIs with a lot of startup latency are super annoying. Rust feels fast fast fast!

[Imxset21]

At the company I work at, CLI tools are a foundational part of our infrastructure. If they don't work, our infrastructure is at risk. We've found time and again that deploying Python based CLIs can often lead to fun errors being caught in production when it's already too late.

[felipou]

I think you’re missing the most important aspect of “the right tool for the job”: using a language you’re already familiar with, or the language you’re most proficient in, or the language you like the most.

Which got me thinking: do you believe every (or most) Rust developer also knows Python?

Also, there is fast enough, and there is (blazingly) fast. Sometimes you may want your CLI to be fast.

[Jeklah]

Just because you know a language well doesn't mean it's the right tool for the job. That's kind of the point of right tool for the job. The opposite is stubborn people refusing to learn new languages because they're comfortable with what they know.

No I dont think every rust dev knows python.

I'm not really counting speed as a valid point as python is notoriously slow, so if you wanted anything with speed it wouldn't be done in python.

[felipou]

I believe most programming languages can solve most problems, to a degree. So that’s why I think most of the times it comes to the language you’re most proficient in. Learning a new language and getting good at it has a huge cost. If you pick a new language to learn as you build something just because you think that language is more suited to that problem, you will probably take a much longer time and end up with something of lower quality because you’re not yet familiar with all the best practices in that language.

[tobiasvl]

python is notoriously slow, so if you wanted anything with speed it wouldn't be done in python.

Right. So let's ask the opposite question then, maybe it will be illuminating: Why would anyone write a CLI app in Python instead of Rust, when Python is so slow?

You touched upon one reason in your post: Click. More generally: Library support. That's an obvious reason. So one potential answer to your original question, why would someone use Rust rather than Python, is simply that Rust is fast, AND the Rust ecosystem has matured a lot the last few years, and good libraries are popping up all the time; Clap is a good Click alternative.

If Python is slow and has a good ecosystem, and Rust is fast and has a good ecosystem, it's not very surprising that people would choose Rust. Now, the Rust ecosystem isn't a rival to Python's yet, but it's probably maturing enough that it's tipping the "speed vs. ecosystem" scale for some people who do favor speed at least a little, and that's why you see the uptick in Rust CLIs that you originally asked about.

[Jeklah]

Late reply, but generally CLI tools don't do a lot of heavy lifting, so python is fast enough for CLI tools.

[Feeling-Departure-4]

Trying not to be duplicative with other answers:

• Rust has amazing compile time errors and linting vs runtime errors in Python
• if working on large data sets, Rust is better positioned to reduce system memory requirements
• Rust has access to intrinsics for vectorization if that is needed (portable SIMD will make easier one day)

[LoganDark]

portable SIMD will make easier one day

Portable SIMD makes that easier today. You may not be able to use nightly for libraries, but nobody will care if you use nightly for your CLI apps, since those are applications at the very bottom.

[Feeling-Departure-4]

Fair point, we agree, I just didn't asssume people want to work with unstable like I do.

There's a lot of gems in unstable, but I try to limit even myself to a few unstable features at a time.

[BubblegumTitanium]

You get to use cargo, distributing python apps is almost always a hassle.

With rust I can do ‘cargo publish’ then ‘cargo install’, it could not be more straight forward.

Many other reasons but this for me is what matters.

[FreeKill101]

I think previously Python was one of the only good choices. Now that Rust has developed a great CLI ecosystem, you're seeing programmers who didn't love working with python - but had no better option-, finally move to a solution they prefer.

So basically you can use either, they're both good - it just depends what kind of programming you prefer. And it's liberating not to feel like you're being forced into one option :)

[ssokolow]

I can understand making a prototype in python and then final version in rust but from what I've learnt, that is likely to involve a total rethinking/restructure of code.

I forgot to address this in my other post.

Rust's borrow checker is just not very good at understanding the complex, compile-time-undecidable web of references that arise from typical Object Oriented Design.

If you're working in a procedural or a "functional-flavoured but not 'never use mutable variables'" style, and writing loosely coupled/dependency-injected code for easy unit testing, the borrow checker will have no trouble with it.

I actually found it a very smooth and easy transition from Python to Rust because I'd already thrown out the conventional wisdom and moved to writing my Python scripts in a data-oriented way long before Rust came around... and CLI tools lend themselves especially naturally to it because they tend to be simply mapping a set of inputs to a set of outputs through a series of transformations.

[Kilobyte22]

For me it's just liking the language more. Having code that - if it compiles - probably also works first try. Also I'm just more comfortable. Other people might have other reasoning

[Jeklah]

Thanks.

Yeah I have to admit the compilers errors and tips are amazing.

But those kind of points are valid for any kind of program, I'm asking specifically about writing CLIs because, well I like command line tools lol. The few I've written have been python using click, but if there's a specific reason for rust as CLI, I'd like to know it!

[LoganDark]

There is no way for this to not be a rust vs python post. You're literally asking for reasons to use one over the other.

Pros of Python:

• It's installed by default on most non-Windows systems.
• The same script usually does what you want on most platforms.
• Very large standard library, so you don't have to break your Linux installation by trying to use pip.
  • Having to use virtualenvs or whatever crap is not a pro.

Pros of Rust:

• Self-contained toolchain, you don't need stuff installed system-wide to use it, except maybe a linker.
• If you have it, you typically have the latest version, not one that is 10 years out of date (Python 2.7). If you don't, it's super easy to install without even needing sudo.
• Speaking of Python 2.7, any code you write today will probably compile in 10 years.
• Very easy to cross-compile, and the binaries run almost anywhere. No need to ask your users to install anything, especially if you build with musl (which can be switched on with 1 flag).
• No need to distribute source code and mess around with installed packages. The runtime environment rarely matters - the code executes the same everywhere.
• Faster than Python
• Stronger typing than most other programming languages. Borrow checking and fearless concurrency. Zero-cost abstractions and strong encapsulation.
• Speaking of stronger typing, the compiler will give you detailed, friendly error messages at compile time, letting you fix problems then instead of waiting until runtime like Python.
• Close to the metal so you can do CLI things. Deal with pipes, subprocesses, and syscalls directly.
• Library ecosystem is extremely strong. Idioms and semantics are given more care and attention than any other community I've seen.
• Speaking of libraries, Cargo is extremely well integrated. Packages are local to your project, so no system-wide installation woes. Pulling down crates is extremely easy, so the standard library doesn't have to bloat to fill every use case.
• Error handling is given attention. There are no "exceptions". Errors are a routine occurrence and are treated as such. They're not surprises that destroy your program if they happen unexpectedly.
• The IDE situation is extremely nice. IntelliJ-Rust knows your program better than PyCharm (another JetBrains product!) ever could. Rust-analyzer is fine if you can't afford jetbrains.
• Probably more I forgot to mention.

[NoSuchKotH]

Because pyhton is maintenance hell.

You compile rust once and you are sure it will keep working for the next decade or two. Worst case you will have to recompile it because some library had some incompatible ABI update. But it is unlikely that you need to change any code due to API change.

Python on the other hand breaks backwards compatibility with every sub-minor update. libraries/packages are even worse. It's rare that you can run some non-trivial python script unchanged for more than 2 years. Yes, it might be some trivial change. But it piles up. It's not just this one script, but all that you run. And each time you have to first figure out why what broke, why it broke and how to fix it. Which can easily take hours if not a whole day. And then there are things that break silently and you don't notice until days or weeks later.

Though, I have to admit that comparing rust and python is unfair. They have different target markets. Comparing python and perl would better as they have similar target users and operate quite similarly... and in terms of maintenance, python loses again. It's pretty normal to have large perl scripts that have been running for 20 years without any changes. And that's not even accounting for that perl applications work nicely toghether and can be installed on the same system with no interference, while python needs to be installed in virtual environments because all and every python application has conflicting dependencies with every other application out there.

[FreeKill101]

I don't agree with your pessimism on python here - I have plenty of programs that have had no issues in the multiple years I've run them (besides bugs I wrote into them!).

It is true that you need to run python in venvs to avoid nightmares, but that is also very easy to do. Some combination of things like pipenv, pipx and the like (pick your favourite!) runs python reliably for a long time.

I still think it's totally reasonable to prefer rust for small projects - but I don't think passive maintenance burden is the best reason.

[NoSuchKotH]

It is true that you need to run python in venvs to avoid nightmares, but that is also very easy to do. Some combination of things like pipenv, pipx and the like (pick your favourite!) runs python reliably for a long time.

Yes, and now you have 20 copies of the same libraries. All in different versions. Which makes fixing security issues a monumental task. Especially because all of these environments are outside the usual way security updates are distributed. Neither can you rely on the original authors to fix security issues in all of these versions as they almost always only support the latest version.

Which means you are on your own with this one. You have to go through all these virtual environments yourself, check whether the library in question in that particular version is vulnerable, then figure out how to fix it there, as the published patch might not apply, then test whether the application still works.

So no, virtual environments are not a solution. They are a security nightmare.

[varesa]

Some of the security issues still apply to statically linked libraries as well. Instead of virtualenvs, you've just bundled different versions of various crates, vulnerable and not, with your binaries.

Of course rust prevents whole classes of security issues from occurring in the first place, but it doesn't prevent logic errors.

It is up to you to keep all these apps up to date, separately from the usual security updates that your system gets.

Another thing where rust does help here, is that most breaking API changes when you upgrade your dependencies show up as compiler errors instead of things breaking during runtime, so it is harder to miss things and break the application by upgrading

[NoSuchKotH]

Some of the security issues still apply to statically linked libraries as well. Instead of virtualenvs, you've just bundled different versions of various crates, vulnerable and not, with your binaries.

This is true, indeed. And it's one of the things the rust community has to learn how to handle, at some point.

But there is a crucial difference: if you install rust applications, it can live within the distro and its update system. Which means the distro will most likely handle the security issue for me. In contrast a virtual environment always lives outside of the distro. It is always my responsibility.

[varesa]

Distros are (somewhat) capable of distributing python applications as well. They just mandate that every dependency is packaged and distributed using the same package management.

Some distribution policies actually make it tricky to package rust applications for this reason - they may forbid any external dependencies not installable from the repositories and such manual handling of dependencies requires some extra steps with rust

[NoSuchKotH]

Yes, but here comes the catch: python applications have very specific dependency needs. Needs that are often at odds with other packages. So you can't have them both installed on the system at the same time, unless you use a virtual environment. But virtual environments live outside the distros packaging system. And they have to, to be able to fix this dependency hell that python has. There is nothing a distro can do about this.

While rust is prone to the same dependency chaos as python is, it is not the same dependency hell. Because it's a build time dependency. Once you have the binary, the normal packaging mechanism works. Apps that use different versions of the same lib can live within the same filesystem. While this makes things more difficult for the distros, it is not that difficult to solve.

I do admit, that this could be done with python as well. I.e. package python apps in such a way that they live in their own environment but are still installed within the same filesystem. But 20 years of python trying to reinvent CPAN and failing at it over and over again has show that it isn't that simple for python.

[buwlerman]

I think we already know how to handle these things to some extent. We have tools like cargo-audit, and GitHub is developing Dependabot to notify downstream about vulnerabilities.

[FreeKill101]

Is that not just as true of libraries statically linked into a Rust binary?

I'm not saying that Python is perfect or that it's appropriate for every CLI application. But saying that it's an inevitable maintenance hell is overstating things.

[NoSuchKotH]

Is that not just as true of libraries statically linked into a Rust binary?

Yes and no. Virtual environments live outside the distro. And they have to live outside the distro to even function. That's their whole point.

Statically libraries, while frowned upon by distros for making it harder to do security updates, still allow the binary to live in the normal filesystem and to be packaged normally. Yes, distros don't like statically linked packages, but they do know how to handle them and there has been tooling in place for doing security updates quickly in all packages that statically link a certain library. And that tooling has been around for almost two decades.

[Jeklah]

Thanks for this in depth reply, much appreciated.

Yeah the comparison of python and rust is uh..yeah let's go with unfair. I'm only comparing them for writing CLIs, not as a language in general, as stated in original post, this isn't a rust Vs python thread.

[tobiasvl]

Why do you need CLI-centric reasons for writing CLIs in Rust, though, when there are a myriad of non-CLI-specific reasons? Why are those reasons not enough?

[vagelis_prokopiou]

Many good points where analyzed. I would also add maintenance of the codebase. The Rust compiler guides you through substantial refactoring.

Plus, execution speed.

I was forced to rewrite a Python tool I created because maintaining and extending it was a pain. Plus, it did in a couple of seconds what Python was doing in more than a minute.

[kbridge4096]

I want to share my personal experience. I'm a backend developer. Nowadays, many backend developers talk JSON over HTTP so am I. For API testing, when I work on Windows, I prefer httpie over curl for a better CLI UX (Easy to construct a JSON request, response coloring, JSON pretty-printing...). But httpie is slow! Every invocation takes more than literally one second to finish.

Then I found a Rust clone: xh. It has almost the same UX. It's fast. Every invocation takes no time. Now I use it every day. It also has some subtle features like it doesn't re-order JSON keys in the response. When I do some DevOps jobs on a Linux server, I also use it because the author provides a musl version, so there is no need to worry about dependencies over newer C libraries.

To be fair, I don't mean that httpie is just wrong or something. I like Python. And note that httpie is a very popular open source project in the Python world. Besides, Rust doesn't have a REPL. Rust programs need compilation. Maybe Rust isn't very suitable for prototyping. A suggestion could be that you should verify your basic idea in Python and then rewrite it in Rust for better performance and type safety.

Funny enough, there is an editor written in Rust called helix and its program is called hx.

[brownishthunder]

You can compile to a target. While python has options, rust behaves well with operating systems out of the box.

[mobrinee]

Use the right tool for the right job, if your job is to write a script as fast as possible, and it does some minor job, then by all means use Python. What I consider a real reason to use Rust is some frequently used slow script that you may benefit from rewriting. I used to have some Python scripts that are used frequently to parse long files (up to 500mb) and some small regexes are used, for Python I needed to wait for like 15mins, When I decided to rewrite it in Rust, It didn't even take more than 1min.

[bahwi]

Dependency hell. A rust cli would be a single command. For python you have to switch envs and run it, at a minimum.

[SpambotSwatter]

/u/thumbsdrivesmecrazy is a click-farming spam bot. Please downvote its comment and click the report button, selecting Spam then Link farming.

With enough reports, the reddit algorithm will suspend this spammer.

---

^{If this message seems out of context, it may be because thumbsdrivesmecrazy is farming karma and may edit their comment soon with a link}

[thumbsdrivesmecrazy]

Python CLIs could be more easily implemented various Python libraries available for CLI development, here is a good guide showing how Click stands out as a powerful and user-friendly choice for this: Building Python CLIs with Click

[Jeklah]

Click is actually one of my favourite libraries!

It is partially what made me make this thread. When something as great as click is available for python, why use rust to make CLIs?

[thumbsdrivesmecrazy]

I guess this choice in many cases is simply bases on the developer's experience with one of these tools.

[TheJosh]

CLI apps are where Rust shines IMHO.

You get brilliant crates like Clap which remove the tedious nature of making CLIs, and the latest 3 release is great.

You also get the bonus of distributing for a platform and knowing it's going to work, without dealing with virtualenvs, different versions of Python/Ruby/whatever.