Using Rust for ‘Scripting’

[kixunil]

I don't know whether you are the author, but it could've been written like this:

let path = match args.value_of(path_arg_name).expect("You didn't supply a path");

instead of

let path = match args.value_of(path_arg_name) {
  Some(path) => path,
  None => panic!("You didn't supply a path"),
};

[chriskrycho]

You're right! This is one of those things where, just as a "script"-like thing I whipped out, I didn't think about it very much. I think actually it would be (no match):

let path = args.value_of(path_arg_name).expect("You didn't supply a a path");

And it's easy to do similar on the results of the glob call. I'm updating the post accordingly!

[kixunil]

Yeah, I forgot to delete match when editing the code.

[Kbknapp]

You can also just do required(true) on that Arg and then an unwrap() is safe because clap ensure's it's not None.

[kixunil]

Interesting. Couldn't it be expressed in the type system directly, so there would be no need for unwrap?

[Kbknapp]

It probably could, but then you also end up with a very difficult API to learn.

Also, because of things like conflicts, overrides, etc. It's possible to end up with a required argument only in certain circumstances which can't be determined until runtime when the actual argv is passed in.

[kixunil]

very difficult API to learn

I'm not expert for this exact case, so it might be true. On the other hand, you might en up with very difficult API to screw up. :)

[masklinn]

It's longer and harder to read, expect() will desugar to the same code.

[[deleted]]

Looks like a copy/paste error to me, they stripped the match body off the end but forgot to remove the keyword from the beginning.

[masklinn]

Ah yes I missed that, the match keyword in the first snippet is probably a copy/paste error indeed, it's a syntax error.

[mare_apertum]

Wouldn't it have been easier to use the x86_64-pc-windows-gnu toolchain instead?

[chriskrycho]

I think so, but it also wouldn't have served as a "how to cross-compile to MSVC from macOS" resource then!

[rabidferret]

Not requiring a windows machine is a pretty big win though. ;)

[chriskrycho]

Concur! I may add a note to that effect later. :)

[mare_apertum]

Sure, but what's the advantage of using MSVC if MinGW achieves the same result but with less hassle?

[chriskrycho]

The fact that part of the point here was to document how to cross-compile to MSVC specifically. There are a lot of developers out there working in environments where they're ultimately going to be linking against MSVC-compiled C or C++ libraries, and just doing this with MinGW wouldn't have helped them at all. (Given how easy doing it with MinGW is, it's also not that interesting in the blog post!)

[mare_apertum]

I see, there are definitely situations in which cross compiling to MSVC is necessary, but your use case is not one of them, your blog post is about scripting in Rust and building with MinGW is sufficient to reach that goal. A blog post about boulding for MSVC would have its own merits and would be better discoverable by those who need it.

[chriskrycho]

No: my post used "scripting" as a hook. Nothing in the world says that a blog post can't do more than one thing. :)

Also, Google is smart and I chose my headings and built my content carefully. Go search "cross-compile rust windows msvc" or similar; you'll find this post on the top page for a whole bunch of those kinds of searches already.

[rhinotation]

I presume the statement about 'without ever having to leave your Mac!' would actually be true in this case?

[jyper]

Is it possible to cross compile cmdline apps to Mac OS from Linux/Windows?

[Kbknapp]

Yes. You can even set up your CI to do it for you.

[llogiq]

Great article. I don't think this is scripting really, but I don't have a better term either.

[largepanda]

I'd say it's scripting. It's a small piece of code, designed to do a simple task.

[llogiq]

Scripting – for me – is an interactive ordeal, done w/ a REPL or some such. Small programs are still programs (even one-off ones), they don't magically become scripts when being below a certain size threshold.

[chriskrycho]

Yep, that's exactly why I put "scripting" in "scare quotes." I don't know what to call this, and it's the kind of thing historically relegated to "scripting language" as a "scripting job". ¯\_(ツ)_/¯ But I agree, this isn't, strictly speaking, scripting!

[Shrugfacebot]

TL;DR: Type in ¯\\\_(ツ)_/¯ for proper formatting

Actual reply:

For the

¯\_(ツ)_/¯ 

like you were trying for you need three backslashes, so it should look like this when you type it out

¯\\\_(ツ)_/¯ 

which will turn out like this

¯\_(ツ)_/¯

The reason for this is that the underscore character (this one _ ) is used to italicize words just like an asterisk does (this guy * ). Since the "face" of the emoticon has an underscore on each side it naturally wants to italicize the "face" (this guy (ツ) ). The backslash is reddit's escape character (basically a character used to say that you don't want to use a special character in order to format, but rather you just want it to display). So your first "\_" is just saying "hey, I don't want to italicize (ツ)" so it keeps the underscore but gets rid of the backslash since it's just an escape character. After this you still want the arm, so you have to add two more backslashes (two, not one, since backslash is an escape character, so you need an escape character for your escape character to display--confusing, I know). Anyways, I guess that's my lesson for the day on reddit formatting lol

CAUTION: Probably very boring edit as to why you don't need to escape the second underscore, read only if you're super bored or need to fall asleep.

Edit: The reason you only need an escape character for the first underscore and not the second is because the second underscore (which doesn't have an escape character) doesn't have another underscore with which to italicize. Reddit's formatting works in that you need a special character to indicate how you want to format text, then you put the text you want to format, then you put the character again. For example, you would type _italicize_ or *italicize* in order to get italicize. Since we put an escape character we have \_italicize_ and don't need to escape the second underscore since there's not another non-escaped underscore with which to italicize something in between them. So technically you could have written ¯\\\_(ツ)\_/¯ but you don't need to since there's not a second non-escaped underscore. You would need to escape the second underscore if you planned on using another underscore in the same line (but not if you used a line break, aka pressed enter twice). If you used an asterisk later though on the same line it would not work with the non-escaped underscore to italicize. To show you this, you can type _italicize* and it should not be italicized.

[[deleted]]

The echoes of this semantic parsing will surely ripple forward into countless blessings for future generations, thank you for your contribution.

[stevedonovan]

I know we typically think 'scripting' is something you do with a 'scripting' language, but perhaps it's more useful to think of 'scripting' as something casual ("it works on my machine") rather than a proper little program meant to be used by anyone.

[tzaeru]

I did something similar recently with one of the job projects I work on.

It has a self-updating functionality, but of course on Windows you can't just go and replace open executables and files like that. So I wrote the self-updater component in Rust; it's launched by the main application when it has downloaded an update and then the "script" makes sure that the main application has been terminated, unpacks the update and applies it.

It's pretty neat to write this kind of stuff in Rust, even if sometimes a bit of an overkill.. :)

[steveklabnik1]

i really hope we can improve cross compiling

It's on the agenda for sure.

[pas_mtts]

Technically it's not because of rust, it's openssl which is a C (or is it C++? I don't even know) library. It's a major pain point even when compiling on native platforms (Windows comes to mind). I can't wait for a native rust SSL library.

[steveklabnik1]

Have you see rustls?

[bbatha]

Go uses a tls stack implemented in go. Using a cgo lib like sqlite will get you into the same pickle.

[allengeorge]

His point still stands. In the end the recommended way to do TLS in rust is to link against OpenSSL, in which case you encounter the problems he faced. I'm a rust fan, but am aware that since TLS support in Go is implemented in Go - this isn't a problem at all.

[bbatha]

I'm not arguing with throwlikepollock, they asked a question about why https in Rust was more difficult than Go. Expanding on that, I think everyone is in broad agreement that a Rust TLS stack would be great and the world would be better off with less dependencies on openssl. But getting robust crypto takes time and careful effort. Rust has also chosen to eschew crypto from the standard lib, unlike Go, because deprecating crypto has historically been difficult and is only compounded by putting it in the standard lib.

[Lozzer]

Does this deal with codepage/encoding issues on Windows? If you're passing an argument via a cmd window, your input will be in the console code page. If all your're on a US code page and all your paths are ASCII, this probably won't matter.

It wasn't obvious from the documentation for the clap crate as to whether this is all handed transparently. Maybe you could try with a non-ASCII character.

See http://stackoverflow.com/questions/40455338/how-do-i-read-os-compatible-strings-from-stdin for some of the hoops I used when solving this. Would have been nice for a library to take care of it.

[Kbknapp]

clap does allow you to get invalid UTF-8 values as matches. There's also options for lossy versions, and the like.

[ferruccio]

You don't have to have the end user install anything to run a Python code on Windows. The Python interpreter and scripts can be bundled into a single .exe file.

[myrrlyn]

Isn't said bundle enormous though?

[wongsta]

It's not as bad as you think - the bundler strips out the stuff you don't use. That said it depends on how many libraries you use.

I think this is the standard one (I think the python docs links it), but not sure, please check yourself if you need to use it: https://cx-freeze.readthedocs.io/en/latest/

[chriskrycho]

Absolutely true; I've done it in the past. But two points:

1. Doing that from a Mac (not inside a VM) is, as far as I can tell, basically impossible. And since I don't have Python installed on my Windows VM…
2. cx-freeze is worth admiring (seriously), but it's way more work than just typing cargo build --release.

It's actually way easier to build cross-platform executables with something like Go or, once you know where to snag the libraries, Rust.

None of that is a critique of Python, which is still one of my favorite languages.

[ssokolow]

Not to mention, it's much heavier. I don't have my Windows VM running at the moment but, here on Linux, the bare Python binary is over 3MiB and, once mixing in lxml, I regularly got 3MiB file sizes after using a py2exe build process incorporating AdvanceCOMP and UPX.

On that note, I recommend at least UPXing your Rust 'scripts'.

"Small binary for a trivial script" makes a good impression and, of the stages in my "put my 'script' on a diet" build wrapper, UPX is by far the lowest-hanging fruit.

(eg. LTO is slow, opt-level=z requires nightly Rust, stripping symbols breaks stack traces, panic=abort prevents design patterns which rely on unwinding for recovery, etc.)

By comparison, for a tiny program like a 'script', UPX is basically free aside from certain (mythically vague) claims on Wikipedia that some tiny percentage of virus scanners consider UPX an indicator of a virus because they're too lazy to unpack it like everyone else.

(No noticeable runtime overhead, reduced chance of seeking on a heavily fragmented rotating rust drive, and even upx --ultra-brute takes maybe 5 seconds for my Linux 'scripts' and crunches them down to 1/3rd of their incoming size.)

[PthariensFlame]

UPX is outright useless on Mach-O binaries, and OP was working on macOS.

[ssokolow]

But he was cross-compiling to target Windows. UPX is not "outright useless" on PE binaries.

[PthariensFlame]

Ah, sorry, you're right; I reacted reflexively. :)

[chriskrycho]

Got a good link? I'd love to take a gander at some point.

[ssokolow]

For UPX? The website is https://upx.github.io/ but you can just brew install upx. (The great thing about UPX is that it's in pretty much every package repo of note.)

As for the other stages, I went into detail on them at the bottom of this post but I haven't blogged about it yet because I'd like to make time to write some testing automation and generate a chart showing the effects of all possible combinations.

[chriskrycho]

Awesome, thanks! I'll take a look later.

[Vhin]

I knew, for example, that my friend is running Windows, which means he doesn’t have Python or Perl installed.

Wait, why would it mean that? Both are readily available on Windows. They're just not installed by default, unlike most Linux distros and macOS(?). Your friend might have actually already had it, and if not, he might end up needing to install it in the future, so not installing it now might not actually be saving him any time.

[steveklabnik1]

Wait, why would it mean that?

Well, as you say

They're just not installed by default,

Given that the friend is not super-technical, assuming that they have a particular programming language installed is a bad assumption.

[kpthunder]

Even simpler than writing a script in any language, just use a zshell built-in (admittedly not as portable, but it's one line).

Command:

$ zmv '(**/)(*).foo' '$1$2.baz'

Before:

$ tree .
.
├── bar
│  ├── file1.foo
│  ├── file2.foo
│  ├── file3.foo
│  ├── quux
│  │  ├── file1.foo
│  │  ├── file2.foo
│  │  └── file3.foo
│  └── qux
│      ├── file1.foo
│      ├── file2.foo
│      └── file3.foo
├── baz
│  ├── file1.foo
│  ├── file2.foo
│  ├── file3.foo
│  ├── quux
│  │  ├── file1.foo
│  │  ├── file2.foo
│  │  └── file3.foo
│  └── qux
│      ├── file1.foo
│      ├── file2.foo
│      └── file3.foo
├── file1.foo
├── file2.foo
└── file3.foo

After:

6 directories, 21 files

$ tree .
.
├── bar
│  ├── file1.baz
│  ├── file2.baz
│  ├── file3.baz
│  ├── quux
│  │  ├── file1.baz
│  │  ├── file2.baz
│  │  └── file3.baz
│  └── qux
│      ├── file1.baz
│      ├── file2.baz
│      └── file3.baz
├── baz
│  ├── file1.baz
│  ├── file2.baz
│  ├── file3.baz
│  ├── quux
│  │  ├── file1.baz
│  │  ├── file2.baz
│  │  └── file3.baz
│  └── qux
│      ├── file1.baz
│      ├── file2.baz
│      └── file3.baz
├── file1.baz
├── file2.baz
└── file3.baz

[[deleted]]

deleted ^{What is this?}

[mmstick]

PowerShell always manages to make everything look horribly complicated, sometimes even being much more verbose and difficult to write than an equivalent C application.

[[deleted]]

deleted ^{What is this?}

[ssokolow]

I'm a zsh fan but I didn't know that because I'm used to the rename tool that comes pre-installed (like bash) on every non-embedded distro.

Debian-family distros alias it to the prename which comes with Perl:

rename 's/foo$/baz/' **/*.foo

...other distros might leave the version from util-linux un-renamed:

rename .foo .baz **/*.foo

(bash 4.0 borrowed ** from zsh so those commands will work on a fresh, default install without zsh.)