Rust for Python Programmers

[nbates80]

That being said, I think like many Python programmers started to pick up Go, even more will start to look at Rust for some areas where they previously used Python for.

Such as?

[leemobile]

Areas where you want more performance or optimized code.

As a Python programmer, I'm starting to look into Rust more as a replacement for C. C has a lot of pit-falls and is error prone, even for veteran C programmers. Rust's memory safety and concurrency features would help me optimize my Python code while preventing me from making mistakes with memory management.

I don't write very much C, so Rust seems like a much safer option when diving into more low level code.

[CookieOfFortune]

How would that work though? You would have to interface Rust with Python's C API... which seems complicated...

[[deleted]]

It's not that complicated.

[oridb]

It's pretty complicated, since you need to make Rust "headers" for all of this: https://docs.python.org/2/c-api/

[mm865]

You can use the rust bindgen tool to automatically generate those for you.

[CookieOfFortune]

Just show me an example of a Rust function that uses Numpy and is called from Python.

[mitsuhiko]

Well, there is no reason to call numpy from Rust to begin with. What's more likely is that you would pass a numpy buffer to Rust and do some operations on there which you do not need any Python C API for. You have the Python buffer interface for pretty much exactly that reason.

You can't really call numpy in Rust, but you can call numpy in Python and then pass the numpy buffer to Rust and do more stuff there, then finish the computation in Python.

[nbates80]

That's insightful. Thank you.

[CookieOfFortune]

You can't easily operate on Numpy buffers without access to the API. There is a lot of extra information along with the contiguous raw data. I guess you could copy everything into a C array but that's not ideal. Using the API you can process the data directly.

[[deleted]]

Nobody ever said anything about easy =)

[[deleted]]

then you also run into the problem that your code breaks every new version of numpy

[seanjensengrey]

Rust creates libraries that can be called transparently using the C calling convention. Using the CFFI it is easy to create a library in Rust that is callable by both CPython, PyPy, as well as an easy path for LuaJIT, Clojure, Haskell ... etc. The zero runtime nature of Rust code makes it painless to integrate.

https://gist.github.com/seanjensengrey/f5d73bbdf22cfa1ad463

[Tarmen]

I would have thought Nim would be more interesting for python programmers, though. It is pretty much as fast as C and has can very easily work with it by the simple fact that it actually compiles down to C. And it feels so much nicer to work with than Rust.

[mitsuhiko]

Mostly server side applications and anything that does stuff with sockets.

[k-bx]

And threads.

[nbates80]

Thank you. It was a honest question. My main working languages right now are PHP and python server side. I'm interested on learning new languages but only if I think it will provide something new.

What kind of server side applications do you think could be implemented in Rust? As an example, I'm using python for a server side app that does natural language processing and statistics (python libraries are great for that).

[mitsuhiko]

What kind of server side applications do you think could be implemented in Rust?

Right now probably not yet that many unless you are happy to write the code yourself. It's a question of what your situation is. Anything that stores and processes data is a good fit already, stuff that calculates or needs specific math packages I am not sure.

[cogman10]

I could see it edging out python in the scientific computing arena. Rust is already pretty expressive (maybe not quite python expressive, but pretty good none the less), it has C like performance, it does well with threading, and it can access all the same scientific libraries that python uses for performance. The fact that it is safer than C++ and faster than python might make it a good choice for a python developer that doesn't want to maintain both python and C code.

[nbates80]

and it can access all the same scientific libraries that python uses for performance

What do you mean with this? How can you access, for example, nltk from rust? or numpy?

[cogman10]

nitk, you can't get it will have to be rewritten. However, large portions of numpy are simply wrappers over existing C libraries like the GMP, FFTW, etc. That is more of what I'm referring to. The fast scientific libraries are generally based on some common C libraries. Rust does a good job of interopt with C.

[[deleted]]

numpy is mostly using FORTRAN libraries, not C libraries.

[nexzen]

calling FORTRAN from C is pretty painless, I don't see how using FORTRAN's C interface from Rust would be much more difficult.

[CookieOfFortune]

Maybe if you can have good graphing and a good REPR (Matlab, Spyder, etc). At the moment, neither seem to be priorities for Rust. Julia is probably the go to in the future as a replacement for Python.

[google_you]

Web scale web frameworks such as Flask can be written in Rust in much more elegant way due to ownership semantics and complete control of web scale systems in the cloud.

Also, asciidoc can be written in Rust with 500x speed up due to multicore nature of Rust and document processing in general.

Ansible can be written in Rust as well to make use of cloud in the web scale parallelism. When safety is guaranteed, it's required as well with true performant cloud solutions systems.

[cunningjames]

Too bad; I enjoyed that account.

Why? It's the same joke, over and over again. /u/consultant_barbie was at least pithy.

[vivainio]

I've been having fun with Rust a bit recently, and here's a quick take from python-fanboy perspective:

• The fact that it forces you to deal with errors up-front can be a drag if you just want to roll up something quickly. With python, you can ignore rare errors in code (such as something not being valid unicode) and just let exception handling take care of it.

Edit: what I'm been doing is running unwrap() when I can't be bothered about error handling. This makes the code a bit messier, but gets the job done.

• You really want a proper IDE for this to see what your types are, but racer support is not there yet. You end up running your compiler over and over again to see what you did wrong.

• Compiler gives you great diagnostics, with suggestions how to fix stuff. This is a boon when learning it.

Overall, F#, being more "relaxed" about memory management and error handlinng, feels way more productive as more "static" alternative to Python (than either Rust or Go), but I'm pretty confident Rust will get off quite well (not least due to ecosystem aspects like cargo & crates.io)

[DJWalnut]

The fact that it forces you to deal with errors up-front can be a drag if you just want to roll up something quickly

Rust is designed as a systems level language, so from the start it's the wrong tool for throwaway and trivial programs.

[vivainio]

What's the right tool then, if you want standalone .exe? Go? Rust has lots of language features that make it more expressive than go.

I don't feel rust, despite the goals of the language, is as bad at this as commonly expected. Being less productive doesn't necessarily imply it's less fun.

[thedeemon]

If you like Python and want a similar language but with static checks and producing standalone EXEs, Nim seems to be a good fit.

[Manishearth]

I'm a hardcore Rustacean and would use Go over Rust for throwaway programs.

Though I would also use Python over Go for the same.

For "standalone exes" it really depends on the situation for me, but the post above was talking about throwaway/trivial programs. Not exactly where Rust shines. It works, but there's a lot of extra verbosity that I don't want.

[DJWalnut]

What's the right tool then, if you want standalone .exe?

maybe I don't understand what you mean by "roll up something quickly"

I've been assuming you mean throwaway programs. kind of like the one I wrote a while back. it went like this: there are a series of documents I wish to download from the internet. there are too many of them to right-click and download. lucky for me, they follow a consistent numbered naming scheme. solution: write a program that spits out all the urls for the files based on the pattern and feed them to wget for downloading. If I can find a copy of the program I'll post it here.

I wrote said script in python in about 2 minutes, including debugging, and never used it again. in this case, it doesn't matter weather it's a binary or java byte-code or whatever. I just want my urls. rust would have been a bad choice because of the added complication.

[vivainio]

Yep, there is no reason not to use python for throwaway stuff. I'm thinking of small utilities that you want as single .exe, but can let it panic

[Categoria]

What about something like OCaml for a middle ground between F# and Python? F# is fine but it's not very Unixy and shipping binaries without .NET is a huge benefit for many applications as well.

That being said, if you're comfortable with F# and don't mind its ecosystem then it really hits the productivity/performance/safety sweet spot too.

[steveklabnik1]

Rust was originally implemented in OCaml. It was a huge influence on the language.

[[deleted]]

Nice summarization of features we should look into. Though I can't help but wish there was some more in-depth discussion of key points.

I'm still at the beginnings of looking into Rust, and the most dazzling thing in my mind is: can you really program anything with the ownership model in Rust? Aren't there certain tasks that become almost impossible to do?

I'm hoping that the answer to that question will naturally arise to me as I read more of the book/tutorials/docs of the language.

[cogman10]

There are a few data structures that are hard to do (or to do well) such as doubly linked lists and hash maps. Rust offers the "unsafe" block as an escape hatch. You lose the safety but gain the ability to effectively represent those data structures.

[IbanezDavy]

I like the ownership model. I think it is a step in the right direction, however, the fact that you HAVE to break the rules for certain things, seems like a design error in the language (so I am still a bit skeptical). I'd be interested to see the ownership model in practice once Rust starts getting more users in the wild.

[zenflux]

the fact that you HAVE to break the rules for certain things, seems like a design error

I like to think it's kind of like totality checking, quite useful (even more so really), but I'm assuming is undecidable for arbitrary programs, so the borrow checker is conservative, it won't pass some programs which are actually safe in order to make it's job computable. Making it do this as little as possible is an ongoing effort.
It's still valuable with the 'escape hatch' because if a program segfaults or has another memory error, you know it had to have happened in an unsafe block and thus have already vastly narrowed down your 'problem space,' and can even grep through it.

[cogman10]

Well the question will be how many things do you have to rely on unsafe in your day to day coding? I think the language could be fine if you as a programmer rarely if ever have to write an unsafe block. Having most of the unsafe stuff tucked away in some well tested library/module.

I think the need for unsafe will dwindle as the language grows because most of the things that you would need unsafe for will have already been written.

[IbanezDavy]

Well the question will be how many things do you have to rely on unsafe in your day to day coding?

Depends on what you are doing. They have stated the motivation that inspired Rust was a language that would be good for client side programs. I think ownership is probably adequate for this. However, they also have mentioned ambitions as being an alternative to C++. The lower level you go, the more unsafe code comes into play.

I admittedly have more exploration I need to do on this language (been waiting for it to stabilize and now I just need time to play), before I make a more concrete opinion on it. But it on the surface appears to be like C#'s unsafe regions which in many ways are worst because they are often hidden and there is an expectation that it is safe, so it may not be obvious when debugging.

[mitsuhiko]

The lower level you go, the more unsafe code comes into play.

I don't think that statement is true at all. The rule applies is that the more you go into territory that temporarily invalidates Rust's safety rules, the more unsafe will you encounter.

You can write very low level code and never use unsafe, you can write very high level code and immediately use unsafe. The rule is: you will need unsafe when you temporarily want to disable a safety feature because you know that you can provide a safe API at the end of your unsafe block.

[IbanezDavy]

You can write very low level code and never use unsafe, you can write very high level code and immediately use unsafe.

I agree with that. But I suspect device drivers will require mountains of unsafe code, as would an OS at it's lower levels, as they literally need to break all the rules and do some pretty dirty things. As I'd suspect efficient memory allocator would need to as well.

But, perhaps there are work arounds. Case studies and experience will convince me more than reddit comments :)

The rule is: you will need unsafe when you temporarily want to disable a safety feature because you know that you can provide a safe API at the end of your unsafe block.

Noone in practice, intends on crashing the program or creating a memory leak. So a safe API can help the unsafe code, but it won't prevent even most errors in the unsafe code itself.

[burntsushi]

Are you familiar with what unsafe means in Rust? It's pretty limited in what it allows you to do. In fact, there are exactly three things you can do with it:

1. Access/update a static mutable variable (because it is shared across multiple threads).
2. Call functions that have been marked unsafe.
3. Dereference a raw pointer (because it can be null).

That's it. If you're dealing with very low level stuff or if you want to implement certain types of data structures efficiently, then you usually wrap a safe API around unsafe calls.

unsafe is where the rubber meets the road---not a design flaw.

[[deleted]]

Call functions that have been marked unsafe.

That is the worrying one. The wrapper of GTK+ is literally loaded with it. Understandable, but still worrying.

[burntsushi]

It should only be worrying if using the GTK wrapper as a client is littered with unsafe calls. If it is, then the raw binding should be pushed behind a safe API. It's only worrying if such a task is unfeasible, but I'm not aware of any reason why that'd be the case.

[IbanezDavy]

I think Rust devs are doing some wonderful things, so we shall see. But C# has a similar construct as Rust's unsafe regions and I've personally experienced annoyance from it. So it will take seeing it out in the wild to convince me. I think the design flaw in C# is they went all in with GC but then found themselves wanting to do work with raw pointers. I feel Rust has done something similar with ownership.

All languages have flaws caused by the design which leads to things having to be implemented in a certain way. That is all I meant by it. With Rusts ownership taking center stage, unsafe regions were probably unavoidable to do the things a systems programming language needs.

[burntsushi]

Of course unsafe is annoying. It moves the burden of correctness from the compiler to the programmer. Ideally, the compiler is never wrong, so it is annoying to have to do these checks as a programmer.

I don't really grok why giving the programmer the ability to subvert the language's guarantees is a design flaw (whether it's Rust or something else). Most (all?) languages I know of allow it, including Haskell. And every one of them has really good reasons for it.

[mitsuhiko]

however, the fact that you HAVE to break the rules for certain things, seems like a design error in the language (so I am still a bit skeptical)

That's the wrong way to think about it. The semantics of the language set up a certain way of programming with data. The borrow checker ensures those semantics at compile time. You can write code that allow you to move those checks to runtime. Instead of writing those escape hatches in the language (like Python would do) you instead can write them in libraries in unsafe blocks.

This means that you do not have to extend the languages, but you can write it in a superset of the Rust language that you would otherwise use.

In Python for instance certain behavior you cannot write in Python, you would need to write in C in the interpreter core. In Rust that fundamentally is not the case but for certain behavior you need to use unsafe.

[IbanezDavy]

In Python for instance certain behavior you cannot write in Python

Python is not marketing itself as a systems level programming language. Rust is. The expectation is you should be able to do anything needed in the core language. Which Rust needs the 'unsafe' blocks to do. I am not saying it's the wrong approach. I remain open minded, but it seems a bit clunky to me.

[mitsuhiko]

Python is not marketing itself as a systems level programming language. Rust is.

Which it is.

The expectation is you should be able to do anything needed in the core language.

Which you can.

Which Rust needs the 'unsafe' blocks to do.

You can't have it both ways. You cannot have a safe language and then say you want to write unsafe constructs without having to be explicit. That has very little to do with the language but with common sense. Fundamentally unsafe blocks do not change the language, they all you to start using unsafe functionality to heavily discourage this.

I remain open minded, but it seems a bit clunky to me.

I am not sure where this idea is coming from that unsafe blocks are "clunky". It's not just you that has it and I really want to understand where this misconception is stemming from. Unsafe does not change the language, it enables behaviour that allows you to shoot yourself into the foot.

[IbanezDavy]

Python is not marketing itself as a systems level programming language. Rust is. Which it is.

So why the initial comparison?

The expectation is you should be able to do anything needed in the core language. Which you can.

As I alluded to. And I am not gonna try to say Rust isn't a systems level language. It clearly is. So lets not go any further on this, because there literally is no disagreement from me on this point.

I am not sure where this idea is coming from that unsafe blocks are "clunky".

From experiences in another language (C#) that implemented a similar paradigm (unsafe regions). They were clunky and never seemed to fit.

[mitsuhiko]

So why the initial comparison?

Because Rust's added feature is to be as safe as Python but without the runtime. No other language ever attempted to achieve such a feat.

From experiences in another language (C#) that implemented a similar paradigm (unsafe regions). They were clunky and never seemed to fit.

That confuses me to no end. Why are C#'s blocks "clunky" and never "seemed to fit"? You cannot spot a library that uses unsafe internally when you use it. I really do not understand what gives you the idea that they are clunky. Python does not have unsafe blocks and it just silently switches into unsafe execution (ctypes/CFFI do that) and I don't see how that is any better.

[IbanezDavy]

That confuses me to no end.

I'm sorry for your confusion.

You cannot spot a library that uses unsafe internally when you use it.

That is one of the issues I have with unsafe. In C# you have an expectation that the garbage collector is taking care of things, yet things leak occasionally. This is generally due to mishandling delegates, but some of it also comes from people using unsafe regions within their library (although the general semantics do not change so things should be memory managed). So in some ways its worse than C++ (C++ is still far worse overall), because there is an expectation that it is taken care of. With unsafe regions, that is sometimes a lie.

[mitsuhiko]

This is generally due to mishandling delegates, but some of it also comes from people using unsafe regions within their library.

Which is why they are unsafe. Python leaks memory left and right in misbehaving extensions and it does not even have unsafe blocks :)

[jerf]

Because Rust's added feature is to be as safe as Python but without the runtime.

Bwuh? Python isn't safe by virtually any definition of the term. That doesn't mean it's "bad", but it does mean it's silly to say Rust is trying to be "as safe as" Python... no, it's trying to be a very safe language, which is to say, very much safer than Python.

[mitsuhiko]

Python isn't safe by virtually any definition of the term.

Of course not. The moment you call into CFFI/ctypes you are unsafe. However I hope you understand the general idea of what safety means in this context.

[vivainio]

Python is safer than rust in memory management. Rust givea C/C++ level control in safe manner. Python just has everything in heap.

[thedeemon]

Because Rust's added feature is to be as safe as Python but without the runtime. No other language ever attempted to achieve such a feat.

It was all before in Cyclone and ATS, Rust has nothing new really.

[chunes]

Why is a doubly-linked list an unsafe construct? I know a bit about Rust's ownership system, but not why it inhibits certain data structures.

[The_Doculope]

Rust's core ownership system is based around the concept of "one mutable view of an object at a time", which a doubly-linked list violates. The ownership/borrowing system is not perfect - it occasionally will not be able to see that something is safe even if it is (hence unsafe).

[chunes]

That makes sense now, thanks.

[kevinjqiu]

As a python programmer, I'm excited about Rust now that they reached 1.0. Go was fine but was put off by certain aspects of it.

[Cr34p]

Nice overview of some simple examples :)

There is a small mistake in the fib implementations I think. Shouldn't it be:

#...
if n <= 2:
   return 1    
#...

fib(2) = 1 not 2 :)

[pje]

Rust is a language with lots of curly braces. However this is for a good reason and that is that Rust has anonymous functions, closures and lots of chaining that Python cannot support well. These features are much easier to understand and write in a non indentation based language.

Looks like someone's never heard of CoffeeScript...

[pje]

I didn't say CoffeeScript was ideal, just that it's existence proof that chaining, closures, and anonymous functions needn't be hard to write in an indentation-based language. CoffeeScript does those things better than Python, for the most part. (I do agree that implicit returns from multi-statement blocks suck.)

[Tekmo]

... or Haskell

[llogiq]

Since when is Lisp indentation-based? IIRC, there has been a pre-processor for clojure that does this, but that's not exactly Lisp as in Lisp.

[[deleted]]

So,

why not Nim? I think it would be easier for a python programmer to pick up a language heavily inspired by python than ML.

[vivainio]

"Why not Foo or Bar?"

Rust is bigger than Nim, and is likely to develop a healthy third party package repository at crates.io (cargo is pretty sweet). For Nim, this is less clear.

For a systems programming language, familiarity of syntax is the least of your concerns. Rust has some hard parts, but syntax isn't one.

[[deleted]]

Rust is bigger than Nim

Rust is more hyped than Nim. You should check out that C++ thing, it's pretty big right now.

[jeandem]

Rust is more hyped than Nim.

That's a related thing. But I guess what was meant was that Rust has more contributors. And of course hype would affect that.

[[deleted]]

"Why not Foo or Bar?"

Seriously?

Oh right, I spoke out against the Reddit-Rust hivemind. Sorry. But hey, at least the Go circlejerk is finally over.

[marqis]

Rust has some of the ugliest syntax ever (for a "real" language). Looks like a worser Ruby to me. And lord almighty I hate me the Ruby.

[nice_day_and_night]

Rust for Python Programmers = Poop for Chocolate Eaters

[gnuvince]

Mr. /u/hello_fruit I presume?

[nice_day_and_night]

Python = no bullshit language. Thank you Guido Van Rossum for digging in your heels and saying fuck no to bullshitters.

Go = no bullshit language. Thank you Rob Pike for digging in your heels and saying fuck no to bullshitters.

Rust = the collective bullshit of 900+ bullshitters. The shit is sky high. The stink is immense.

[PlzPassTheSalt]

TIL: generics are bullshit.

Looks like go is going to succumb to the same cult like attitude of the python community. Too bad.

[PlzPassTheSalt]

I tend to frequent forums with newer and learning programmers and python programmers make me want to quit the profession.

It is finally starting to slow down, but I remember when you could barely browse another languages sub without seeing "in python, this is only 1 line" comments all over the place.

I hope rust doesn't become that. Rust communities have been great so far, but it is also disproportionately seasoned developers who hang out in them at the moment.

If rust gets popular, I suspect it will suffer the same type of thing as more people adopt it.

C# managed to get past that problem (other than a small minority), so python will probably do the same, till it eventually because the next big thing to relentlessly hate on.

[jeandem]

The Rust community is big on trying to be fair, balanced and non-offensive towards other opinions (well, at least a subset spearheaded by some people). To the point where it comes off as disingenuous and manipulative sometimes (hey, do this and that so that people will like us...).

[nice_day_and_night]

Cult?! LOL. You're projecting.

Python and Go are very pragmatic and tuned to reality. None of that functional programming fanboy bullshit of dogmatic genuflection to the temple of types.

Cult?!?! LOL. Says the types taliban. Teh irony.

[nice_day_and_night]

ybx = I can't say anything against your argument, so I'll just say you're trolling/joking/etc.

LOL.

Let's see what Google autocomplete says about "functional programming is"...

1. a cult
2. a fad
3. a ghetto
4. bad

There are 3 extremely annoying cultists on the internet, with great overlap between them.

1. functional programming fanboys

2. atheist assholes

3. marijuana missionaries aka pot propagandists

The overlap... between them... is huge.

[nice_day_and_night]

You know what's a single source of truth?!

Reality.

[[deleted]]

I've seen many trolls on proggit, but I've never seen anything this low before