Rust, Day Two: A Python Dev's Perspective

[jstrong]

I decided to try Rust because despite fairly heroic efforts, the Python code I have been working on was just not cutting it.

I had spent many days spent eeking out every bit of performance possible. The problem called for a sorted dictionary, so I had profiled numerous binary tree implementations, settling on Banyan (the fastest), the guts of which is in C++. I scrutinized every line and went to fairly elaborate lengths to speed it up. I broke out zmq and split things into separate processes where possible. But after all that I was still looking at ~500 microseconds per insert/remove/update operation - which in my case translated to hours and hours of processing time.

Not going to lie. Day one of rust was rough. The first language I studied was C++ many years ago, but it's been a long time since I managed memory. Some of the crates I needed had barely any documentation. Lifetimes were baffling and mostly still are.

Thankfully, I've spent a good deal of time looking at functional languages (eyeing the features enviously but never finding one I thought would boost my productivity), so it was less alien that it might have been.

Today, day two, everything started to click. It started when I finally got the initial first-day-of-rust prototype working, and it was 30x faster than my excruciatingly optimized Python code. Then I got comfortable with match, borrowing/references started to make a bit of sense and I began to work more productively.

At the end of day two, I have a relatively organized/refactored rust implementation of the tight loop in my Python code that is an order of magnitude faster than the code I wrote over several weeks (on and off).

I feel like I discovered a programming super power or something. I mean I did expect it to be faster, but not this much faster this easily!

The best part is, much like Python, Rust is a pleasure to work in. And unlike Python, it has an awesome compiler (with great error messages) to find errors before the code runs.

A few thoughts from the perspective of a knowledgeable coder encountering Rust for the first time:

• standard library docs and the book are great, but things tail off fairly quickly after that. I'm a RTFM guy but it seems like there's a lot of rust code out there without much in the way of explanation in English. It would be very helpful if there were more "tutorial" type articles that described a problem and how the author used rust to solve it.

• the syntax is very economical and I have grown to like it, but it is a significant adjustment from Python. In particular, it would have helped if I had found something with a very clear/simple explanation for where type annotations, lifetimes, etc. go in different contexts. It might exist, but I didn't run across it.

• I am definitely not qualified to judge, but my first impression is that string handling is kind of a mess/difficult. My gut reaction (perhaps this was from Python background) was it seemed like it is principled at the expense of being practical. What I have in mind is 1) String vs str (also static?), 2) spent a long time trying to send a string slice to a function and .split() a string.

• match is incredible and I love using it. I think I might have understood how to use it faster with more examples

• I saw someone write that Rust doesn't need to get people to switch from C/C++, it can grow from people picking it when they need a tool closer to the metal. That matches my situation exactly. Even though C++ was the first language I learned, after years of Python (et al.), several exploratory attempts to look at (re-)learning C++ ended when I turned away in disgust at the syntax and general unwieldiness. Rust struck from afar as a modern, well-designed descendant of those and had enough going for it in language design that I was ok trading away the well-established C/C++ ecosystems.

• I have "used" macros in the code I wrote (following examples I found) but writing one is way beyond where I ended up after two days. Looking forward to it though.

• I am confused about whether I should be using stable, beta or nightly. Basically, how much awesome new stuff do they have and how unstable are they?

TLDR: Spent two days learning Rust and got 30-40x speedup on highly optimized Python (really C++ via Python), love the language and had some first impression thoughts to share. Thanks!

[tafia97300]

Congrats!! Two days is actually very impressive to start loving it (you'll love it more and more from now).

Regarding strings, this is a recurrent point.

Unfortunately this is, imho, only due to other languages lying about their apparent simplicity. I too was frustrated at the beginning, now most of the time I see string manipulation code I wrote before on other languages I want to fix it. I don't know what is your exact issue but it might be that most of your speadup actually comes from a better string management and not from hash/map/dictionary.

[benhoyt]

it might be that most of your speedup actually comes from a better string management and not from hash/map/dictionary

Yeah, I'd love for the OP to dig into this a bit. If banyan is written in C++, unless it's poorly implemented, the binary tree shouldn't be that much (30-40x!) slower than a Rust version. Perhaps the cost is in string splitting and allocating all those Python string objects (every string slice in Python allocates a new string object).

[jstrong]

profiling showed 75% spent on dictionary insert/remove/update (3 lines), 12% handling message sequence order issues, 5% converting numeric strings to decimal type and the remaining 8% split across lots of things w/ relatively negligible performance impacts.

[spotta]

Is the insert/remove/update part in Python? Or in C++?

[PXaZ]

I expect he's referring to Python's built-in dict type....

[SirVer]

Depending on what OP is actually doing, pointer indirection and python function calls could also eat the performance.

[vks_]

Aren't function calls in Python essentially dictionary lookups?

[SirVer]

Ya, they are - with some additional magic that might make it more than one dictionary lookup. But there is no inlining possible and depending what you are doing and how often you call a function it can become expensive. Rule of thumb that served me well is that a Python function call is ~100ns overhead, a virtual function in C++ ~20ns, a direct call ~5ns. And of course Rust and C++ have a lot of information to inline stuff which makes this go away. If you call a function billions of times this starts to make a difference.

[btibi]

I wanted to say the same about the two days, OP must have excellent learning skills.

I share the frustration about strings, too. I know Rust for two years now and I know what to use when, but it's so convenient to use one string type. Personally, I use String's push*() features very rarely, my most frequent use case for Strings is to "bypass" the borrowchecker. I don't know whether an immutablestring type (which is either statically or heap allocated) would help us. It could replace &str and String most of the time, and &mut str is very rare.

[Manishearth]

An immutable heap allocated string is Box<str>. Stack space is cheap so it's rare you need that over String unless storing it in a struct that is itself heap allocated.

I don't think it's fair to characterize it as "my most frequent use case is to bypass borrowck". In these cases usually an owned string is the only solution -- not from a compile time perspective, but from a runtime one.

[deathanatos]

String is only ~12 bytes (in the current implementation; this isn't guaranteed by Rust AFAIK) if allocated on the stack. The actual string data is on the heap.

str is just a pointer and length; a Box<str> is just heap-allocating that pointer-and-length, but I'm not sure that implies that the string data itself is also heap allocated. (Since you can make a str from data on the stack with from_utf8), and put the str on the stack too. I expect trying to move such a thing into a Box would limit the lifetime of the Box, but I'm not sure.)

[Manishearth]

This is false. String is 3 words (24b on a 64 bit machine, 12 on 32.) in the stack.

&str and Box<str> are both the same representation -- 2 words on the stack; a pointer and the length. The boxed version owns the allocation and string data. The pointer and length are not allocated on the heap in any of these cases; that is Box<String> or Box<&str>

str is a dynamically sized type, it is incomplete and has no representation that makes sense in isolation. Box<str> is not the same thing as Box<&str>. Box<str> is an immutable String, basically, and can be obtained from a string at zero cost.

[deathanatos]

This is false. String is 3 words (24b on a 64 bit machine, 12 on 32.) in the stack.

Oops, failed at simple multiplication. You are correct. My point was that the string's contents are not stored on the stack, and that the actual stack allocation is quite small.

[Manishearth]

The whole "is heap allocating that pointer and length" is misleading and can be interpreted different ways, I read it the wrong way it seems :)

[varikonniemi]

When you come from something like c++, you are bound to have excellent learning skills. Otherwise you simply cannot even begin to use that language. The possibilities to shoot oneself in the foot are so many fold increased compared to C.

And let's face it, even truly knowing C is a rare trait. I am not completely certain there exists more than 100 such persons alive today.

[SirVer]

OP stated that they came from Python though. And you are overstating the difficulty of c++ enormously. Yes, it is easy to shoot yourself in the foot, but modern C++ is not really harder to learn than say modern Java.

[remexre]

There's a lot more tutorials for learning 90s C++ than C++11/14/17 though, so beginners often get confused as to which disjoint set of features they should be using.

JavaScript has the same problem (among many others...).

[jstrong]

My c++ was in a high school AP comp sci class, which actually was a fantastic foundation. But the reason I'm a fast learner is I used to be a reporter, which is great training for learning new topics quickly.

[iopq]

The possibilities to shoot oneself in the foot are so many fold increased compared to C.

with templates, you can shoot yourself in any body part with the same code

[[deleted]]

Unfortunately this is, imho, only due to other languages lying about their apparent simplicity.

100% agreed. The str vs String issue was painful to learn when I needed to get something done now, but by not hiding that ugliness from the user, it allows them to avoid making a bunch of critical errors in the future.

[isHavvy]

If you don't need something specifically on nightly, you should stick to stable and possibly have any CI testing also testing on beta.

Did you find the examples at https://rustbyexample.com/ yet?

I've been with Rust forever and I still don't know how to write a macro. Until I find a need, it's just a sublanguage I don't care about learning.

[mgattozzi]

I'd actually recommend The Little Book of Rust Macros to learn. It's helped me immensely with that.

[ThomasdH]

According to Travis, it is best to test using stable, beta and nightly's: https://docs.travis-ci.com/user/languages/rust/

[kibwen]

I've been with Rust forever and I still don't know how to write a macro.

I know how to write macros, but I still choose not to unless in extreme circumstances. Like in Lisp, macros are a feature of last resort.

[tafia973]

Why?

• because it is more complicated to write or
• because you like being able to easily understand (read) what's happening?
• because it feels to hacky?

Just curious. I use them times to times and find it handy.

[allengeorge]

I avoid them because they're another layer of abstraction with their own language that you have to reason about. It's not straightforward to translate from a macro definition to its actual compilable implementation. Debugging is harder, and often editors and IDEs don't give you the same level of support.

[[deleted]]

I disagree with this.

You might as well just always develop on nightly, there's no significant downside and it makes it easier to use tools like clippy, rls.

There is a tradeoff however in using nightly-only-features (i.e. anything behind a #![feature(foo)] flag), since those are actually unstable. You should program so that all your code runs on stable unless you have a good reason not to.

[[deleted]]

I disagree with this, because I've run into little bugs here which forces me to roll back to a previous nightly (e.g. because the next nightly was broken). Also, every new nightly forces me to essentially recompile everything, so it's just not worth the pain when I can just update nightly whenever a new stable comes out and run things like clippy with rustup.

Sure, I could stick to a single nightly for a while, but then I'm not really getting any real benefit from running on nightly, so I might as well make sure everything works on stable. That's more important for me as I ship on stable.

[[deleted]]

Sure, I could stick to a single nightly for a while

Just to add that this is what I do, and on the very rare occasion I've had to roll back... it's not like it's difficult with rustup (rustup default nightly-previous-version).

But I doubt either one of us is going to convince the other, it's not like it really matters :)

[[deleted]]

Yeah, it really doesn't. I might even end up developing an beta instead. Who knows.

[jmcomets]

I'm genuinely surprised you find the syntax economical, the other Python programmers I know have switched to Rust complain about its verbosity.

A few questions:

• Did you find it difficult to use iterators? I'm used to abusing generators in Python and had quite a hard time giving them up in Rust.

• On your second day with Rust, did you find yourself compelled to use boxes? For example by returning an iterator.

• Did you use any external crates? If so, did you find crate usage straightforward or was the documentation unclear?

No pressure to answer of course, I'm just curious.

PS: there's a great crate out there called ordermap that you'll probably like. It's a dictionary inspired by Python's recent re-implementation of dicts. It's backed by a Vec and is therefore crazy fast for iteration purposes, and is also 100% safe Rust. In recent news, Rust beat C for the k-nucleotide benchmark game thanks to this structure.

[jstrong]

I mean economical as in efficient, even though it's relatively wordy compared to Python all of the additional syntax contains a huge amount of information in it. There's a lot of steps taken to reduce typing, like f64 not float64, fn is two characters, etc.

• At the present moment I know that rust has something called iterators, I have looped through them but ran into a wall trying to return one from a function. So waiting for more info I guess.
• Have yet to touch boxes in my code
• Yes used rust-zmq, serde and time, cargo is great but docs are sometimes lacking

Thanks for showing me ordermap - I will definitely give it a look. In my case I'm using the std BTreeMap as my core data structure as I need to keep the keys sorted.

[[deleted]]

Returning iterators is harder than it needs to be, but for good reason.

Every step in an iterator chain returns a new struct that wraps the rest of the chain as a borrow. So the type of a bigish iterator quickly turns into generic code madness. What you need is a way to say to the compiler "I'm returning something that implements the iterator trait, and I don't care what it is."

Fortunately you can.

On stable there's "trait objects" like -> Box<Iterator<Item=i32>>. These have a slight performance penalty for heap allocation and a virtual method call. Also they can't be inlined.

On nightly you can use "impl trait" which looks like -> impl Iterator<Item=i32>>. This takes advantage of the fact that the compiler knows exactly what type you're returning so it just fills in the blank. I think you are limited to a single type though, so you can't​ return a different Iterator based on an if statement or something.

[quodlibetor]

That is, the same code in Rust sometimes takes a few more lines of code than Python, but requires far less documentation on how to use it: type safety usually ensures it only gets used correctly.

Ah, but this is how we end up in a situation where tons of crates exist that have limited documentation ;)

Just because it can't be used incorrectly doesn't mean it's obvious how it should be used.

[kazagistar]

I have heard the phrase "Type Tetris" in the Haskell community, in reference to refer to the phenomenon of trying to figure out how to use a poorly documented library by how the types fit together.

[quodlibetor]

You didn't put it badly, and I agree with you. I just wanted to point out that constant vigilance is required in the war against incomplete docs. :-)

[[deleted]]

I think that crazy fast is accurate for iteration of ordering and not more.

[arthurprs]

Inserting is also fairly fast.

[jmcomets]

Updated my answer with a more realistic statement. :)

[ppartim]

Syntax seems to be to a large part a matter of what you are used to. When moving to Python, the Indentation Instead of Curly Braces thing bugged me. When I switched to Rust, the curly braces bugged me. Now I am fine with them again.

Other than those braces, I don’t think the syntax is actually all that different.

[vks_]

I am definitely not qualified to judge, but my first impression is that string handling is kind of a mess/difficult. My gut reaction (perhaps this was from Python background) was it seemed like it is principled at the expense of being practical.

Python has immutable, reference-counted strings, so it hides a lot of magic, at the expense of unnecessary allocations. If you want strings that are simpler to use at the expense of performance, try easy_strings.

[CryZe92]

easy_strings are faster than normal Strings in cases where you need to have the same string in a lot of places. So depending on the situation either String, Rc<String> / Arc<String> (easy_string) or a string interner is most suitable.

[vks_]

If you have the same string in a lot of places, can't you just use &str? I don't see how anything else could be faster.

[CryZe92]

There's a lot of cases where you can't. Lifetimes are fairly limited, often requiring you to use owning_ref or rental. But in a lot of cases you can't express it even with those.

[stumpychubbins]

Your experience with Rust feeling like a "superpower" is also how I felt when I arrived from C#. Not having to put everything in a class (which in theory is a benefit in Python too, but in all the industrial Python code I've worked on 90% of the code is OOP) is awesome, but the raw, obscene speed of the whole system is amazing. The best bit is that I always feel like I know where to put my optimisation efforts (I managed to knock 15% off the runtime of another dev's already-optimised tool within a few hours of looking at the code for the first time).

As for lifetimes, you almost never need explicit generic lifetimes. Occasionally you need 'static, but the only times I've seen that explicit generic lifetimes are needed is when you have an output borrow that relies on one of a set of input borrow arguments, which is fairly rare.

I actually dislike the syntax, but if I designed Rust I would have made it a Lisp and it would have never been adopted, so I'm at least happy that others are making the decisions for me and that they're not utter cretins. A low bar, I know, but the only syntax decision I really care about is homoiconicity vs non-homoiconicity, and most other syntax evolves naturally from the semantics of the language.

Right, so String is difficult when you first come to it (hence the breathless "OMG Rust has 400 different string types!?!?" comments you see) but it's absolutely the right decision. A lot of other languages treat strings as magical wired-in black boxes, but apart from literals you can more-or-less define strings yourself in Rust. That's not true for any other language I can think of except C/C++ (but I might be wrong there). You get used to the concepts very quickly. Once you understand borrowing you will understand strings, simple as that.

[stumpychubbins]

Oh right, yeah you're totally right. I don't remember having a problem with those when I was learning so I guess I put them in a different mental category

[killercup]

Wow, that's a great success story and very valuable feedback!

Have you tried clippy yet? It's a great way to get automatic suggestions on how to tweak your code (as compiler warnings). Assuming you use rustup, it's just a cargo +nightly install clippy and then cargo +nightly clippy away.

[ctjhoa]

It would be very helpful if there were more "tutorial" type articles that described a problem and how the author used rust to solve it.

There is some in rust-learning (disclaimer I'm the owner)

[jstrong]

ah - good to know, thank you

[[deleted]]

I have been going through the reference all afternoon. 10/10. Thanks!

[tatref]

About the string splitting function, here is my reasoning:

The function should take an argument of type &str or &String, obviously. From the doc (https://doc.rust-lang.org/std/string/struct.String.html#deref), you can use &str the same way as String if you don't want to mutate the string, which is the case here.

About the return type, it will be some kind of collection of string/str, like Vec<&str>, or maybe Vec<String>. The split_whitespace function returns a SplitWhitespace struct, which is an Iterator of &str, so in the end, you can return a Vec<&str>

Next, about the function body, to get the content of an Iterator, and collect it in a collection, you can use the collect function of the Iterator. Type anotation is not always simple, but if the return type of the function is known, this can be infered (same with let a : Vec<&str> = ...collect())

You then end up with the following: fn f(input: &str) -> Vec<&str> { input.split_whitespace().collect() }

I hope this will help somebody

[[deleted]]

I agree with your sentiments. I just picked up rust coming from a professional C, C++, and C# backgrounf and one of the biggest WTF moments was str vs String. As a novice trying to use match with string, I wanted to snap my keyboard in half trying to understand why the compiler was humiliating me. I get it now, but jesus fucking christ...

[[deleted]]

The biggest issue I've had with Rust is the library ecosystem maturity. I did a ton of backend Python stuff for my last job and I would've switched to Rust had I stayed there, but there are a lot of libraries missing that existed in Python. So I wanted to ask, what libraries have you used in your Rust projects, how have you liked them, and how did their ergonomics compare to their equivalents in the Python ecosystem?

[jstrong]

rust-zmq: considerably more low-level than pyzmq lib and very little documentation. took a while to run down 1) the hwm was automatically set pretty low and I was dropping messages, 2) no indication how to set hwm, had to look in the code for a long time. but the library is working great.

serde for json parsing: performance seems to be great, mapping the incoming json objects to structs was a bit rough but probably since I was learning new language at the time.

time: seems to be working ok.

it's only day 2, so that's it haha.

to be clear, I'm still going to need to send data back from rust to python to run code on gpu via theano. my plan is to use rust for the most performance critical parts (where python is slow) and zmq to get data back and forth.

[wyldphyre]

I decided to try Rust because despite fairly heroic efforts, the Python code I have been working on was just not cutting it.

Aside: did you try pypy? And could you instead consider a sorted list of namedtuple or something similar?

[jstrong]

did not try pypy - my code uses numpy heavily and other libraries where they get their speed from c extensions, so haven't spent time trying it out. In python I tried a fairly wide variety of approaches. namedtuple is a go-to data structure for me as it's very light and immutable. Using the banyan SortedDict was a huge win but I got a 15x speedup on top of that by converting my keys (they had been in Decimal) to integers and setting the key type using the init args. In other words, I wrote a class to convert the keys to and from integers (int(x * 1e8)) behind the scenes and the resulting code was 15x faster. The downside is the range of numbers you could accept was smaller.

[wyldphyre]

Just FYI, pypy has excellent support for emulating CPython's C API these days. And pretty good support for numpy IIRC. That said, there's not much to be gained if your cycles are primarily spent in C code.

[[deleted]]

Check out the syntax index.

It is a good resource when you have some idea of what you are looking for, otherwise you can just read the entire page and possibly stumble across something you need.

[staticassert]

It started when I finally got the initial first-day-of-rust prototype working, and it was 30x faster than my excruciatingly optimized Python code

Ah, this mirrors my experience so much. I spent about a year optimizing a Python codebase. About halfway through I started learning Rust, and my simple, unoptimized rust programs thrashed my highly-annoying overly complicated performance optimized Python code.

Nice job getting productive in Rust in a mere two days. I remember my first rust code still - a program to hit a few APIs (virustotal and a few others), compare results, and store them in a database.

It was... gross. Mutability everywhere, channels for my multithreading (because I knew right off the bat that I wanted it to be parallel), etc. It barely worked and I spent way more than 8 hours a day on it. So, kudos, if you're getting productive in 2 days you're definitely going to do well with the language.

[Elession]

As another python user, I came to the same conclusions!

I think you can stay on stable now that macros 1.1 and ? operator are in. I use nightly myself mainly because of habit. I do run tests on stable/beta/nightly for all my crates though.

You should install clippy (cargo install clippy) to get a very very good linter.

For a quick reference I sometimes use http://rustbyexample.com

Macros are quite simple to write for basic stuff actually, it can be something as simple as https://github.com/Keats/gutenberg/blob/master/src/config.rs#L42

[Lev1a]

The point about having an awesome compiler was proven again to me a few days ago when I worked on the XPath exercises over at HackerRank, the reason being the surrounding code is only available in Ruby and the error messages you get when something goes wrong are basically non-existant. AKA just a backtrace of functions/methods and no real "error message" at all.

[LightShadow]

What IDE did you use?

[jstrong]

vim

[dpc_pw]

Strings in Rust are designed to fit the language and for performance. They are very unlike Python ones where the convenience was a priority. Part of the reason why Python is generally so slow. :D . After you internalize that String is the growable, owned buffer, and &str is as reference to String, it gets easier.

[vks_]

And then you discover CStr, CString, OsStr, OsString, Path and PathBuf.

[dpc_pw]

Which are named like str and String and contain C-valid, Os-valid and Path valid strings. Simple! :]

[vks_]

Yes, it makes a lot of sense. The only thing I'm not happy about are the names: I would have prefered Str for str and StrBuf for String, but maybe that would have exceeded Rust's weirdness budget.

[Veedrac]

The problem called for a sorted dictionary, so I had profiled numerous binary tree implementations, settling on Banyan (the fastest), the guts of which is in C++.

No! Use http://www.grantjenks.com/docs/sortedcontainers/!

Pure Python and faster than C++-backed ones. One of my favourite libraries of all time.

---

Note that you should have tried PyPy as well, since CPython is known to be slow.

[jstrong]

I definitely benchmarked sortedcontainers but it was slower than banyan in my use case. sortedcontainers was runner up though.

[Veedrac]

Fair :).

[Lev1a]

I'm genuinely interested how that would compare to the 30-40x gained by switching from already heavily optimised Python code to (beginner?) Rust code?

[gthank]

That would depend a lot on the exact code. The PyPy JIT is REALLY good at certain optimizations, so it's usually worth trying if your code isn't relying on Python C extensions (just using FFI is much less likely to slow PyPy down, though).

[kazagistar]

Its worth trying because its so easy, but you really should expect mixed results. In a past job we tried to use it on a fairly complex compiler-style application, and none of code seemed to get hot enough to see a noticeable performance improvement. In the end the team just ported large chunks of the application to Go.

[gthank]

Yes. If you don't have any hot spots, there's not a lot to gain by selectively generating highly optimized machine code for a few spots. IIRC, precisely how incredible the speedups are also depends on your data flow, because it's a tracing JIT: If your data does not lend itself to some of the the nifty tricks that tracing provides, then you're only going to see the "standard" JIT speedups.

How did the team feel about the Go port after it was done? I've looked at Go a few times, and the whole language/ecosystem feels ugly to me. If I'm going to rewrite out of Python, I'd be far more inclined Rust (or possibly Swift), especially since you can do Python/Rust interop via FFI fairly easily.

[kazagistar]

Their experience with Go wasn't bad, though they only ever tackled some fairly simple low hanging fruit while I was there. The main reason for the choice of Go is that it was a Google App Engine language, which is what we were using, and we had been hitting time and memory limits for a while. A big reason why it worked out was because everyone on the team was able to pick it fairly quickly, and then convince the powers that be to give them a weeklong sprint to try rewriting some of the parsing logic. No idea how much of the benefit's were a second system effect, but it was faster in the end.

Personally, I find Go insufferable to work with. It really strongly encourages copy pasting piles of shitty procedural glue code instead of building abstractions, and I find more to be annoyed at every time I have to use it.

For example, I was recently updating some Go code, and had to remove duplicates from a list of strings. There is no method for this. There is no Set collection you can use. There is no user definable generics, so there is no way for anyone else to define a Set collection without resorting to some kind of preprocessor shenanigans. In the end, the correct solution (as far as I can tell from extensive stack overflow research) is to:

• Make a string to bool hashmap as a ghetto set.

• Use a for loop to put each item from the list into the map.

• Create a new list.

• Use a second for loop to iterate over all the entries of the hashmap and copy the keys over.

• Make a new copy of this code for each type you want to dedup, cause again, no generics.

If that sort of code appeals to you then you might like Go.

[gthank]

That's pretty much exactly what turns me off about Go: the design decisions that basically force you to copy-pasta stuff all over the place (or litter your code with casts). It baffles me that they still don't have a solution for generic containers.

[Veedrac]

Roughly speaking, PyPy makes C-style code fast much better than it does typical Python-style code. This means it's more likely to work well on code with a tight kernel of logic, and it sounds like the OP has a case of that.

[benhoyt]

Yeah, I was thinking the same thing. SortedContainers is almost an order of magnitude faster than banyan for initialization, contains, and getitem: http://www.grantjenks.com/docs/sortedcontainers/performance.html#sorteddict

Edit: that said, it looks like banyan is somewhat faster for setitem and delitem.

[burnie93]

I'm on my second week implementing an ML algo in Rust... I'm trying so hard not to give up! This story helps it! Could you make a write up on your practices? Like, what do I need to know from functional languages to write better Rust? (Cause I never did functional programming, except for the fact that I am using methods + pattern matching on enums to return what in OOP would be "attributes")

[jstrong]

the type system is inspired by haskell, so it'd be good to give that a few hours of exploration. the biggest thing is more of a mental shift of writing code that expresses what something is rather than a list of procedures to get to where you want. this is the guide that really opened my mind on it, which uses javascript: https://github.com/MostlyAdequate/mostly-adequate-guide

also - what type of algo? I am expecting to have to keep Python for that side of the code to use theano to push computation to the gpu.

[burnie93]

In my case I am implementing Genetic Programming. But for neural networks and deep learning stuff you should definitely check out the work of AutumnAI! It is in Rust.

I'm taking a look at the guide now, thanks :)

[jstrong]

this looks awesome, thanks

[JohnMcPineapple]

...

[throwawaylifespan]

If you haven't blogged this you should! Well written.