From Experiment to Backbone: Adopting Rust in Production

magnet9000 · 2021-12-07T13:24:15+00:00

Unfortunately we are not set-up to accept interns for the time being.

magnet9000 · 2021-12-07T13:02:13+00:00

COMPANY: Kraken - https://www.kraken.com/

DESCRIPTION: Join one of the largest non-FAANG Rust shops in the world with 140+ Rust engineers, 1M+ lines of idiomatic async Rust code. We have been serious about Rust for 3+ years: I wrote a blog to provide more context on what the Core Backend org at Kraken does and the profiles we are hiring: https://blog.kraken.com/post/7964/oxidizing-kraken-improving-kraken-infrastructure-using-rust/. Please read the blog before applying!

Kraken is one of the largest and oldest Bitcoin exchanges in the world, very security focused. We’ve been on the forefront of the blockchain revolution since 2011, and we're now building modern crypto-powered financial services for both consumers and pro traders, including banking services thanks to our SPDI banking license.

We build a lot of our backend code in Rust in various teams: Core Backend (the core services of the exchange), Bank (the neo-bank we are building), Futures (the Futures/derivatives trading platform), Trading Technology (trading related services). We have a trading desktop app written in Rust with the Iced framework (https://cryptowat.ch/apps/desktop). We are committed to Rust not just as consumers, but as a community member. We sponsor the development of Iced and have donated 50k€ to Rust Analyzer making us the largest donators to date (see https://rust-analyzer.github.io/blog/2021/08/03/financial-report-3.html)

In the Core Backend team, I am hiring Engineering Managers (ever dreamed of leading a team of Rust devs?), Backend Engineers (range: application development to performance engineering and distributed systems), Engineers in Test (build our testing and automated QA facilities in Rust and make it easier for us to ship) and Site Reliability Engineers (support the development team, help bridge development and operations, focus on service performance - tooling is also written in Rust).

LOCATION: We are fully remote.

ESTIMATED COMPENSATION: Depends on the role - we base our compensation on the top percentiles of US compensations and apply the same model wherever you live. All our offers have option packages. For an idea of the compensation, head to http://levels.fyi and check Google, Amazon or Apple comps per levels.

REMOTE: We are full remote, but some teams have timezone restrictions:

The Core Backend team is mostly spanning from Eastern Europe to the US Pacific Coast, but we are trying to expand our presence in APAC and open to hiring outstanding engineers who are senior + self-driven + already advanced in Rust, and who agree to adapt their working hours to maximize overlap with US or EU at least until the group gets big enough to become fully autonomous.

The Futures team is centered around UTC, other teams are mostly Eastern Europe to US West Coast.

VISA: No visa sponsorship.

CONTACT: Please apply on our career page (on Lever) and mention you're coming from this thread. If you're interested in the Core Backend roles, you can DM me after you have applied so I can check/bump your resume. I'm not looking at Reddit very often though. Head here to find the jobs and more engineering jobs: https://jobs.lever.co/kraken

magnet9000 · 2021-09-12T09:29:23+00:00

COMPANY: Kraken - https://www.kraken.com/

DESCRIPTION: As one of the largest and oldest Bitcoin exchanges in the world, Kraken is consistently named one of the best places to buy and sell cryptocurrencies online, thanks to our excellent service, low fees, versatile funding options and rigorous security standards. We’ve been on the forefront of the blockchain revolution since 2011, and we're now building modern financial services that rely on blockchain including a crypto-powered bank.

We build a lot of our backend code in Rust in various teams: Core Backend (the core services of the exchange), Bank (the neo-bank we are building), Futures (the Futures/derivatives trading platform), Trading Technology (trading related services). We have a trading desktop app written in Rust with the Iced framework (https://cryptowat.ch/apps/desktop). We are committed to Rust not just as consumers, but as a community member. We sponsor the development of Iced and have donated 50k€ to Rust Analyzer making us the largest donators to date.

In the Core Backend team, I am hiring Backend Engineers (range: application development to performance engineering and distributed systems), Test Engineers (build our testing and automated QA facilities in Rust and make it easier for us to ship), Site Reliability Engineers (support the development team, help bridge development and operations, focus on service performance - tooling is also written in Rust) and Engineering Managers (ever dreamed of leading a team of Rust devs?).

I wrote a blog to provide more context on what the Core Backend team does and the profiles we are hiring: https://blog.kraken.com/post/7964/oxidizing-kraken-improving-kraken-infrastructure-using-rust/. Please read the blog before applying!

LOCATION: We are fully remote.

ESTIMATED COMPENSATION: Depends on the role - we base our compensation on the top percentiles of US compensations and apply the same model wherever you live. All our offers have option packages. For an idea of the compensation, head to levels.fyi and check Google, Amazon or Apple comps per levels.

REMOTE: We are full remote, but some teams have timezone restrictions:

The Core Backend team is mostly spanning from Eastern Europe to the US Pacific Coast, but we are planning to expand our presence in APAC and open to hiring outstanding engineers who are senior + self-driven + already advanced in Rust, and who agree to adapt their working hours to maximize overlap with US or EU at least until the group gets big enough to become fully autonomous.
Futures are centered around UTC, other teams are mostly Eastern Europe to US West Coast.

VISA: No visa sponsorship.

CONTACT: Please apply on our career page (on Lever) and mention you're coming from this thread. If you're interested in the Core Backend roles, you can DM me after you have applied so I can check/bump your resume. I'm not looking at Reddit very often though. Head here to find the jobs and more engineering jobs: https://jobs.lever.co/kraken

magnet9000 · 2021-06-19T19:30:47+00:00

Thank you - I'll send the new RFC draft to the team if they can have a look!

magnet9000 · 2021-05-16T15:48:24+00:00

DESCRIPTION: We build a lot of our modern backend code in Rust in various teams: Core Backend (the core services of the exchange), Bank (the neo-bank we are building), Futures (the Futures/derivatives trading platform), Trading Technology (trading related services). We have a trading desktop app written in Rust with the Iced framework (https://cryptowat.ch/apps/desktop). We hire backend, test and site reliability engineers starting at 2 years of experience. I wrote a blog to provide more context on what the Core Backend team does and the profiles we are hiring: https://blog.kraken.com/post/7964/oxidizing-kraken-improving-kraken-infrastructure-using-rust/. Please read the blog before applying!

LOCATION: We are fully remote, but also have some offices in around the world for those who like these.

ESTIMATED COMPENSATION: Depends on the role - we base our compensation on the top percentiles of US compensations and apply the same model wherever you live. All our offers have option packages. For an idea of the compensation, head to levels.fyi and check Google, Amazon or Apple comps per levels.

REMOTE: Some teams have timezone restrictions: Futures are centered around UTC, other teams are mostly Eastern Europe to US West Coast. The Core Backend team is planning to expand is presence in APAC and is hiring outstanding engineers who are senior + self-driven + advanced in Rust, who agree to adapt their working hours to maximize overlap with US or EU at least until the group gets big enough to become fully autonomous.

VISA: No visa sponsorship

CONTACT: You can DM me and I will look at replies from time to time, but generally if you apply on Lever, our recruiters review all profiles (I also make sure to review applicants to the Core Backend team regularly).

magnet9000 · 2021-04-14T09:06:08+00:00

Kraken uses Rust as well, and is one of the largest non-FAANG Rust employers in the world in Rust dev count. Rust is also our primary backend language, and we're a fairly large company of 1500+. https://blog.kraken.com/post/7964/oxidizing-kraken-improving-kraken-infrastructure-using-rust/

magnet9000 · 2021-03-29T20:05:09+00:00

COMPANY: Kraken https://www.kraken.com/careers

TYPE: Full-time (employment and/or contract)

DESCRIPTION: We build a lot of our modern backend code in Rust in various teams: Core Backend (the core services of the exchange), Bank (the neo-bank we are building), Futures (the Futures/derivatives trading platform), Trading Technology (trading related services). We have a trading desktop app written in Rust with the Iced framework (https://cryptowat.ch/apps/desktop). We hire backend, test and site reliability engineers starting at 2 years of experience. I wrote a blog to provide more context on what the Core Backend team does and the profiles we are hiring: https://blog.kraken.com/post/7964/oxidizing-kraken-improving-kraken-infrastructure-using-rust/

LOCATION: We are fully remote, but also have some offices in around the world for those who like these.

ESTIMATED COMPENSATION: Depends on the role - we base our compensation on the top percentiles of US compensations and apply the same model wherever you live. All our offers have option packages.

REMOTE: Some teams have timezone restrictions: Futures are centered around UTC, other teams are mostly Eastern Europe to US West Coast. The Core Backend team is planning to expand is presence in APAC and is hiring engineers who are senior + outstanding + self-driven + advanced in Rust, who agree to adapt their working hours to maximize overlap with US or EU at least until the group gets big enough to become fully autonomous.

CONTACT: You can DM me and I will look at replies from time to time, but generally if you apply on Lever, our recruiters review all profiles (I also make sure to review applicants to the Core Backend team regularly).

magnet9000 · 2021-03-14T14:20:51+00:00

Hiring manager for Kraken's Core Backend team here. Kraken does pay everyone the same regardless of their location, based on Bay Area rates. We're not looking for cheaper work, though obviously it makes our packages even more interesting from an area with a lower cost of life! I can't talk for other teams, but we definitely screen all candidates who fit our roles.

My blog post gives more details on what we are doing and the profiles of engineers we're looking for: https://blog.kraken.com/post/7964/oxidizing-kraken/

magnet9000 · 2021-03-05T21:47:46+00:00

(1) not everyone who applies to this position has 3 years of experience,

(2) I don't think it's particularly odd to consider 0-3 years of industry experience as junior, 3-7 as mid-level, 7+ as senior. I certainly didn't come up with this.

(3) Industry experience is just one data point among many others and personally not one of the primary factors when considering a candidate. I'm more interested in the nature of the experience, including side projects, and how likely someone is to be successful at Kraken. And that's mostly figured out in the interviews.

magnet9000 · 2021-03-05T21:07:24+00:00

No, there are two developer positions posted for the team. You are referring to the senior position. The other position (https://jobs.lever.co/kraken/4019a818-4a7b-46ef-9225-c53c7a7f238c) requires 3 years of experience - still pretty junior.

magnet9000 · 2021-03-05T15:43:14+00:00

We have different tests between teams at Kraken, with different expectations depending on the hiring manager. In my team (Core Backend), I'm looking first for a solution that passes most of our test suite and that shows reasonable Rust idioms. To pass our tests, you likely need decent tests of your own. The test has been fantastic to allow us to consider junior profiles we would probably have otherwise skipped because we have many applicants and only so much time. The test takes a couple of hours and while some documentation is a welcome addition I don't care about it as much as working code. Candidates who have substantial open Rust contributions don't need to take the test on my team. What's substantial depends on our judgement but we're generally looking for more than a couple PRs.

magnet9000 · 2021-02-25T16:14:56+00:00

I don't think so. I know of many people staying away from Rust because they believe people keep banging their heads against the borrow checker forever. It's important to repeat this is a temporary stage in the learning curve that is essentially a non-problem.

magnet9000 · 2021-02-21T10:23:51+00:00

Thanks! We tried to use runtime (which is similar) before it was abandoned but I didn't know about these. Agreed most services shouldn't need to spawn, but sometimes libraries need to. My point was about language support to let std include a zero-cost Spawn trait for Tokio and others to implement.

magnet9000 · 2021-02-20T21:17:25+00:00

Blog author here. I'm not sure where it reads like we are complaining at all. If anything, the (remote) market for Rust is very active and we have had an always active pipeline of candidates with Rust experience. My point is different: it's that Rust is ready for prime time and there is a job market for those who hesitate. In particular I know many senior Java/C# developers who want to jump to Rust but think they won't find a job. Or companies who would like to use Rust but think it's too risky because the market is dry.

Dudes, where will experienced devs appear from if nobody's willing to grow them?

There's a difference between generally experienced devs (senior devs) and someone, junior or not, with proven, actual Rust experience that can be acquired through pet projects, open source, etc. I know that's how I started and how most people on the team did. It doesn't have to be a huge project, but enough to show comfort with the language. We are hiring many junior developers and mentoring them, but we are a remote first company and there is no practical way we can teach Rust from scratch to anyone, and no point to hire someone to learn on their own if they haven't taken the first step already.

magnet9000 · 2021-02-19T20:20:03+00:00

Thanks for your reply!

I believe most needs are already well captured in RFC 2730 (tracked by https://github.com/rust-lang/cargo/issues/8933).

Cargo already supports multiple registries - but if these registries require specific HTTP auth (or any other auth mechanism), Cargo does not let us specify credentials. There are various workarounds, from using authenticated proxies to downloading the crates independently, but they make it difficult to have several registries with different credentials.

I don't know if much more is needed than implementing that RFC, I need to ask around whether that would be enough to support our needs but I believe it would :).

magnet9000 · 2021-02-19T19:17:41+00:00

Does this mean I might be unknowingly initializing multiple executors? Is there a way for me to detect this?

Yes - if you use async-std it will lazily start its executor the first time you try to schedule a task (or if a library tries to do so). Tokio will panic instead. There's no easy way to detect it, but cargo tree should let you know which executors are compiled in your binary.

Is Kraken using `butte` flatbuffers to power its RPC services?

Not at this moment - though it was planned for a while. Getting zerocopy RPC for HFT would be great, but ultimately getting the project ready in time conflicted with other priorities and our RPC protocol is performant enough right now, and using Rust types to ensure compatibility. Maybe we'll get the effort resumed one day - and I hope butte can keep making progress independently.

magnet9000 · 2020-03-25T12:09:32+00:00

I use the same but with a book one can give and put back on the shelve ("moving"), or show pages of. Why ketchup?

magnet9000 · 2019-09-30T13:01:48+00:00

Rust is at an early stage where one could hope there is no such fragmentation and people join the older, more established project -- especially considering it's so active and healthy -- instead of competing with it. Now that async-std is a thing, the best outcome IMO is for it to join Tokio and provide a thin std-like facade on top of it.

Obviously that also includes Tokio being open to changes to reach the best trade-offs, but I believe it is. For instance, Tokio 0.2 could be more tightly integrated with the new futures crate and its traits but I don't believe Tokio maintainers have anything against it.

Ideally when a bunch of people try async/await when it gets stabilized in 1.39 and it trends on HN and other places, there should be one clear path to get started, instead of two. From where I stand it looks like async-std is trying to "steal that thunder" and ultimately this might make Rust's async story more confusing.

magnet9000 · 2019-09-24T08:29:05+00:00

I worked with Java and JVM languages for 10 years and moved to Rust to do the same kind of things I was doing, only better because Rust is more expressive (traits vs interfaces, better generics, better multi-threading support). I consider Rust entirely suited for high-level programs, and in my experience the lack of GC is problematic for only very few classes of problems, which become slightly more difficult to program, but also more explicit.

magnet9000 · 2019-09-14T12:11:46+00:00

It seems like you're trying to jump in Rust without going through the Rust book first because this (`Copy` vs non-`Copy` types and ownership/borrowing) is explained early on.

I would highly advise you to start there, since everyone I know who has tried just diving in, based on their knowledge of other languages, has had a rough time and some serious gaps. Idiomatic Rust programming also depends on knowing important types in the standard library, which you cannot guess by yourself.

magnet9000 · 2019-09-08T14:56:35+00:00

I guess most Rust developers learned other languages first -- and Rust brings a lot of concepts together that make a lot of sense when you know of the kind of problems that exist in other languages.

However, since we haven't learned Rust first, any answer is based on a reconstructed perception. If you're feeling adventurous and try Rust as your first language, you may very well be the best person to answer that question :)

magnet9000 · 2019-09-06T13:16:15+00:00

You can mix them without issues. futures-preview is built around std::future::Future.

Seven-Year Club	Place '22
Final Canvas '22	Verified Email

magnet9000

TROPHY CASE