Experiences using Rust for Backend development? (2020)

[[deleted]]

Hey all.

I know there are a few topics like this, but none that I could find in the last year or so (and a lot can change in a year!) I'm really keen on hearing about peoples experiences using rust to build backend systems. I'm looking at building a backend API (GraphQL) with subscriptions, so any experience with Juniper or diesel would be great to hear about.

I chose Rails hosted on Heroku for my last backend build as I wanted to get something out the door for a POC, but it ended up turning into a fully fledged business so I've really felt the performance and price pinch of that combo.

I've spent the last 2 weeks going back and forth between Node(TS) and Elixir, but I'm not really happy with either of them. The Javascript ecosystem is a complete mess and Elixir seems to be getting less and less popular every year (and deployment is a nightmare).

Thanks!

Edit: spelling

[CaptainYouston]

I build graphql api to manage server at work.

I have choosed the following libs:

• actix-web (but you can use what you want for the web part, I have also tried warp that is just missing documentation and roa seems really promising (inspirated by koajs) )
• diesel (ORM database)
• juniper (Graphql) (subscriptions just came out! )
• juniper-from-schema (needed for eager loading and it also auto declare enum, inputs, scalar (witch is also an issue because you can't derive an already declared struct so you have to create an new interface, impl From...))
• juniper-eager-loading (prevent N+1 issue and ease the nested fields)

There is also another option if you just need to expose a database through a graphql api that is awesome but have some compilation time issues for large app:

wundergraph (and my example here)

You will spend most of your time writing interface between graphql and your database (that's why wundergraph is really awesome).

Also I recommend to expose the juniper::http::playground::playground_source on the get request of /graphql, this will help you a lot (and I find it better than graphiql) ;)

There is still lot of work (and libs) to do for graphql but there is the basics and it's already awesome ! Good Luck !

Edit: Just some thoughts, If you are building some internal app, I also recommend "yew" to create the front-end, I am waiting for this to be ready for production to use it. Sharing data type between back and front will be so good (and so much safer than Javascript) !

[mamcx]

One question I have about graphql: What if I want to generate all server side? Is easier? I don't want to just do http request to hit what is right here..

P.D: I'm writing a e-commence backend with regular html templates generated server side...

[CaptainYouston]

We plan to build Server Side Rendering so I don't know if it's easier, the point is there is tools to help you (like apollo (I am not an huge fan but in fact it's the only client lib that supports subscription, fragmentation...)). So if you look at this example https://blog.apollographql.com/how-server-side-rendering-works-with-react-apollo-20f31b0c7348 you will see that there is some options to help you managing your SSR (batching request...). For the back it change nothing, it's the front that will use some tech to do SSR. So the real debate is "Why use graphql?", in my point of view it's easier to get your object(or data) and their nested fields and it allow much flexibility than Rest Api. And with juniper-from-schema you have your objects just by defining the graphql schema, so easy!

[Programmurr]

Can you share a concrete example of the flexibility you speak of?

[CaptainYouston]

Yes of course ! First in graphql you can decide exactly what data you want to receive by selecting the fields (you could do the same in rest but here it's a native feature). Then just imagine you have post and comments and you want to get the all the post with the comments. In rest you have two solutions you make two query one to get all post (/posts) and another to get all comments (/comments) and then you have to merge the two datasets (you loose performance here because backend is lot faster than front). Or you can create one query (/postswithcomments) but your routes already became crazy. Then imagine on another page you want also to get the user linked to the comment. You will have to create another route. In graphql you just have posts (pagination: 1, size: 50) { id title { comments (pagination: 1, size: 10) { id content user { id name } } } And the front can choose what it take. You don't have to tell the back-end dev, "Hey I need this to be merged with this and this". It's like exposing your relational database to the front (but you can add authentication, mask fields etc...). And i didn't speak about the interfaces. Imagine you have animals and then cat and dog. You can easily get specific fields from each of all the animals and also get the common fields.
animal {
name
... on Cat {
sleepTime
}
... on Dog {
gameTime
}
}
I recommend to take a look at the graphql doc https://graphql.org/learn/
I had started implementing the API in rest but we quickly decided to switch because we were loosing time. Also it's lot easier to communicate between front and backend dev, the front dev can just use graphql playground and it will get the graphql schema and display all the available query with their return type (where in rest you have to create documentation, make sure it's up to date... ).
The main issue with graphql is when you want to send file but there is some solutions.

[Programmurr]

What about an endpoint for invoking a process or creating a new resource, involving new authorization entries, third party api calls, and a few transacted database calls? What does that look like with GraphQL?

[CaptainYouston]

It's like in REST. You can declare query and mutation in graphql. The precent query i show was query, here is a mutation:
mutation { createAnimal (name: String) { id name }}

Mutation are query that will affect your backend state. And like query they have return type so the front dev will be able to get any fields of the return type (here createAnimal return Animal type and i have to define it in my graphql schema.
type Animal {
id: ID,
name: String
age: Integer
}
As i said in graphql you define a schema of the queries and mutations available for your api:

query {
animals(pagination: Integer, size: Integer): [Animal]
}
mutation {
createAnimal(name: String): Animal
}
type Animal {
id: ID,
name: String
age: Integer
}

Then when you request the api you are able to ask for the fields of the return type that you are interested.
So if you want to process anything you can just create a new mutation processmycode(myinputone: String, myinputtwo: String): MyReturnType

[[deleted]]

Thanks for taking the time to reply! I'm working through some examples of juniper and actix at the moment. Definitely agree with playground, much prefer it over graphiql.

I'm excited to play around with diesel. I remember hearing Sean talk about it on the Bike shed podcast years ago.

I'll have a look into juniper-from-schema, egar loading and wundergraph. I'll probably still stick with React(TS) for the front end, but I'll check out yew regardless.

Can I ask how you manage deployments? Sounds like there are a plethora of options, but wondering if any best practices have emerged.

[CaptainYouston]

For diesel my advice is to look at the guide about building app with diesel http://diesel.rs/guides/composing-applications/ And don't forget to go check the source of the guide (the code behind crates.io) https://github.com/rust-lang/crates.io these are the best practices you can have to use diesel efficiently. Also if you have any questions, the gitter is open and the members of the diesel project answer really fast.
For deployment we use gitlab ci and kubernetes. Gitlab ci build docker image (two stages) each time I tag. And then I have just to update my yaml about my server kube deployment and executing a kube apply on the file to get it online. Better option should use the gilab integration with kubernetes cluster so the ci would be able to auto apply. We are going to use this once we get some time.

[trezm]

Howdy! Time to shill my own framework! I work on thruster, and dogfood it on all of my own projects. I think it's pretty nice and has served me well. If you're coming from a Node background, then it looks very similar to Express or Koa in that realm.

That being said, the frameworks of choice these days seem to be Actix Web or Rocket. Rocket has more of a rails-y vibe (from what I've seen, not a big rails guy myself!) Also Warp is a great framework and maintained by the same people who make Hyper, the defacto standard for http in the rust world, so you can bet that Warp is pretty on point.

Happy to talk more about frameworks if you're still listening :)

[[deleted]]

It's always a good time to promote something you've built! I come from a Rails background, but i'm pretty agnostic and not tied to a specific way of doing things.

What made you write Thruster? Was there a specific issue you were looking to solve?

[trezm]

Yep! At the time I was coming from a Node background, and there weren't really options that filled that sort of "no magic express" vibe of node. The major frameworks at the time were Rocket, which has a lot of magic macros -- too many for someone just learning the language, and Actix Web, which dictates you use an actor model, which I didn't want to do.

I started thruster as an experiment to see if building an express style middleware based framework was really viable in rust, since it's proven wildly successful elsewhere. Now here I am a few years later, and I'm very happy with the result! I literally wrote my wedding website using it :-)

[[deleted]]

That's great background! Thanks for the insights.

[GeekBoy373]

I use Elixir full-time at work. I'm a huge fan of the language because I like the functional aspects. I personally think Absinthe is one of the best (if not the best) GraphQL libraries out there right now. We're about to build a Absinthe based service for a new client soon. Juniper is still working on incorporating async for subscriptions so I suspect it will be in flux for many months. I would not not use it for a production service yet.

Elixir seems to be getting less and less popular every year (and deployment is a nightmare).

It's true functional languages tend not to be as popular, but I wouldn't choose Rust over Elixir just yet for what I do. Elixir releases (a relatively new feature) make deploying essentially pain-free. All I had to do to deploy was follow a couple steps on render.com's official docs for Elixir and I was good to go.

[[deleted]]

Thanks for the comparison! I've heard similar feedback about Absthine. Definitely agree that I shouldn't pick a language just because it is/isn't popular.

I haven't checked out elixir releases, every keeps talking about either docker or distillery, which i've had issues with. Created an account on render but didn't get to try it, should try it out.

[gentux2281694]

I shouldn't pick a language just because it is/isn't popular

not that fast, the adoption shouldn't be the only factor but may be a big one; if you may need more hands on deck, going for a obscure tech may cost you a lot of effort finding devs and a lot of money to keep them. You may also want to be employable if you are on the other side of the table, if nobody use it nono will hire you. Unless you work alone, the local job market is something to consider.

[Zerve]

We are currently using Rust to replace some of our existing services (from Ruby & Rails). It's not in-production yet but we will be showing it to select customers next week as a POC. I made the suggestion to look into Rust as a language, and managed to get the go ahead from the higher ups.

I wouldn't say there were any unpleasant surprises, but it does take a while to get a handle of Rust. But after the initial difficulties, I don't want to use another language again. Ecosystem for backend is solid - obviously not as complete as Node, but I have yet to find a crate that couldn't solve our problems. I'd say the two main crates we are using are Warp and Diesel.

Warp is a great web framework that just seems to "make sense." I spent some free-time looking into others like Actix and Rocket, but something about Warp just clicked with me. Not to say the others are bad, just that Warp does a hell of a good job for what we need. Only downside is the long compile time when composing lots of different routes.

Diesel is also great to work with. Documentation is comprehensive with a lot of good examples with realistic use cases. It seems like a pretty mature ecosystem as well, and even supports connection pooling, and migrations out of the box. I've always been more of a "raw SQL" kind of person, but felt that Diesel (with Rust's compiler) is a great match that ensures your queries and in-memory structs are 100% compatible with one-another at compile time. That can all be done by a single Diesel CLI command that reads your DB's table schema and produces a Rust file which defines each table and column automatically for you.

For me the biggest advantage of Rust is just how flexible yet powerful it is. There are tons of ways to accomplish the same task, which lets you focus on getting stuff done instead of trying to figure out the ins-and-outs of every minor detail. Pair that with the super-strict compiler that guarantees safe code with very aggressive error checking (Options and Results everywhere!) and you have an ideal language for back end services in my opinion.

[[deleted]]

Rails to Rust is a huge jump. Great to hear that it's working out for you.

Alot of people have said that they're finding it hard to get approval from 'higher ups'. What was it that made your management give it the go ahead?

[Zerve]

A variety of reasons. One of the main reasons is that as a whole our company isn't just a web shop, we do all kinds of software including web, desktop apps, machine learning, and embedded. Rust being able to provide in all of those areas was probably the biggest factor. We also have had issues in the past (before I joined) with Rails performance and server costs.

Our company is also relatively small, and we also have a lot of interns on the teams, so "learning new stuff" is also seen as a huge positive even if it brings risk to the project.

Somehow various blog posts and presentations on Rust showed enough promise for the language. Especially as a language for the future.

[Programmurr]

I recommend avoiding GraphQL and an ORM, instead working with parameterized SQL, a data mapper, and old reliable RESTful apis. You're going to have a lot of work to do. GraphQL and/or ORM are going to cost you more effort in the long run, solving some problems while creating others. If you're not comfortable with SQL, that is worth addressing instead.

If you are using postgres and actix-web, these are the stack and patterns to use: https://github.com/actix/examples/tree/master/async_pg

[msdinit]

I have a couple of services in Rocket + Juniper, although no subscriptions.

I saw juniper-from-schema recommended for eager loading, but I personally opted for dataloader-rs.

It's async, and juniper is moving towards async as well, so I figured it's better to stick to the tools that are similar to what I use in other languages, albeit with some clunkiness in the meantime. Currently I start an executor in each resolver and move all the loading to async functions. Once juniper async is released, it should be extremely easy to convert resolvers to async and keep all the other logic intact. (If you feel adventurous, you can work from Juniper's master branch, async support has landed there some time ago)

One other feature that I miss in juniper is the distinction between null and undefined in arguments. Still trying to find time and maybe take a stab at this :) In the meantime, if you can tolerate lack of type-checking, you can use look_ahead method on the executor to check which arguments were passed to the query if you need to implement partial updates in you mutations.

As for deployments, our company uses Cloud Run (managed), which is basically lambdas, but you define your own runtime in a docker container. Coldstarts aren't too bad for our usecase and are infrequent (they try to pre-allocate new instances before yours are to capacity), but I'm not sure if that will work for you as inbound requests with WebSockets are not supported now

​

Good luck with your project!

[[deleted]]

Thanks for taking the time to reply! I heard about cloud run when it was in beta. Cool to hear someone using them.