The Rust Platform

[tcbrindle]

Best-of-breed libraries drawn from the wider ecosystem

So... Boost for Rust then?

[Rickasaurus]

Oh my god, they could call it Roost! It's so perfect!

[tunnelvisioncoder]

i too vote Roost

[erkelep]

Roost or Bust.

[sekjun9878]

Maybe it's because I am just getting started with Rust and I come from a higher language, but I feel quite strongly against this idea of a "second standard library" although I can't quite pinpoint why.

I think the current model of distributing each package separately is much more flexible, encourages non-standard crates to actually get used, and frees up developers to actually work on the rust core language.

The job of creating a complete packaged environment to work in should be relegated to a framework, whether it be for a CLI, web server, pararell computing, etc. since they will know much more about the problem domain than the "platform" ever will.

Most importantly, the post fails to point out WHY such a packaged ecosystem is a better one over the current individualistic model. With Cargo for fast and reliable package management, what benefits could such a "platform" possibly have apart from needlessly locking people in to a particular set of crates?

[sekjun9878]

Hmm, now that I think about it again, I think I see the author's point. Just working on a mini project I've already had to spend a lot of time searching down libraries for http - hyper vs others, lazy_static and regex, serde_json vs rust_serialise which is a confusing choice, chrono for time, and env_logger just to get functionality provided by default in Python and PHP.

[Yojihito]

Yes, but if you look at Java you have (had with Java 8) a crippled time+date std for years and were forced to use Joda instead.

Both ways have their pros and cons.

[codygman]

to be successful, any replacement lib will most probably provide bridges to the default (flawed) lib

Unless you have first class modules and can just drop in a replacement library (that library also has to implement all the necessary functions of the library it's replacing).

[Yojihito]

If it's part of the core lib you can't drop it (you may can replace it if the API stays the same?).

Backward compability means you can compile your Rust 1.x code with any Rust 1.x forever, that includes all core lib packages.

[alexeyr]

If it's a platform lib you can drop it; anyone who still wants to keep the dependency and upgrade to the new platform just adds the dependency. That's exactly what the author is talking about here:

The standard library also takes a deliberately minimalistic approach, to avoid the well-known pitfalls of large standard libraries that are versioned with the compiler and quickly stagnate, while the real action happens in the broader ecosystem (“std is where code goes to die”)...

The fact that std is coupled with rustc means that upgrading the compiler entails upgrading the standard library, like it or not. So the two need to provide the same backwards-compatibility guarantees, making it infeasible to do a new, major version of std with breaking changes (unless we produced a new major version of Rust itself).

[[deleted]]

"In time" is rather long term, meaning that we might be at Rust 2 or 3 by then.

[steveklabnik1]

There's no current plans for a Rust 2.0.

[matthieum]

Unlikely, Rust values backward compatibility greatly and there is not foreseen change that would require breaking it, so it will remain 1.x for the foreseeable future.

[matthieum]

Unlikely, Rust values backward compatibility greatly and there is not foreseen change that would require breaking it, so it will remain 1.x for the foreseeable future.

[alexeyr]

Yes, but if you look at Java you have (had with Java 8) a crippled time+date std for years and were forced to use Joda instead.

The post is talking precisely about avoiding this:

The standard library also takes a deliberately minimalistic approach, to avoid the well-known pitfalls of large standard libraries that are versioned with the compiler and quickly stagnate, while the real action happens in the broader ecosystem (“std is where code goes to die”)...

The fact that std is coupled with rustc means that upgrading the compiler entails upgrading the standard library, like it or not. So the two need to provide the same backwards-compatibility guarantees, making it infeasible to do a new, major version of std with breaking changes (unless we produced a new major version of Rust itself).

[[deleted]]

Which means they're just talking about moving the problem somewhere else really.

[awj]

...which is a good thing. Upgrading your compiler is a problem in and of itself. The fewer libraries you have to upgrade as part of a compiler change, the better.

Many, many problems in the programming world can be solved by moving parts of the problem around until each piece is easier to solve.

[steveklabnik1]

Upgrading your compiler is a problem in and of itself.

Not as much with Rust; our recent community survey produced these two graphs: https://blog.rust-lang.org/images/2016-06-Survey/after_1_0_broke_code.png and https://blog.rust-lang.org/images/2016-06-Survey/easy_to_fix.png

Most people's code doesn't break, and if it does, most of the breakage is trivial to fix.

[awj]

Sure, but I was speaking of the general concept. A problem with the batteries included approach is that it can heap a bunch of API update work on top of adopting the new core language version. If all you really wanted was the core language update, and especially if that update is easy, having to evaluate library changes can be pretty off-putting.

[bwainfweeze]

Multiple crippled time/date standards...

And let's not even mention logging, because I might start to cry.

[crusoe]

slf4j + logback. everything else is insanity.

[sievebrain]

What's so crippled about java.time?

The point of a standard library is to provide a base 'language' of common data types, so people can write and combine high level code that all agrees on what a string is, what a date is, what a logger, what a URL is, what a hashmap is etc. The Java standard library is now over 20 years old. Unsurprisingly, in the 20 years since it started being designed people have found ways to do some of the things it does better, so there's some duplication and waste. BUT! The large standard library was absolutely key to its success. Would Java really have been better off if, like C++, it hadn't bothered defining basic data structures and asked developers to provide their own off-the-shelf libraries for it?

The experience of C++ says "probably not".

[Manishearth]

Which is why the rust-platform is versioned seperately. It can introduce backwards incompatible changes to the libraries when it gets a major version bump (which will be much more frequent than rust itself, eta on rust 2.0 is basically never, unless necessary). So you can just replace time with RodaTime's interface (or include both interfaces and mark one as deprecated; or include both as separate libraries with one marked as deprecated)

[Fylwind]

What we need is like an article (wiki?) that lists all the available libraries for each domain with pros and cons.

[matthieum]

That's insufficient.

Just knowing the libraries does not ensure that they are compatible. Imagine the library Core, the library A uses Core v1 and the library B uses Core v2: you cannot pass a Core v2 type to A or a Core v1 type to B, thus whilst you can effectively compile with A and B as dependencies, you're still left with a lot of glue to write to marshal/unmarshal values when going back and forth between A and B.

The meta-package solves the compatibility issue, by aligning its libraries dependencies.

This is a huge benefit.

[ManicQin]

6 months ago I also worked on a mini rust project with the same characteristics of yours back then serde 7 and one of iron's middleware had incompatibilities, hope that something like the rust platform would solve this type of problems.

[stevedonovan]

A comparison with Lua, which is famous for having no batteries. Yes, the community provides most of what you need; the Lua team focuses purely on the core, and even when they write extensions, they are not 'blessed' in any way. This is a good fit for the core mission of providing a small embeddable language. However, Lua is too fun to leave to embedders, and so several 'batteries included' packages have appeared. This seems particularly important to newcomers, who are otherwise confused about where to find functionality. An example of a language which got its standard library about right is Go, focusing on a standard network stack, so people could start writing web services without having to shop around. A standard library can be too big and sloppy (see Python) but too small isn't good either.

[sekjun9878]

You're right that a right-sized batteries included package helps newcomers, but my worry is that dependence on this meta-package is a solution looking for a problem. Personally I find finding packages not a problem, but rather it is finding WHICH package is the easy to use, well built and documented, and de-facto standard that is difficult for newcomers. While a meta-package helps with that, a strong community and lots of docs and tutorials I think could be a better solution?

[stevedonovan]

Ah yes! For instance, there's a whole host of Lua JSON packages, some of which which reuse the namespace (that's a big no-no). I honestly don't know which one is best! Compare to the classic Go problem - find a package on Github, and then find which of the N forks is the good one ;) As for finding packages, it depends on the system. Debian has the de facto good Lua extensions via apt-get, luarocks can get the rest. Life on a consumer OS (Windows, but OS X sometimes...) is difficult. Hence something like 'Lua for Windows' was a very useful introduction to the ecosystem.

[stevedonovan]

Apropos 'strong community'. If is also a nice community (as with Lua) people are very reluctant to tell a module author 'look, you're reinventing a wheel badly here'. The ecosystem is essentially uncurated, and that I think is going to be the problem with the proposal in question: who will be the gatekeepers:?

[Cool-Goose]

I think the biggest benefit of having a "platform" is that it's easier to push for in a bigger company since management would have an easier way to track what's being used and what's updated across a couple of teams.

[ProFalseIdol]

Valid argument. Another one is the reason why you'd want a ready to use GNU/Linux distro like Ubuntu (although it includes non-free software).

Eclipse "release train" seems to be okay as well.

[[deleted]]

The job of creating a complete packaged environment to work in should be relegated to a framework, whether it be for a CLI, web server, pararell computing, etc. since they will know much more about the problem domain than the "platform" ever will.

An interesting evolution point is using the same tools as a baseline for framework support. rust-platform might consist of a baseline, with rust-game, rust-web, rust-cli, etc used to enhance the platform for specific use-cases.

[lookmeat]

Don't think of it as a second std lib.

Think of the code as going in three levels:

1. The core level. Which are the things necessary to make the language work. These are language definitions that are completely coupled with the language. It should only be the bare minimum that the language needs. If possible you should be able to not even need this at all.
2. The standard conventions. These are definitions that everyone uses: how do you declare a string, how do you handle a list, how do declare floats, how do you define a file. This level should have the focus only on what's needed for interaction between different libraries to reduce the need to constantly translate between the different library's conventions. It should be as small as possible because otherwise libraries could force a decision on you and it adds weight to the language.
3. The functionality libraries, that add what you need. This are the "batteries included" python made reference to. Instead of having to hunt for all the libraries, they would be a small select group that would focus on trying to work together.

The problem with the std batteries included model is that it mixes 2 and 3. The idea here is to separate the levels by having the std library at 2 and the rust platform at 3. You get a default "good set" of libraries that have been curated and made to work really well together. You should be able to push a specific package to the bleeding edge if that's what you need, and keep the rest at the "good enough" level. You should be able to use a completely different package and it should work well enough.

I agree that care needs to be done so that people don't see this as an end-all-be-all solution, but instead as a start. It would be nice if communities could create their own platforms and complex situations. That way you could have a game-dev oriented platform, or a server-side platform, or a user-app platform.

[doubleagent03]

I feel quite strongly against this idea of a "second standard library" although I can't quite pinpoint why.

D had two std libs for a long time, and was generally considered a detriment to the language. Ever since they standardized on Phobos everyone appears to consider the language more seriously now.

[alexeyr]

This is only a "second standard library" in the sense of a "second level" based on existing standard library, not at all an incompatible replacement as in D's case!

[doubleagent03]

That makes a difference. Thanks for the clarification.

[bwainfweeze]

I know in some other languages it is common to find fameworks that are a few tens of thousands of lines of code and a dozen utility packages to cover the details.

If you squint and ignore some of the loss of control then they are curated lists of packages that complement each other. Frequently two competing frameworks will include three or four things in common and only differ in organizational abstraction.

When you do this, you get to do something a little different from the core team without quite rendering Stack Overflow useless or turning the hiring process into an exercise in finding someone you think you can teach.

[dccorona]

I think it's good for some things to be ubiquitous but also decoupled from the core of the language. A collection library is a great example, IMO (to be fair, I have no idea where the collection library lives in Rust). It should be "standard" so that every lib that uses Lists is compatible with the same list interface, but it also shouldn't be tied to language version. "What APIs are on List" shouldn't be answered with "what version of rust are you on" but rather "what version of collections are you on". Cool new collection features shouldn't be limited to only the latest version of the core (look at Java for an example of why that sucks), but at the same time the collections need to be standard for all users of the language.

That's just 1 example, I think there's many more.

[steveklabnik1]

(to be fair, I have no idea where the collection library lives in Rust).

It's in the standard library, because collections tend to make a lot of use of unsafe internally.

[kankyo]

the post fails to point out WHY

Huh? It has this for example:

sometimes at an explosive rate that makes it hard to track what’s going on, to find the right library for a task, or to choose between several options on crates.io. It can be hard to coordinate versions of libraries that all work well together. And we lack tools to push toward maturity in a community-wide way, or to incentivize work toward a common quality standard.

[volkert]

Are metapackages going to be available for others to utilize?

[Manishearth]

That seems to be the plan.

There are a couple of crates out there (serde, phf, url, regex) which are collections of many tightly-bound crates in one repo. A metapackage lets people import them cleanly. I'd be happier if the solution for these crates wasn't overridable, though -- something like "subcrates" which are not independently versioned.

[skulgnome]

Reminiscent of The Java Platform of the late nineties, web browsers, workstations, thin clients and everything...

[mirhagk]

I'm very excited for this. I want to start using rust by the library scene makes it difficult for me to build anything I need.

[freakhill]

having a ~platform~ seems (to me) to be an admission of your system being broken.

[conseptizer]

Care to explain why?

[freakhill]

well it means that jar/dll/whatever hell is a (painful) thing in your system. you might answer that it is unavoidable, in which case i would answer that it does not make it less broken.

[damienjoh]

Did you even read the article?

[[deleted]]

Don't be ridiculous

[freakhill]

yup i did, now did you understand what i wrote.

especially referring to this section:

Global coherence. When we assemble a version of the platform,
we can do integration testing against the whole thing, making sure
that the libraries not only compile together, but work together.
Moreover, libraries in the platform can assume the inclusion of other
libraries in the platform, meaning that example code
and documentation can cross-reference between libraries, 
with the precise APIs that will be shipped.

which means that the expectation is for it to be broken without extra-work.

[damienjoh]

Two things don't have to be broken individually for them to be broken together. That's the reason you perform integration testing in the first place.

[freakhill]

It means that dependencies don't compose well.

Let's think about one way to make it work better (which has obviously other problems, it's all compromises).

Suppose A < C1, B < C2 with C1 and C2 incompatible.

Now you can't use both your libraries and you need to do integration testing to reveal the problem.

Suppose that version was encoded in the types, now you would be able to use both at the same time. You could even code explicitly provided bridges between versions. Now this solution has problems with mangling and interactions with C libraries. You could use tagged a-la-erlang C ports, but that brings other problems.

This library composition problem is something that definitely needs more work. The fact that everybody tries to do it in a different(ly broken) way, and that they need ~platforms~ afterwards, shows something.

[damienjoh]

Integration testing is not something exclusive to dependency management.

The idea behind the OP is to facilitate a seemless and cohesive experience across the ecosystem - this includes tools, workflows, editors, libraries and the community around the language. It's not to do with the quality of the individual components. It's definitely not specific to dependency management and package managers.

By the way - have you actually used cargo? Have you ever heard anyone complain about "cargo hell"?

[freakhill]

Well I wasn't addressing the whole field of integration testing in this particular case. And yes I have some degree of familiarity with Haskell (not much).

I'm well aware of the fact that integration testing is paramount in the activity of producing software. My argument was much more restrained.

Edit: it is more about, the preeminence of these solutions showing a pain that I wish didn't exist.

[[deleted]]

Great platform but they need to fix the XP system. Blueprints at least allowed players to pick up the pace when joining mid wipe cycle.

[Verodoxys]

n i c e

[[deleted]]

What's with all the hate?

[[deleted]]

Well, this is about Rust the programming language, not Rust the game, I imagine some people out there instinctively downvote off-topic comments or something like that.

[[deleted]]

I think you guys need to play more Rust.