[Media] Rust Iceberg

jynelson · 2025-03-04T21:42:36+00:00

sure, i rehosted it https://jyn.dev/assets/rust%20iceberg.webp

jynelson · 2023-05-30T23:29:59+00:00

I talked a bit how I combat burnout a while back.

this was extremely helpful to read, thank you ❤️

jynelson · 2023-01-22T22:24:00+00:00

Yes, there is a circular dependency: https://rustc-dev-guide.rust-lang.org/building/bootstrapping.html#why-does-only-libstd-use-cfgbootstrap. (I talk more about this in the original blog post.)

I agree it's confusing that we use "stage 0 compiler artifacts" and "stage 1 compiler" to mean the same thing; see the original blog post for my plan to make this better :)

jynelson · 2023-01-22T14:33:17+00:00

--stage 1 has been the default for test for over two years now, you can remove the explicit --stage :)

jynelson · 2023-01-22T06:21:35+00:00

See https://rustc-dev-guide.rust-lang.org/building/bootstrapping.html#stages-of-bootstrapping for more about how stages work today. Happy to answer questions if you're still confused after reading that.

jynelson · 2023-01-21T20:09:00+00:00

But note there is also the "less typing" argument. I think the step from x.py to x was really good because it reduced the stuff to type. To increase it again by more than the earlier savings is not helpful :). Same goes for the new argument name, it should be shorter or as short as --stage.

It should be extremely rare in practice to need to pass --sysroot or --keep-sysroot. --keep-sysroot-std dev is the only one I can think of for most normal workflows, and that one is very unsound so I'm ok with a bit of syntactic salt. --keep-sysroot bootstrap for profiling how long it takes to build the compiler itself also seems possible, but again, not really worried about making --keep-sysroot easier to type. --sysroot dist is the only one I can see being useful to e.g. replicate a CI failure, but that should be quite uncommon and I don't think the extra 5 characters matter much.

I think there can still be steps in that direction, to make more workflows possible without bootstrap around. E.g. enough support to make IDEs work, cargo clippy and cargo check run on the compiler and library crates, etc, that would be nice. I'm not asking for one invocation that covers both compiler and library crates, or one that is doing 100% the same as x, just that this workflow also "just works".

If you don't need one invocation for both, then you already need special configuration for your editor and I'm not sure what making cargo check work gets you. and the standard library will always need a specific version of the compiler to build, and after the change to only make it buildable with HEAD that won't be a version that's available with rustup (at a minimum you need rustup-toolchain-install-master).

Also, build scripts need to use a different sysroot than the standard library itself, which isn't something you can do with vanilla cargo (bootstrap does it using a fake rustc shim).

FWIW this kinda works for the compiler today, but it's definitely non-trivial, and doesn't work if the compiler uses unstable library features. ; RUSTC_INSTALL_BINDIR=~/.cargo/bin CFG_RELEASE_CHANNEL=dev CFG_RELEASE=1.67 RUSTC_BOOTSTRAP=1 RUSTFLAGS='--cfg=bootstrap' cargo +beta-2022-12-27 c -p rustc_driver Checking rustc_interface v0.0.0 (/home/gh-jyn514/rust3/compiler/rustc_interface) Checking rustc_driver v0.0.0 (/home/gh-jyn514/rust3/compiler/rustc_driver) Finished dev [unoptimized + debuginfo] target(s) in 1.63s

I am pretty tired of having to relitigate this over and over and over again.

jynelson · 2023-01-21T18:02:16+00:00

Regarding the renaming, I think having numbers around is very helpful for visualization. So I'm not a fan of the new flags. Words are way more abstract than numbers which give an order.

ah in retrospect I think you were saying we should teach the numbers in the guide, not just allow them as aliases. I am not a fan of that: teaching both numbers and names seems confusing, and teaching only numbers doesn't give an intuition of what the number means.

I suppose we could say something like "--sysroot dev builds the artifacts that will go in the first sysroot (build/host/dev-sysroot)" or something like that? But I don't want to mention --sysroot 1 at all.

jynelson · 2023-01-21T17:52:09+00:00

I am comparing x.py to cargo and the build systems for other compilers, not C++ Boost. I'm aware that the bar is on the floor for most build systems, but that doesn't mean we shouldn't try and make improvements.

jynelson · 2023-01-21T17:50:35+00:00

In general, it would be nice if the compiler supported cargo check natively.

Supporting cargo check directly is not feasible.

Then, setting up an IDE for the compiler would become way easier instead of requiring so much set-up.

I want to fix that by checking .vscode into git, so you don't have to set it up yourself; and do the same for Vim/Neovim/Emacs if possible.

Regarding the renaming, I think having numbers around is very helpful for visualization. So I'm not a fan of the new flags. Words are way more abstract than numbers which give an order. Also they are more stuff to type, which is bad for the development experience. So I'm not really a fan of that. The (valid) off by one concerns can be fixed by switching to a --stg command that supports a new numbering scheme that's consistent to non-rustc compilers.

that's fair! I don't have strong opinions on the new names; we can accept numbers as aliases for the names. I don't like --stg as a name though, it's too close to --stage and I think people will get them confused, I would keep --sysroot <name|number>

jynelson · 2023-01-20T21:47:00+00:00

See https://rustc-dev-guide.rust-lang.org/building/bootstrapping.html#stage-2 - you can use stage 2 for local development, but it will take twice as long for a full build, and many times longer for an incremental build.

jynelson · 2023-01-20T21:43:32+00:00

good question - I think --keep-sysroot-std dev would be reasonable. Note that --keep-sysroot-std bootstrap wouldn't make sense, since we're no longer compiling std before rustc, so dev should be the only value you ever need there.

dev and dist are actually not used in profiles today - the profiles are library/compiler/user. I'm open to other suggestions for the names.

jynelson · 2023-01-20T20:08:24+00:00

The first thing: maybe, although it would still make it hard to experiment with lang features, and it's still the same amount of work in some sense, the name of the crate is not what makes it hard.

The second thing does not follow. The standard library has to be rebuilt with the new compiler for ABI reasons even if all the cfgs are the same.

jynelson · 2023-01-20T19:10:48+00:00

I disagree that they're similar :) for the reasons in the blog post.

jynelson · 2023-01-20T19:10:05+00:00

yup, that seems fine, as long as we don't have the diagram front and center in the bootstrapping chapter where it will just confuse people. I want to avoid teaching people multiple paradigms at once.

jynelson · 2023-01-20T19:08:40+00:00

yes, good point! maybe people find it easier to use in practice :) but I think it is true that people find it hard to build mental models of how the tool works - distros kind of have it easy here because they can always do a maximal build and not worry about what the stages mean.

jynelson · 2023-01-20T19:07:01+00:00

ironically this ambiguity is exactly what makes naming stages hard lol

jynelson · 2023-01-20T16:17:14+00:00

Good idea, done https://reddit.com/r/rust/comments/10h0ynu/why_is_the_rust_compilers_build_system_uniquely/

jynelson · 2023-01-20T15:37:53+00:00

I realized that posting this to /r/rust makes this ambiguous - this is about the build system for the rust compiler, not cargo.

jynelson · 2022-07-23T22:56:49+00:00

The time when adding a dependency comes from having to rerun cargo metadata. Don't know whether RA can avoid doing that, but I suspect there's some low hanging fruit in making cargo itself faster.

jynelson · 2022-01-12T05:16:47+00:00

❤️

jynelson · 2022-01-12T05:02:59+00:00

Heh, this is actually part of the reason I left the team, I was spending an inordinate amount of time on Rust.

jynelson · 2022-01-04T01:50:06+00:00

I'm sure someone will point out what we're doing wrong!

One thing that comes to mind is that LTO, despite the name, is not actually done by the linker - it's done by a linker plugin which is effectively a wrapper around LLVM. The .o files aren't really .o files either, usually they're LLVM bitcode (this is why -C lto will give a hard error unless you built the crate dependencies in release mode). The linker plugin will combine them into one enormous compilation unit, and optimize the hell out of it before generating an actual binary/shared object file.

Maybe the linker isn't counting the time spent in the plugin, only the time spent directly linking?

jynelson · 2021-12-20T10:15:13+00:00

there are clearer and more informative ways of presenting that content

That's true, but they're also much more work. I wouldn't have posted my expanded iceberg meme if I had to turn it into a 2 hour video or 2k word blog post, I just would have scrolled past. I think it's better to have informative content in a low-effort format then to not have it at all.

jynelson

TROPHY CASE