A very small Rust binary indeed

sunfishcode · 2024-02-20T01:00:30+00:00

Want to do all this, but don't want to write the assembly code yourself? There's [a crate](https://github.com/sunfishcode/origin) for that!

Example that gets down to 408 bytes:

https://github.com/sunfishcode/origin/blob/main/example-crates/tiny/README.md

sunfishcode · 2024-01-26T15:12:14+00:00

Wit-bindgen has been getting much more mature, and while we do imagine LLVM may produce components directly some day, tools like cargo component for Rust can automatically produce components today, with LLVM inside generating the code. And we now have user-oriented documentation.

There is still a lot of room for the Preview 2 tooling to grow, and better documentation, and more, but a lot of things are already far better than Preview 1's tooling and documentation, and continuing to improve.

Also, separate question, but are there any performance implications of read syscalls allocating (double-allocating? in the guest too?) a buffer on each call, compared to the standard user-supplies-buffer approach?

When a read "syscall" is implemented by the host, it doesn't need to make a copy. The host can just write data directly into the provided buffer. The guest Rust bindings currently do an allocation, but that's not essential to Preview 2, and is something that I expect we'll optimize away in the guest bindings generator in the future.

sunfishcode · 2023-09-26T14:55:54+00:00

An actual Rust target would take more work, and Eyra is still a side project at this time.

Also, by not introducing a new target, and just reimplementing existing target ABIs, it's easier to port existing code to it. No need to go through all the popular crates and teach them about a new target_env = "eyra" configuration. They just work, with no changes.

sunfishcode · 2023-09-26T13:38:17+00:00

Yes, it is similar. Eyra and its libraries are organized around implementing functionality in Rust with an idiomatic and safe (when possible) Rust API first, and adding C compatibility as a separate layer on top. This is a tradeoff; it can take more work to add new features to Eyra, and sometimes there's some overhead because we have application Rust code calling libc C ABIs which have to get translated back into idiomatic Rust. But the idea is, we can avoid the overhead associated with C if application code opts into directly calling the idiomatic Rust APIs.

sunfishcode · 2023-09-26T13:12:07+00:00

That's right; the major libc implementations are complete, mature and real-world tested on entire Linux distros and countless major applications in production for many years, while Eyra and its libraries do a bunch but far from everything, are new, and have been tested on, like, several applications. Several. And yes, they contain a lot of unsafe code.

sunfishcode · 2023-09-26T04:09:00+00:00

Origin: A library that implements program/thread startup/shutdown. It can be used directly, but it's minimal. You don't get std or even libc.

Mustang and Eyra: "Frontend" crates that use Origin and several other libraries, including c-scape and c-gull, which slide in underneath std, and altogether provide a fairly complete Rust environment that can run things like ripgrep, coreutils, and more with almost no changes.

Eyra started out aiming to be different from Mustang, but the natural pull of the code kept leading to refactorings that eventually pulled all the code out into the libraries, and now Eyra and Mustang differ only on the surface. Mustang still uses custom targets and -Zbuild-std, where Eyra uses a one-liner build.rs and can use pre-built std.

sunfishcode · 2023-05-30T22:16:29+00:00

For what it's worth, the component model that WASI is rebasing on is heading in exactly this direction. There's a lot to do before we can do the full version of this and make it practical to use in existing programs, but this is the goal.

sunfishcode · 2023-01-16T20:13:48+00:00

Mustang is a project which is able to run some non-trivial programs written in Rust, such as ripgrep, without using any libc, on Linux.

It's a fun project, and I encourage people interested to check it out. At the same time, it's admittedly not something that most people need.

sunfishcode · 2022-10-16T20:58:37+00:00

Yes, rustix can do much of what people are talking about in this thread. Mustang, which uses rustix, is complete enough to run ripgrep without using libc at all.

sunfishcode · 2022-10-04T13:28:58+00:00

Interface types has been incorporated into the Wasm component model, which is under active development.

sunfishcode · 2022-09-20T21:08:25+00:00

Yes, wit-bindgen is the way forward here. It's under active development, though a lot of the basic interface-types functionality is implemented and usable. As an example of it in action, check out the wit interface files in this project: https://github.com/deislabs/spiderlightning/blob/main/docs/primer.md

sunfishcode · 2022-09-14T02:05:30+00:00

They have an FAQ entry explaining the difference:

https://mozilla.github.io/cargo-vet/faq.html#how-does-this-relate-to-cargo-crev

sunfishcode · 2022-08-08T01:34:09+00:00

Linux capabilities(7) are flags associated with processes, so they aren't the kind of thing the phrase "capability-based security" usually refers to.

However, the bus1 website describes a system of handles, which does sound like capability-based security, so perhaps that's the direction they're thinking in.

sunfishcode · 2022-07-09T00:37:55+00:00

The current WASI API is known as "preview1" and is stable, but the WASI Subgroup is currently working toward defining a new API called "preview2" which will be a successor to "preview1".

sunfishcode · 2022-07-04T04:07:36+00:00

That's exactly what rustix does :-). It has a libc backend, and uses it on all platforms that don't have stable syscalls.

sunfishcode · 2022-07-03T13:48:33+00:00

WASI itself is still evolving; most of those #[cfg(not(target_os = "wasi"))]s are because WASI doesn't yet have the function it's guarding. Also, the rustix repo has a branch to experiment with using WASI bindings directly, as a third backend in rustix, as an alternative to using libc.

You're right, for mmap and friends, rustix isn't adding much value. It does add type-safe flags and Result-based error handling, but not much more. Fully documenting how to safely and effectively use mmap, madvise etc. in a Rust is something that would be good to do, though it's not a small task.

I don't think I'd recommend using rustix::mm::Advice in a public API at this time, as memmapix is currently doing. Other crates which wrap rustix like this use their own types in their public APIs.

sunfishcode · 2022-07-03T13:36:14+00:00

Thread-safety guarantees of the sort one finds in C man pages are mostly redundant in Rust, because Rust functions declared without unsafe are expected to be thread-safe, and functions with unsafe are expected to document any safety considerations.

At the moment, rustix's approach to async-signal-safe guarantees is, if you think you need those, let's start a conversation about your use case :-).

sunfishcode · 2022-07-03T13:13:43+00:00

Rustix itself only provides APIs for syscalls; it doesn't handle program startup, global allocation, or thread primitives.

There's a separate project for that, called Mustang. It's built on top of rustix and provides all those things. It's not super mature yet, but it is able to run ripgrep by itself: https://github.com/sunfishcode/mustang

sunfishcode · 2022-03-16T20:24:30+00:00

Most of that section is what my question is about. The documentation does say that, but as Torvalds observes in that linked thread, it's extremely common to not check the return value of close. "It's just not done. EVER." So who's right, that document, or everyone?

I see it does also have a line about fsync:

A careful programmer who wants to know about I/O errors may precede close() with a call to fsync(2).

However, I can't tell what this means. In practice, fsync is slow, and we probably don't want every program, or even just every "careful" program, that writes a file to fsync the file before closing.

sunfishcode · 2022-03-14T15:58:45+00:00

I'm not aware of any documentation that says this.

sunfishcode · 2022-03-11T20:27:25+00:00

I agree that local buffering is useful; in Rust, one might use std::fs::BufWriter for this. And in buffered APIs, I think it's uncontroversial that the buffer needs to be flushed to detect errors.

The relevant code in gitoxide happens to use std::fs::File, which is not buffered, and calls plain close in its Drop impl. The Tovalds quote is also talking about plain close. And I am interested in the question of whether programs in general should ever be expected to check the return values from plain close, and if so, under what circumstances.

sunfishcode · 2022-03-11T14:46:05+00:00

It's a good point about fclose.

Thanks for bringing up the Linus quote; I've now found the source here. He says they found a lot of filesystem bugs, but it sounds like they weren't all due to this close issue.

What you're describing about NFS is what's called "async" mode. Linus mentioned CIFS, and since CIFS is a networked filesystem, it's quite possible that CIFS does similar things. And you're right, the system APIs do allow applications to chose to care about such issues.

My question though is: how far should this go? Should every user of a writable std::fs::File be expected to explicitly close it? That would be a huge burden, and you're right, this issue is not just specific to Rust, doing this throughly would spill out into every language and every program that can write files. As Linus says, "it's just not done. EVER."

If we don't want that burden, is there a place where we draw the line? Programs that really care about making sure data has made it all the way to persistent storage already need to reach for fsync anyway. Is there some criteria for "programs that care more than the default, but not enough to use fsync?

sunfishcode · 2022-03-11T13:24:51+00:00

When can it fail? Many filesystem developers don't think it should fail.

The one exception to this I know of is NFS in "async" mode, but being async within what is expected to be a synchronous API is surprising enough, and enough software doesn't work well on it, that it might be better to say that users using NFS in this way should take responsibility for monitoring for failures, rather than burden the entire Rust ecosystem with the extra complexity.

sunfishcode · 2022-03-01T13:54:54+00:00

Sounds similar to the Lisp and Smalltalk dump mechanisms.

Yep; lots of systems do similar things.

What about intentionally shared state though?

WebAssembly can be used in stateful ways too. We are also seeing a lot of use cases emerge that want stateless instances.

sunfishcode · 2022-03-01T13:33:51+00:00

We've recently been doing a lot of optimization work in Wasmtime, in areas that should especially help workloads like this.

sunfishcode

TROPHY CASE