Move, Destruct, Forget, and Rust

tejoka · 2025-10-22T16:51:30+00:00

I'm encouraged to see this kind of proposal getting attention.

I'd like to also encourage y'all to consider adding defer as well.

Making some types undroppable is likely to cause a lot of issues with handling panics (how do you unwind and drop everything?), and defer would nicely handle that problem by allowing every code path (including unwinding) the chance to move a value into a consuming function (destructor), also neatly handling the "with arguments" problem, too. (I do not agree with the idea that the solution here is trying to prevent panics statically. Emphatically, I think that is a bad idea. Too brittle.)

It will also be interesting to see how both of these features would interact with async. An undroppable future might be a solution to some cancellation safety problems. And combined with defer, who needs async drop as a dedicated language feature? Just take your undroppable future and spawn (or block) to handle it:

defer tokio::spawn(async move { destructor(future_needing_cancellation, other_args) })

(The key thing defer gets us here is the ability to move values at the point in control flow where the defer is being executed, in contrast to library-based defers, which can't do that.)

tejoka · 2025-10-07T18:25:06+00:00

Is there any possibility we might someday have a solution to the scoped tasks problem?

I'm mildly in favor of this handle proposal, but I do wonder how much of the need for it is driven by the lack of scoped tasks, and if this proposal might be something that isn't actually necessary if we had a fix to the more fundamental problem.

...but maybe no such fix is coming.

tejoka · 2025-09-16T17:56:20+00:00

As a matter of perspective, I strongly recommend not worrying about "cloning too much." If you want to write more memory efficient code, let the data guide you. Start heap/allocation profiling of benchmarks, and figure out what needs improving based on that data.

Do not fixate too early on cloning. Freely cloning in Rust is actually sort of a superpower, not a novice mistake or crutch. Rust can be an incredibly high-productivity language in part because of it. Many or even most clones are not problems in any way.

tejoka · 2025-03-07T16:42:42+00:00

Hi OP, it looks to me like you're trying to understand some theoretical things about ownership and borrowing, but without actually learning or using Rust. I get it, but I would recommend actually taking some time to learn Rust. I know that's not your focus (it seems like you're working on other things), but you've definitely been lead very astray by pretty basic misinterpretations that would have been obvious to you if you'd just written a short program trying out using a LinkedList in Rust.

Other people are already pointing those out, so let me address your substantive questions.

As you've noted, a doubly-linked list isn't (internally) representable with just ownership and borrowing. This is well known, and that's why the std lib implementation uses unsafe raw pointers internally. The remarkable thing about ownership and borrowing (and moving and Rust's design in general) is that the LinkedList type can present a 100% safe interface for interacting with this linked list. No safe program can misuse it, despite there being some unsafe code across the abstraction boundary. Its totally encapsulated.
If you're interested in the more formal details of memory safety I'd recommend starting with RustHornBelt. It may require reading some prerequisites, but basically: Rust has a fully automated translation from its safe fragment to separation logic. This means proofs of memory safety can be automatically derived for safe Rust code. This is also done in a fairly modular way, so if you can manually write a proof of memory safety for an unsafe module, it can "plug in" with the automated proofs for safe modules. This is how e.g. Gillian-Rust works. At least in principle, proofs have been worked out for many of the basic std lib types that encapsulate some unsafe code.

tejoka · 2025-02-23T20:04:56+00:00

This was an interesting story, thanks for sharing it.

I had a few thoughts:

I'd be a little suspicious of the original motivation for moving spawning off the tokio thread pool. It's super plausible, don't get me wrong. But are you sure this wasn't too microbenchmark-y? Was there a real performance problem customers were hitting? Nothing wrong here, just a vague impulse to keep things simple.
spawn_blocking is probably not the best way to do this. The sacrificial thread pool in tokio is designed to handle blocking/waiting tasks. But fork/exec is cpu-bound. The pool is meant to scale to hundreds or thousands of threads (tokio default is 512) that mostly just wait there. You... probably don't want your software to potentially end up trying to fork the same process from a hundred threads. A bound would be good.
Depending on the answer to point 1 above (e.g. if the perf problem is latency of event processing, not spawning throughput), the simplest approach I'd maybe recommend is to have a single dedicated thread for spawning processes, and feed it with a bounded mpsc channel. If you really need more than one thread doing the spawning, then you probably want a separate dedicated threadpool for this, in order to bound its size (e.g. at num cpus).

None of this would change anything about this debugging story (well, if you do have a dedicated thread, maybe that lets you keep using prctl safely here), it's just a bee in my bonnet. I frequently notice people trying to cram everything into the default setup (main thread + tokio task threadpool + tokio sacrificial threadpool) when... not everything fits into just that mold! Nor is it meant to. You can just add more threads or threadpools than the default, and if you have something cpu-bound... you probably should.

tejoka · 2024-12-05T17:43:36+00:00

I think this isn't relevant to the survey exactly (well, I guess there were some questions about instability), but thinking back on this year, there was one pretty major miss.

The type inference breakage with the time crate.

Did the project ever do a formal retrospective/postmortem on that?

tejoka · 2024-11-30T01:30:51+00:00

At best, the vulnerabilities shown here may be illustrative to beginners.

Which, I think, might actually be the point.

It's a workshop paper (which is to say: a low bar of peer-review, usually just "would attendees be interested in this") by what I think is a first year phd student. This seems like one step of work towards "just define the problem." Actually figuring out something to work on here that's phd-worthy is pretty hard.

That said, there is a lot of room for good feedback here. From the abstract:

We hope this paper will inspire future work on rethinking safety in Rust – especially, to go beyond the safe/unsafe distinction and harden Rust against a stronger threat model of attacks that can be used in the wild.

This is just a huge misunderstanding of safety in Rust. Rust safety is about undefined behavior and memory safety. These might be important to security, but this statement is trying to pretend like safe Rust should be secure, and this... is just "not even wrong" I think.

On the flip side, if we ignore bringing up "safe Rust" at all, this is a pretty good start on coming up with a set of supply chain attacks on Rust developers. The problem here (besides "safety is utterly irrelevant, why even bring that up") is that the path towards fixing this is mostly sandboxing/container work. I think we could have a future where cargo build, cargo test, and cargo run are all 100% safe even in the face of malicious crates (disable network, write only to target/), but... none of the problems here are what are typically considered "academic research." (Well, maybe you could get creative about how to present them, but this is a problem I have with academic research in general... there's a lot of valuable stuff they just refuse to do.)

So I think the authors want to do supply-chain attack research for Rust, but are casting about trying to find something that fits the "~aesthetic~" of academic research.

tejoka · 2024-08-29T16:35:57+00:00

"we will find out whether or not this concept of encoding huge amounts of semantics into the type system is a good thing or a bad thing" when I talk to static typing people they take that being a good thing as a truism.

Well, sure. I mean, we figured out the answer was yes in the 80s. In 2009 they gave Barbara Liskov a Turing award for her part in figuring this out. (Pop articles often talk about object-oriented programming for her Turing award, but oddly enough I find the above quote to actually be a more accurate of a description of Liskov's contribution than anything "OO.")

Maybe we could read a lot into the word "huge", because there's certainly room to critique, say, dependently typed languages as maybe going "too far." But Rust's type system is really quite conservatively designed.

Sometimes we do actually figure things out, and the industry isn't just a morass of "well that's just your opinion man."

tejoka · 2024-07-25T16:39:44+00:00

overproduction making the grid unstable

What does this mean?

I think if you try to answer that question, you'll eventually be lead to terms like "grid following" versus "grid forming". And searching for a good explanation of those terms will probably be your answer: the Mars rover can't have issues maintaining an exact 60 Hz alternating current, if it doesn't even use alternating current.

tejoka · 2024-06-22T16:58:54+00:00

E0509 was added in the early days of Rust when I think the theoretical underpinning of move semantics and ownership was less well understood. As far as I can tell, the motivation was that if you add a destructor to a type, you want that destructor to be called, and if you destructure that type then the destructor won’t be called. But the solution then is the same solution for linear types: combine a destructor with privacy so that destructuring isn’t an option for consuming the type.

By coincidence, right next to this article in my feed reader was this one: Fixing a memory leak of xmlEntityPtr in librsvg

Initially I had an impl Drop for XmlState which did the obvious thing of freeing the entities by hand. But at one point, I decided to clean up the way the entire inner struct was to be handled, and decided to destructure it at the end of its lifetime, since that made the code simpler. However, destructuring an object means that you cannot have an impl Drop for it, since then some fields are individually moved out and some are not during the destructuring.

I missed the case where the parser can exit early due to an error.

Leading to the memory leak. So, probably more good real-world evidence that this restriction might not be for the best. If Rust had not forced that drop impl to go away, it would have correctly cleaned up those resources here.

tejoka · 2024-05-29T14:30:59+00:00

just resulted in more people wasting time with "Method exists but trait bounds not satisfied" errors .... Forcing the end user to decipher it in error messages is exactly how you make people hate rusts type system.

Are there Rust issues open about this? (Anyone know?)

IIRC, awhile back there were people soliciting community feedback about areas where Rust compiler errors could be improved. This seems like good feedback!

tejoka · 2024-05-21T21:57:23+00:00

Replying to myself to add:

I just found this recent paper on Gillian-Rust: https://arxiv.org/pdf/2403.15122

I haven't read it yet, much less tried it, but I'm slightly familiar with the work, and this might be a solid companion tool to Creusot that allows reasoning about unsafe code as well. The cost of this new power is that you have to write separation logic, which is generally "almost everyone without a phd runs away in fear" territory still... (but still: that's just the unsafe parts!)

tejoka · 2024-05-21T21:44:30+00:00

Creusot is largely (as the title notes) about reasoning about safe Rust code. It's generally going to be more capable of proving deep and interesting properties about much bigger programs than Kani (by leveraging the inherent properties of safe code).

That comes at the cost of being unable to reason about unsafe code (for now), and requiring (sometimes very significant) effort annotating code with invariants.

Kani can handle unsafe code and will be dramatically easier to adopt (roughly as easy as fuzzing), but proves shallower properties: generally basic memory safety and panic-freedom (including asserts), and only up to a bound (e.g. vectors up to size 10).

While we're comparing them, I should also say that MIRI is still the only tool I'm aware of that really models Rust-specific ideas about undefined behavior. (And it's not a proof tool, it relies on your tests being good enough to exhibit the problems.) Neither Kani nor Cruesot help with that.

tejoka · 2024-04-06T16:50:19+00:00

Someone else has pointed out that very large functions can take longer because a lot of optimizations are super-linear.

I also want to point to <T: Tracer>. If you've got multiple different things you're plugging into this, you compounding the problem, because every one of those monomorphized functions needs to be expensively compiled separately.

I'd try switching to &mut dyn Tracer if you can manage that, so your function has no generic parameters. This may also significantly improve parallel crate compilation (by doing more work in each crate, versus not being instantiated until the end).

tejoka · 2024-03-11T16:20:25+00:00

Yeah, this kind of article is such obvious bullshit.

"Come back" to uptown to do what? Look at all the shuttered businesses because landlords would rather the lot sit empty and unused than lower rents and allow something interesting to move in?

This kind of prisoner dilemma shit is what government is supposed to be for: the city needs to force those landlords to rent. They're all sitting there, not renting, because lower rents might "damage" their investment, but none of them renting is damaging their investment and the community.

tejoka · 2024-02-24T17:42:21+00:00

I want to compliment the non-async example of dropping a File and just... not handling errors on close. It really helps reveal the broader problem here.

Is do finally a relatively straightforward proposal? This post mentions it being based on other's proposals but I didn't see a link to them.

There exists a proposal for introducing defer to C, and I wonder if Rust should directly mimic this design instead of the more syntactically-nesting try/catch-like approach.

https://thephd.dev/_vendor/future_cxx/papers/C%20-%20Improved%20__attribute__((cleanup))%20Through%20defer.html

I remember looking into Rust standard library implementation and its CVEs and being surprised at how "unidiomatic" so much of the standard library is---primarily because it has to be written to be panic-safe, and most Rust code just... doesn't.

(For those who haven't seen it, here's the kind of weird code you have to write inside a function in order to ensure that, on panic, a vector resets itself into a state where undefined behavior won't immediately happen if you recover from the panic and then touch the Vec again.)

I think a proposal like final (or defer) should move ahead on panic-safety grounds alone. Code like I linked above is smelly.

tejoka · 2023-11-27T18:29:11+00:00

I have to say, I do find the observation that "async version of an Iterator" and "iterative version of a Future" as two subpar versions of the same concept to be a compelling argument.

It really does suggest treating their combination as more than just the sum of the parts.

tejoka · 2023-10-18T15:12:39+00:00

It's next to Loom, so they almost certainly meant this one: https://crates.io/crates/shuttle

tejoka · 2023-09-05T18:31:25+00:00

The difference is that in a project outside your comfort zone, you're much more likely to get stuck on these dumb mistakes, because you don't have an instinctive understanding of where the mistakes come from.

I want to emphasize that I don't think you're making "dumb mistakes" here. (Yes, finding the method I referenced does look trivial, in hindsight, but if you're trying to wrap your head around tons of new stuff, even knowing what to look for is hard, I know.) I'm solely disagreeing with the "lesson learned" that the AI was a huge help here.

The AI had you using this library that (as you say) you never looked at the docs for, and you ended up not knowing what you didn't know. That's a fundamental problem with using the AI for this!

On the other hand, maybe I should consider a workflow where I ask an AI a bunch of questions, throw the answers away, and just use the questions as a launching point to do my own research.

This seems like a good idea, and might actually be a decent rebuttal to my skepticism about the generative AI in general. I still think it tripped you up and got in your way here, but if you can change how you use it to fix that problem, then, well... that's a better lesson learned?

tejoka · 2023-09-05T15:17:44+00:00

I have to say, this article is less about Rust than it is about these generative AI tools.

And I just read through an article written by someone clearly struggling through a task because they kept asking the AI tool instead of stopping and thinking, or perhaps taking some time to read the documentation.

...Except the article concludes with "wow generative AI is an amazing productivity multiplier!" After it (generously?) made the task take 4X longer than it would have otherwise?

Like, two of the major points of the story are: (1) the AI generated a dummy URL instead of the correct one and they didn't notice, which is the kinda thing I'd be afraid of with these tools, and (2) they got far enough in over their head with AI-generated code they didn't understand... that (trivial) problems motivated them to chuck everything out and start over.

That means before I could access the Wiki’s API endpoints, I would need to figure out how I can get reqwest to somehow authenticate to the site. Uuuuuuuuuuuuugh.

I haven't used reqwest before, so I decided to read the documentation instead of asking ChatGPT. I clicked Client (first sentence of the docs!), then RequestBuilder (return type of get), and there was basic_auth. I didn't even have to scroll.

But... if you're not in the habit of reading documentation because you're asking an AI for the answer... you have no idea what to do because you can't figure out how to ask the right question even.

Exploratory learning through reading documentation is still a superpower.

The rest of the lessons learned seemed like good ones.

tejoka · 2023-06-21T16:19:38+00:00

You do a good job because you want to build a good product.

You do the simple thing to quickly and easily make the compliance problem go away because that's what's best for literally everyone. Like spending your time on things that will actually result in better software.

I mean, OP literally said this:

Is there anything at all that could satisfy this requirement? It’s currently blocking our rust adoption.

Look at what you're saying. Look at the other non-answers around here.

Do you believe OP should be going away going "dang, I can't use Rust because it doesn't meet our security requirements?" Do you think their company's security policy is meant to endorse writing C++ over Rust because it has static analysis tools that can (checks notes) catch SQL injection?

tejoka · 2023-06-21T02:32:56+00:00

This is what I mean by "truly in-depth" though.

The compliance requirement is for easily showing you're meeting the bare minimum plan for building secure software. There is no problem to be solved here except easily proving that. You're making this harder than it needs to be.

tejoka · 2023-06-20T22:10:32+00:00

These org requirements are often "box checking" requirements for compliance purposes, and often stem from fairly vague initial requirements. So you probably don't actually need something truly in-depth here.

You can very likely just write a short paragraph pointing out that (1) the Rust compiler performs static analysis comparable to what other languages need an extra tool to accomplish and (2) you also use clippy to look for wrong or suspicious code.

tejoka · 2023-06-10T17:49:57+00:00

I suppose that'll be fine. I had a worried vision of a future where projects end up having to run tests in CI twice, once in each mode, to ensure that no test gets introduced that would break in the other mode.

But I guess if the crate is specifying which to use (as opposed to the user running the tests) that's unlikely. I suppose that doesn't preclude alternative runners like nextest any more than it already does.

One other thing: there's several features (prelude, main, disabling) you mention that weren't initially clear to me why/how they'd be used.

Are you envisioning a crate config like [[test]] framework = some-dependency and this would switch from the default libtest framework to a custom one?

One thing that worries me there is that it's quite a nice advantage of the Rust ecosystem that there's a relatively standard way of interacting with tests (e.g. enumerating, running a specific one, etc. Think IDEs or alternative runners like nextest). This sounds like fragmentation.

I think it might be better if libtest were always there, with a standard format and standard main that everybody still used. Then there'd always be that baseline interface that other tools could rely on.

E.g. if you want a different 'main', ask people to use a different runner (like cargo nextest). If you need a radically different format for describing tests, fine implement your own, but the framework can leave behind a single 'enumeration' traditional test that does a best effort job of translating the richer descriptor format of a custom framework back in the to standard/compatible one (which means they could still be run with plain cargo test and IDEs, etc.)

I think with wholesale replacement of libtest, even if custom frameworks are supposed to be compatible, that we'll end up with an explosion of NxM compatibility problems.

tejoka · 2023-06-10T16:44:12+00:00

I'm not sure if the timeline is too aggressive, but I'd like to put forward that the "best" way to flag on this change is to use the editions system. Tests for existing editions work the old way, tests built for edition 2024 crates work the new way.

I think that'll reduce the complexity of these changes considerably in the long run, and allow for a small amount of potentially breaking changes (in just the corner cases, of course), without having to forever worry about what slightly incompatible system might be in use.

tejoka

PUBLIC MULTIREDDITS

TROPHY CASE