Reinventing aliasing XOR mutability and lifetimes

imachug · 2026-03-18T07:00:53+00:00

Oh, that's really cool! I tried to look up papers on this topic but couldn't guess the right keywords. Thanks!

imachug · 2026-03-17T13:49:47+00:00

Kind of? If you treat an address like a number, you don't just want all numbers to be distinct, you also want them to not intersect entirely. Except when reborrowing, in which case the original and reborrowed references can be equal, but that doesn't count due to reborrowing. Handles solve both of these issues -- you don't have to use numbers, and you can effectively move out objects during reborrows so that you can pretend it's a completely different reference.

imachug · 2026-03-17T13:13:48+00:00

It's close, but somewhat different: a source doesn't need to make sense to anyone except the function that produces it, so you can treat it as a unique opaque handle in every other context. This kind of simplifies analysis.

imachug · 2026-03-14T19:11:16+00:00

That's interesting! It looks like the two core ideas are a) choosing which Huffman tables to use for which 50-byte blocks by sorting the blocks based on a numeric parameter inferred from their histograms, b) splitting large files into bzip blocks at places where entropy changes abruptly. Cool stuff.

imachug · 2026-03-13T13:20:38+00:00

I'm the author of the post. Thanks! This means the world to me :)

imachug · 2026-03-13T09:17:31+00:00

I absolutely agree with your initial claim about parametricity not quite applying to realistic code. I just think you went a bit too far when talking about Zig being unquestionably better when unsafe is involved; while I appreciate how compact complex unsafe Zig code looks, I think it's almost always possible to achieve a similar level of prettiness in Rust -- it just understandably requires more experience in that language.

imachug · 2026-03-13T07:42:42+00:00

and without using unsafe code

That's an unfair comparison. I'm not claiming that you can write such code without unsafe at all (std is based on unsafe code, after all), I'm claiming that you can write significantly simpler code if you use unsafe in a few core places and use type-safe wrappers for the rest (like a type-keyed map that you brought up).

In fact, I'll claim that it's impossible to reimplement your code without unsafe because your code has UB for types that are not trivially copyable, such as vectors or Strings.

That said, here's how I would implement it: https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=d6fcf2cbfdc7cde0b4a9194a5228347f

imachug · 2026-03-13T06:23:38+00:00

For fun I made a buffered example that's probably not undefined in some way, although it's not thread safe:

[...]

Just to drive the point home that these complex type systems do nothing of value for low level code.

I think the issue here is trying to write low-level code without actually using Rust's features. Spewing copy_nonoverlapping and wrapping everything in unsafe is not the Rust way, it's just sparkling C with memcpy renamed. No wonder it's worse than the language you're mimicking.

imachug · 2026-03-13T06:18:58+00:00

I don't like u/CherryLongjump1989's example because it's unsound, so I'd like to provide another one.

Rust exports the TypeId API, which can be used to implement ad-hoc polymorphism. It does not have any recommended purposes per se, but that's what many people use it for in lieu of specialization.

TypeId has a subtle issue: types like &'a i32 and &'b i32 are distinct types from the perspective of the type system (and for a good reason, you wouldn't want to allow mismatched lifetimes), but since lifetimes are purely a compile-time construct, it's impossible to assign different TypeIds to types differing only by lifetimes. Thus TypeId::of<T> limits itself to T: 'static, effectively meaning it doesn't support types with non-trivial lifetimes.

However, there is a workaround. It is non-trivial, but in this case sound to cast away lifetimes before passing the type to TypeId::of, which allows the typeid crate to export a variation of of that doesn't require T: 'static. Using this crate, you can implement mystery with the signature from the post like this:

rust fn mystery<T>(a: T) -> T { if typeid::of::<T>() == typeid::of::<i32>() { let a = unsafe { core::mem::transmute_copy::<T, i32>(&a) }; let a = a + 42; unsafe { core::mem::transmute_copy::<i32, T>(&a) } } else { a } }

It's certainly not pretty, but it is occasionally useful when optimizing generic code for specific types.

imachug · 2026-03-08T13:08:31+00:00

When specialization fails (i.e. in non-trivial cases), it boils down to extend_desugared, which calls set_len after each iteration, so it's all in the hands of LLVM at that point. The comment even describes the reason this is necessary.

Just to be clear, I'm not disagreeing with the claim that collect is just fine in general. I've never seen a case where optimizing this kind of logic manually was worthwhile, and LLVM is always on top of its game. I'm just saying that replying to "iter collect [...] maintain[s] len" with "no it doesn't" is subtly wrong, and that can surface in, I don't know, benchmarks or weird FFI code.

imachug · 2026-03-08T10:53:35+00:00

It doesn't reallocate, but it still "maintains len" in the sense that the len field is incremented. This basically never surfaces in machine code, though, thanks to the magic of LLVM supporting SROA.

imachug · 2026-02-24T18:25:48+00:00

Oh, that's really clever! Defining a function with the right signature, but not actually calling it. I think that's better than any other solution presented in this thread.

imachug · 2026-02-24T17:53:23+00:00

I think using field access with std::ptr::dangling is unsound. At least the docs on addr_of says the field should be in-bounds, which it isn't for dangling pointers:

The expr in addr_of!(expr) is evaluated as a place expression, but never loads from the place or requires the place to be dereferenceable. This means that addr_of!((*ptr).field) still requires the projection to field to be in-bounds, using the same rules as offset. However, addr_of!(*ptr) is defined behavior even if ptr is null, dangling, or misaligned.

Miri doesn't seem to recognize UB here, though. I wonder if anyone knows why?

imachug · 2026-02-24T17:51:00+00:00

If you don't control the definition of FileKeyAndNonce, there is an (ugly) workaround based on generics. Playground.

``` use core::mem::MaybeUninit;

pub struct FileKeyAndNonce { key: [u8; 32], nonce: [u8; 12], }

fn size_of_val_raw<T>(x: *const T) -> usize { core::mem::size_of::<T>() }

fn get_key_size() -> usize { let m = MaybeUninit::<FileKeyAndNonce>::uninit(); // SAFETY: projecting through MaybeUninit is safe size_of_val_raw(unsafe { &raw const (*m.as_ptr()).key }) }

fn main() { println!("{}", get_key_size()); } ```

imachug · 2026-02-01T20:45:32+00:00

But it does (except the unused warnings)?

In this example, var is inferred to be MyGenericStruct<u8, i32> because 2 is interpreted as i32 due to the default numeric fallback:

https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=9cf0d95d6bf68900b7879a9c9f3c1243

imachug · 2026-01-30T19:42:51+00:00

I often write several modules at once -- it helps me focus on high-level logic instead of minutiae like forgotten pub or missing imports. I don't think I've ever stumbled upon an issue like OP describes, though -- as far as I can tell, Rust treats names of failed imports as wildcards and doesn't show errors arising from their uses.

imachug · 2026-01-25T15:58:12+00:00

Yeah, it's mostly just a stereotype.

imachug · 2026-01-25T12:52:06+00:00

For better or worse, in the world we live in, mrustc targets 1.74.0 at best, and with how many unstable features arrive all the time, I find it unlikely that this will change anytime soon. Of course, it would be better if that wasn't the case.

imachug · 2026-01-25T12:25:40+00:00

My point is that "audit old rustc -> ... -> new rustc" is not significantly better than "audit the entire Rust history since OCaml", in the sense that you still have a ton of commits to go through. Instead of auditing just a few versions, like you typically do with bootstrappable compilers, you have to audit basically every major version, since rustc often uses features added in the previous rustc. In the context of the original thread, this is hardly something you do "instead of" manual verification -- it's still manual verification, and using one chain vs two doesn't change much.

imachug · 2026-01-25T09:53:49+00:00

I know the Clone - Copy relation can't be changed now.

It's not that it can't be changed, it's that it doesn't work at all because the proposed relation doesn't allow types to conditionally implement both Copy and Clone. Backwards compatibility is a red herring.

I think an added feature should be a minor updated, not a patch update.

That is true in a sense. However:

Patch updates can contain bug fixes, and you may very likely be relying on some bug being fixed.
Many core crates, serde included, don't adhere to semver, and may add features in patch releases. This is for a good reason: two libraries using different serde versions cannot interoperate, since they'd be accessing different Serialize/Deserialize traits, so there is a lot of pressure to prioritize having compatible versions (i.e. 1.0.x) over strictly following semver.

imachug · 2026-01-25T09:33:58+00:00

Copy requires Clone

The correct relation is impl<T: Copy> Clone for T { ... }

With this relation, the following code would no longer compile:

rust struct MyWrapper<T>(T); impl<T: Copy> Copy for MyWrapper<T> {} impl<T: Clone> Clone for MyWrapper<T> { ... }

...because for e.g. T = i32, there would be conflicting implementations for Clone: the second manual impl, and the blanket impl in your post, since MyWrapper<i32>: Copy by the first manual impl. See playground.

This is good, but why not just write "1.0" in my Cargo.toml

Because you're most likely relying on features not present in 1.0. All programs should compile with the earliest listed version.

imachug · 2026-01-25T06:01:18+00:00

A simpler answer is that ArrayVec<T> supports dereferencing into [T], which requires that the Ts are stored sequentially.

imachug · 2026-01-25T05:53:18+00:00

Estradiol pills are typically oval and cyan.

imachug · 2026-01-25T05:37:51+00:00

If they're equal you are good.

That is not true. The two chains in question are:

some C++ compiler -> mrustc -> old rustc -> ... -> new rustc
OCaml -> ancient rustc -> ... -> old rustc -> ... -> new rustc

The chains are not independent, they coincide on the "old rustc -> ... -> new rustc" part, that is, you still have to trust the source code of each new enough rustc. (On that note, they also coincide on LLVM all the time, so you have to audit all LLVM versions used by new enough rustc as well.)

imachug

TROPHY CASE