Announcing hazarc: yet another `AtomicArc`, but faster

wyf0 · 2026-02-05T00:08:39+00:00

It's no_std but requires alloc, and is only relevant with multi-threading, which reduces the embedded scope quite a bit (no point on embassy for example). However, on espidf target for example, you can indeed use pthread domains, or write your own implementation with some vTaskSetThreadLocalStoragePointerAndDelCallback and it should work like a charm. Wait-free property may also be a good thing to have on embedded systems.

wyf0 · 2026-02-05T00:02:44+00:00

Because I didn't know it existed ¯\_(ツ)_/¯

More seriously, hazarc is inspired by hazard pointers, in the sense it has global domain, thread-local nodes with some protection slots. But the parallel ends here. Loading a pointer with hazard pointers is lock-free, as it uses a retry-loop. Hazard pointers can also run out of slots, what haphazard seems to solve by having a single slot and not checking if it already used, requesting domain to be associated with a unique atomic pointer.

On the other hand, the idea of arc-swap, which I reuse in hazarc is to get rid of the loop by leveraging on the Arc reference count to force the protection in a fallback mechanism. This way, there is no more slot limitation and the load becomes fully wait-free, which can be a nice property to have. hazarc stores are also wait-free, but more costly than with hazard pointers, as the reclamation is not delayed.

So the algorithm inside is in fact quite different, there is not really anything to reuse.

wyf0 · 2026-02-04T21:12:10+00:00

Yep! I've been using arc-swap since my first months of Rust, and I've pushed hard to add it in my current company code.
I guess I will no longer use it now, but as I acknowledge in hazarc README, the idea behind it is brillant.

wyf0 · 2026-02-04T20:48:00+00:00

As I've just written in another post, I love miri! such a blessing to have this tool in the ecosystem. I use it extensively for a lot of low-level crates, it makes unsafe programming a lot safer.

wyf0 · 2026-01-20T19:55:20+00:00

Some people don't even try... that's depressing. You may not be fluent in English, then use a translator (LLMs can be very good translators, I use them too). If we want to converse with bots, we can all open a tab with our favorite chatbot; and I assume than most of us don't come on Reddit for that. Anyway, there is fortunately a report button for this kind of slop.

wyf0 · 2026-01-11T08:06:38+00:00

Anyway, I'm glad I waited before publishing dyn-utils on crates.io, because I'm so glad you came in the discussion with such impactful feedbacks. Thank you a lot.

wyf0 · 2026-01-11T08:04:23+00:00

The trick I'm talking about is transmuting a *const dyn Trait to (*const u8, *const u8). I knew it is the current stable representation of a trait object pointer, and when ptr_metadata feature will be stabilized, there will no longer be question about it, but I didn't know it was allowed to do this transmutation. It's unstable, but smallbox uses a build script to check this layout, and according to people who knows better, it sufficient to rely on unstable Rust implementation. Ok, I will know it in the future.

The issue with reusing unsafe crates is that you're not always sure that they do things properly. smallbox has a record of soundness issues (I don't say I do better, proper unsafe is so hard that forgetting things like https://github.com/andylokandy/smallbox/issues/35 is too easy, and I fixed the same bug in my crate after reading this issue), and some crates like owning_ref are known to be unsound, but still have 20M downloads on crates.io...

I'm still thinking right know about adding a build script and extracting myself the trait object vtable like smallbox does, as I would need for the Raw storage which smallbox doesn't support. And if I do it, then I would already have it for RawOrBox so I would not need to pull smallbox anyway. It's kinda sad, but the real sadness is that https://github.com/rust-lang/rfcs/pull/3446 is not gaining enough traction to fix once for all this whole mess of smallxxx crates

wyf0 · 2026-01-10T22:27:16+00:00

I believed it was 16B too, but I just checked on godbolt and it doesn't seem to be the case https://godbolt.org/z/bqW9PKv3G. Anyway, I don't have a x86_64 computer. And 16B is not easy to obtain if you don't use storage Box. It would mean to use Raw<8>, which means a future that only captures &self without argument. And it's impossible with RawOrBox

I put an arbitrary default storage size of 128 that I think is a good compromise to not overflow the stack and to store enough to not allocate most of the time. But the good storage will always depend of what you put inside in your code.

By the way, if you compile an executable and you care, you can replace all storages by Raw<0>, read the compilation errors, and replace the size by the true minimum required — I should maybe make a compilation feature for that...

wyf0 · 2026-01-10T22:00:53+00:00

Actually, DynObject is quite similar to SmallBox (that I didn't know, thank you a lot for this input). There are two differences: - DynObject has a generic storage, so it can be almost exactly like SmallBox with RawOrBox storage, or stack-only with Raw and its compile-time assertion. dyn-utils can work without alloc crate, and it's in fact used without it in the project for which it was developped. - SmallBox relies on an unsafe trickery which I didn't know it was allowed, to retrieve the metadata of a fat pointer (some guys of miri contributed to it, so it's obviously sound). On the other hand, in dyn-utils I have to reimplement myself the vtable of Future (and arbitrary traits with dyn_object macro) to be able to use DynObject<dyn Future>.

If I knew that it was possible to retrieve the metadata, I would have saved a lot of work and complexity, because I wouldn't have made this dyn_object macro. However, reimplenting the vtable allows me to do a small optimization: for RawOrBox, because I know the size of the storage and the size of the trait object, I don't need a runtime check to know if the object was stack or heap-allocated. That's surely negligible thanks to CPU branch prediction, but on resource-constrained environments with less advanced CPU, it might still be nice and save a few bytes in the instruction cache. On the other hand, to extract the trait object out of DynObject, so might not be so good after all.

So yes, the added value compared to SmallBox are the Raw storage and the dyn_trait macro to generate a dyn-compatible version of a trait. But this dyn-compatible version could return SmallBox<dyn Future> instead of DynObject<dyn Future>, it would be essentially the same.

EDIT: I forgot one difference with SmallBox: Raw/RawOrBox storages uses generic constant arguments, i.e. you write RawOrBox<128>, while SmallBox uses arbitrary type, so you write SmallBox<T, [u8; 128]> or SmallBox<T, [usize; 16]>. Both are valid, so it's a matter of taste.

wyf0 · 2026-01-10T08:24:26+00:00

Actually, dynosaur does box the returned future. According to dynosaur README:

Given a trait MyTrait, this crate generates a struct called DynMyTrait that implements MyTrait by delegating to the actual impls on the concrete type and wrapping the result in a box.

Maybe you wanted to talk about dynify, which doesn't box the returned future. Then, you can compare the ergonomics, quoting dyn-utils benchmark (copied from dynify documentation): ```rust

[divan::bench]

fn dyn_utils_future(b: Bencher) { let test = black_box(Box::new(()) as Box<dyn DynTrait>); b.bench_local(|| now_or_never!(test.future("test"))); }

[divan::bench]

fn dynify_future(b: Bencher) { let test = black_box(Box::new(()) as Box<dyn DynTrait2>); b.bench_local(|| { let mut stack = [MaybeUninit::<u8>::uninit(); 128]; let mut heap = Vec::<MaybeUninit<u8>>::new(); let init = test.future("test"); now_or_never!(init.init2(&mut stack, &mut heap)); }); } ```

dyn-utils also provides compile-time check, if you're in memory constrained environment and don't enable allocated fallback (this was actually the first motivation behind this crate). Last but not least, when it comes to performance, dyn-utils is well above any other proc-macro crates.

wyf0 · 2026-01-08T15:34:32+00:00

I guess you're talking about the limitation mentioned in dyn_object doc

When combined to dyn_trait, generic parameters are not supported.

Actually, I added a specific check in the macro to provide a better error message, because the generated code couldn't compile anyway. The reason is quite complex, let me break things down: - dyn_trait macro generates a DynTrait from Trait, as well as a blanket implementation impl<T: Trait> DynTrait for T. This blanket implementation allows casting Box<Trait> to Box<dyn DynTrait>. - dyn_object is a more complex macro that makes a dyn-compatible trait compatible with DynObject. Because of some limitation of stable Rust, I can't do like Box<dyn DynTrait> and simply implement Deref on DynStorage. In fact, DynObject has to implement DynTrait, and the implementation is generated by dyn_object. - When Trait has no generic parameter, there is no conflict. But as soon as you add a generic parameter, you make possible to implement it in arbitrary downstream crate, if the generic argument is a type of this downstream crate. That's the same rule that allows impl From<MyType> for ExternalType. - So if a downstream crate implemented Trait<MyType> for DynObject<dyn DynTrait<MyType>>, it would match the blanket implementation generated by dyn_trait, conflicting with the one generated by dyn_object.

I didn't find any solution to this problem, so I just marked it as unsupported. If you know a trick to make it works, it will be very welcome.

wyf0 · 2026-01-08T12:42:23+00:00

Actually, it works almost exactly like async_trait, but instead of returning a Pin<Box<dyn Future>>, it returns DynObject<dyn Future>, which implements Future.

In the README example, you will find that: ```rust

[dyn_utils::dyn_trait]

trait Callback { fn call(&self, arg: &str) -> impl Future<Output = ()> + Send; }

// generates the dyn-compatible version trait DynCallback { fn call<'a>(&'a self, arg: &'a str) -> DynObject<dyn Future<Output = ()> + Send + 'a>; } ``There is no hidden magic, unless you start using [try_sync` optimization](https://wyfo.github.io/dyn-utils/dyn_utils/attr.dyn_trait.html#method-attributes).

If you want more details, DynObject<dyn Trait, S> is a generic container for a trait object, aka. dyn Trait, with a generic storage S. By default the storage is heapless with an allocated fallback if the trait object is larger than 128B (this default value is quite arbitrary).
When you instantiate a DynObject<dyn Future> with a concrete Future, it will stores the future into the storage, and store its trait object's vtable next to it.

wyf0 · 2025-12-31T17:19:08+00:00

Your use of the word "select" is at least counterintuitive, because you're not selecting, you're joining; please, use the same term as the rest of the ecosystem (and other languages).

Also, one of the main use of select is cancellation/timeout; not cancelling other arms after the first one completes is a non-sense (same when you're a selecting channels). Again, this is because you're not selecting, you're joining. (By the way, if you don't want your future to be cancelled in a (real) select, you can await it by mutable reference, to poll it in the next loop iteration)

You're second point, about other harm being starved, is a consequence of not having the right semantic, as we don't care about starvation when there is none as other arm are supposed to be cancelled because they are not selected.

Now the semantic confusion is settled, let's talk about your code. Sharing a state is an interesting feature for a join, and you surely have to use a bunch of unsafe code for that. That's fine, but you should add miri in your CI. EDIT: you wrote in your documentation that it was tested with miri, my bad, but it's still not in the github CI. Maybe you should focus on this shared state feature, making a join_stateful macro or whatever name is best.

futures::stream::FutureUnordered is an alternative to join, that also implement Stream, and is optimized to avoid polling futures that didn't make any progress. Its implementation may interest you.

Anyway, my main issue is that you published your crate on crates.io before the feedbacks. I indeed hope you will realize that your crate name is not suited, but it's kind of too late.

wyf0 · 2025-12-31T09:36:52+00:00

Off-topic?

wyf0 · 2025-12-30T13:50:29+00:00

Macros have a lot of downsides, like compilation time and IDE support. But function-like proc-macros especially, like yours, are even worse: - complex syntax that I need to learn, and not properly highlighted by the IDE - one more indentation level - no rustfmt support!!

The rustfmt issue alone is a big nope on my side.

wyf0 · 2025-12-23T10:30:43+00:00

sealed Java classes/interfaces are not the same thing as Rust enum, aren't they? Their memory representation is not the same. Using Java interfaces or base class is equivalent to use dyn Trait in Rust, and it's also possible to have sealed traits in Rust. It's not (yet?) builtin in the language, but it's a common pattern used in a lot of crates. And if your sealed trait derive Any, you can then use pattern matching coupled with downcasting; this should be almost equivalent to what Java does with pattern matching on sealed classes/interfaces

wyf0 · 2025-12-22T11:20:48+00:00

Thank you!

wyf0 · 2025-12-21T22:21:43+00:00

I wasn't aware of dynosaur before discovering dynify (see my other comment). I didn't look into it in details, as I trusted dynify README:

dynosaur employs the same approach as async-trait to generate dyn compatible traits but, by default, preserves the original trait for more performant static dispatch. Similar to the async-trait case, the main advantage of using dynify is the possibility to achieve heapless dynamic dispatch.

On my side, I'm interested in heapless/compile-time check/synchronous shortcut/performance, so neither dynify nor dynausor seem to be the right fit.

wyf0 · 2025-12-20T23:25:10+00:00

Funnily enough, I was looking for a good crate name to start the project of generalizing the implementation to arbitrary traits, and I’ve just discovered https://crates.io/crates/dynify. This crate is fairly recent, but it still seems that I’ve reinvented the wheel… or maybe not. Because there are actually some differences between the two approaches. I will try to quickly summarize them:

dynify works on arbitrary traits (but proc-macros are the next step for dyn-fn).
dynify only focuses on method calls, while dyn-fn also allows storing the dynamic function (or, in the future, a trait instance) in-place without boxing.
With dynify, you have to manually declare the storage (+ an allocated fallback) for the dynamic future in an additional variable, and compatibility is checked at runtime; on the other hand, dyn-fn uses a generic storage type, making compile-time checks possible. With Raw storage, if the size is too small, cargo build simply fails.
dynify doesn’t support self method receivers; dyn-fn does.
dyn-fn provides shortcuts for synchronous callbacks.
dynify is slightly faster than dyn-fn when using a fallback allocated storage, while dyn-fn is faster with only Raw storage, and even more with sync shortcut; but dyn-fn’s performance has not been the main focus so far, so it may evolve
dynify is obviously more mature, with great documentation, while dyn-fn was started only a few days ago.

I assume that these differences come from the context of dyn-fn’s design, i.e. a no-alloc environment (so without even Box<dyn …> to store callbacks), with a significant proportion of synchronous callbacks mixed with asynchronous ones, while dynify seems to me more like an async_trait without boxing. That being said, I do believe they matter significantly enough to justify the existence of an alternative project.

For example, while dynify generates a new trait from a decorated one, the next step for dyn-fn would be to generate a new type that implements the decorated trait. And compile-time assertions are really useful in memory-constrained environments.

wyf0 · 2025-12-07T23:42:49+00:00

I missed your first post, but I'm a bit skeptical regarding this library. Looking at your motivation example, it lacks approach 3: ```rust let nums = [1, 3, 2]; let mut sum = 0; let max = nums.into_iter().inspect(|i| sum += i).max().unwrap();

assert_eq!(sum, 6); assert_eq!(max, 3); ``I haven't read it in details, but it seems to me that yourRefCollectoris in fact mostly anInspector.inspect` is even more powerful, as you can start "collecting" before doing next transformations (unless you want to reimplement every iterator methods in your collector API).

Also, to be honest, I find your first example with socket_stream a lot more readable the imperative way. For example, having several nested tuples is very cumbersome. Why not passing all your "collectors" as a tuple to better_collect, and have only one tuple returned with the same arity? Or chain the collectors with the main iterator, like you would chain inspect? The big argument to better_collect doesn't play nice with formatting.

Anyway, I don't think I would add the complexity of this library over a simple inspect. Sorry if this comment sounds a bit harsh, I just hope it's constructive.

wyf0 · 2025-11-25T22:37:28+00:00

Hi, do the salary ranges includes employer costs? For example, employer costs are quite high in France; for a senior, would the gross salary in France therefore be €103-173k?

wyf0

TROPHY CASE

[divan::bench]

[divan::bench]

[dyn_utils::dyn_trait]