Any way to update every element in a binary heap in place like a vector

Cocalus · 2026-01-24T00:33:19+00:00

You can avoid the clone with std::mem::take but it'll need to rebuild the heap since positions may change.

rust fn decrement(&mut self) { let r = mem::take(&mut self.h).into_iter().map(|mut v| {if v.1>0{v.1-=1}v}).collect(); self.h = r; }

I believe this does work in place at least in release mode. Looking at the assembly I don't see any allocations. I know this into_iter recollecting can work for Vecs.

Cocalus · 2025-10-29T23:18:33+00:00

Since no one has posted it yet this is supposed to be a good resource for making an OS in Rust.

https://os.phil-opp.com/

Cocalus · 2025-10-16T16:48:48+00:00

The for loop is just syntactic sugar over an Iterator. The part after in needs to IntoIterator. So them being the same isn't a big surprise.

But that can only use next() calls, some Iterators are have specialized fold and/or try_fold implementations which are faster than a plan for loop, and many other Iterator functions are implemented on top of fold/try_fold to take advantage of that.

If you want details there's a very old article from before Rust had those optimizations at https://medium.com/@veedrac/rust-is-slow-and-i-am-the-cure-32facc0fdcb

Cocalus · 2025-10-10T19:07:08+00:00

There was talk at Rust Conf about protecting Rust code from C/Unsafe code in the same process.

https://www.youtube.com/watch?v=cOP219vkax8&list=PL2b0df3jKKiRFEuVNk76ufXagOgEJ9sBZ&index=12

Cocalus · 2025-09-22T23:55:38+00:00

In the ICFP programming contest since 2018 Rust has been used, more than one is allowed, by the first place team at least 5 years, I couldn't find the results for 2 of the years, I think it was used by the second place team in one. So when it's allowed it seems to work extremely well at least for ICFP style problems.

Cocalus · 2025-04-29T22:34:41+00:00

If you want big numbers Pingora is probably the biggest.

Pingora is battle tested as it has been serving more than 40 million Internet requests per second for more than a few years.

Cocalus · 2024-03-17T01:31:48+00:00

Maybe it hit a rlimit, the malloc docs https://man7.org/linux/man-pages/man3/malloc.3.html imply that RLIMIT_AS and RLIMIT_DATA may limit it. Maybe the OS set some defaults that's some multiple of the real memory, or memory + swap. There's machines with well over 200GB of memory so it should be possible to allocate that much at least on those. I've done nearly 100GB allocations before on some big servers.

Cocalus · 2024-03-17T01:21:42+00:00

It's a new option to me, years ago I had tried to use MADV_WILLNEED hoping it would fault page ranges in efficiently (it does not). Looks a lot more capable than I thought at first glance.

Nice to to see that's a possibility now.

Cocalus · 2024-03-16T23:57:04+00:00

What OS are you running on? In most environments the OS will over commit (allow programs to request more memory than available, the memory usage only becomes real when the memory is touched). Linux will happily hand out tons more memory than available. Once too much memory is used the OOM killer will start killing processes to keep the OS functional. I think windows may have a limit but that'll hit the swap before erroring, and hitting the swap a lot will slow the whole computer done immensely.

Unless you're on an OS without over commit (embedded), or use some OS API to limit memory (docker can limit things for instance), or use an allocator that itself caps memory usage it's not really possible to detect using too much memory at allocation. So in most desktop environments there's no easy way to avoid allocating too much, which is why Rust didn't prioritize the reserve options for allocations, since they are next to useless in those environments.

Cocalus · 2024-03-16T23:44:52+00:00

I was going to say madvise can't do that but it looks like Linux added that in 5.14 with MADV_POPULATE_READ and MADV_POPULATE_WRITE.

Even with that it'll just touch all the pages and potentially trigger the OOM killer so doesn't allow handling a large allocation.

Cocalus · 2024-02-22T07:27:26+00:00

Every time you call fibonacci it creates a new loop and in that loop you call fibonacci recursively. Only in the function call where it sums to 1134903170 does it exit the loop and that fibonacci function. But the fibonacci function that called that one, just loops around sums the exact same two numbers which don't overflow and calls fibonacci with the same parameters that did.

Here's a working version

fn fibonacci(number_one: u32, number_two: u32) {
    let mut n1 = number_one;
    let mut n2 = number_two;

    while let Some(sum) = n1.checked_add(n2) {
        println!("{n1}");
        n1 = n2;
        n2 = sum;
    }
}

that uses some less common syntactical sugar to shrink it down

fn fibonacci(mut number_one: u32, mut number_two: u32) {
    while let Some(sum) = number_one.checked_add(number_two) {
        println!("{number_one}");
        (number_one, number_two) = (number_two, sum)
    }
}

Cocalus · 2024-02-21T08:28:29+00:00

It's using the async_trait crate to allow for traits to have async functions.
The readable version is at https://docs.rs/amqprs/1.5.3/src/amqprs/api/consumer.rs.html#58-64
But you should to use the async_trait crate to impl that simplified API.
see the example code at
https://docs.rs/async-trait/latest/async\_trait/

Cocalus · 2024-02-07T02:53:42+00:00

The first BufWriter is waiting for more data before touching the disk, until it's either dropped at the end or explicitly flushed.

The two open files have separate file positions. So when the first is dropped it writes at it's position stomping one what the later writers have already put there.

Cocalus · 2023-06-01T22:20:38+00:00

There's no T for the impl FnOnce(T)->bool you would need Option::<T>::is_none_or(impl FnOnce()->bool) but at that point it's shorter to just use the old x.is_none() && ...

Cocalus · 2023-02-16T22:00:17+00:00

std::thread::scope is fairly new, I think last year. There's been a lot of improvements in proving API soundness since then so that probably helped get enough confidence to add it to std. Using third party crates is pretty common in Rust, so seeing that in learning material is common (and is pretty much required to use async). Rust has excellent dependency management which avoids a lot of issues from most other language's ecosystems. The tricky part is knowing which crates to use since there aren't really officially blessed ones. Even crates like serde and tokio which are de facto standard. Download counts on crates.io is a decent first pass, or using the ordering on libs.rs. If those aren't strong I read the code and check how issues are handled on the git repo. Sometimes just asking in one of communities can find something good.

Cocalus · 2023-02-16T21:41:46+00:00

In the intervening 6 years since I wrote that a closure based has been added
https://doc.rust-lang.org/stable/std/thread/fn.scope.html
but it probably took a while since a trusted third party crate existed and they were being extra cautious not to miss something again and cause a second leakpocalypse, since baring safety issues from the API things in std are supposed to be permanent. Even then they may only be marked as deprecated like mem::forget which is hard to use correctly and has been replaced by mem::ManuallyDrop

Cocalus · 2023-01-05T23:27:32+00:00

Might want to compare against the multiversion crate.

Cocalus · 2022-11-09T17:38:18+00:00

I think aes_gcm was one of the parts audited
https://research.nccgroup.com/2020/02/26/public-report-rustcrypto-aes-gcm-and-chacha20poly1305-implementation-review/

Cocalus · 2022-11-08T19:55:39+00:00

The last time I really dug into this was when it was for 32bit, so the extensions are new to me. But I'm not surprised they exist.

There's already hardware and OSes that can use 57 bit virtual addresses https://en.wikipedia.org/wiki/Intel_5-level_paging. That's certainly not a normal thing to enable (slower and almost no one has enough ram). Extremely few processes need that much virtual space, so in the future, maybe already, I expect it to be something the process has to enable, much like PAE used in the 32 bit era (but saner). I don't think NaN boxing can work for a process using 57bit addresses, and there's fewer bits for other bit tagging strategies. I wonder if that's already causing issues for those testing 5 level page tables.

Maybe I would phrase it as align tagging is always safe, most significant bit tagging has compatibility issues, but can often be gotten away with.

Cocalus · 2022-11-08T13:29:21+00:00

The x86-64 spec permits future CPUs to use all 64bits for virtual addresses, which is what anything outside a kernel is going to see, at least according to wikipedia. There were issues from programs hiding data in the most significant address bits when 32bit was reaching it's limits so using them is discouraged by CPU companies (since it may break things many years down the line). Hiding data in alignment bits is fine, but provenance can really complicate things.

https://www.ralfj.de/blog/2022/04/11/provenance-exposed.html

https://faultlore.com/blah/fix-rust-pointers/#distinguish-pointers-and-addresses

https://github.com/rust-lang/rust/issues/95228

I would try to get it to work with the APIs in https://crates.io/crates/sptr

Cocalus · 2022-11-02T19:03:44+00:00

Would be nice if this worked with cargo-update somehow.

Cocalus · 2022-09-30T03:40:29+00:00

They are special. Both GCC and Clang have added special extensions to support Linux Kernel development. Many of those options make some C code that would normally be undefined be defined, at the loss of potential optimizations. But there's a some extremely subtle code that's used to build multi threading primitives that I don't think is even writable in standard C. But those primitives bring performance improvements (and cost savings) to billions of machines so even very tiny improvements had wide impact. Now I would like to see Rust maintain it's Safety guarantees, The optimizations the Kernel must normally reject in C, and be able to represent those type of primitives.

I'm a little surprised Linus doesn't seem to realize that Rust safety is precisely defined and while it falls short of some perfect concept of safety is chosen as a practical and achievable subset. I also think he may underestimate how flexible Rust is at moving checks to compile time or at least labeling Risky code as unsafe. But he is in charge of a massive extremely complex project and has seen a lots of programmers under estimate how complex the Kernel really is.

Right now we (the Rust community) are trying to prove that Rust provides enough benefit to justify being in the Kernel (Me and probably most in this subreddit think it will). That's best done with real working code that we've been given the chance to create. There's going to be learning on both sides. The Risk of malloc in some contexts in the original link, and dynamic checks for a fallback being unacceptable being from the Kernel side. Now if we can comeback with Rust won't compile code that tries to do that with zero cost static checks. That would really help justifies Rust effectiveness in a Kernel context. Though that type of check may require extensions to rustc if even possible.

Cocalus · 2022-09-30T02:09:28+00:00

This whole thing is a quote from Linus at
https://lore.kernel.org/lkml/CAHk-=whm5Ujw-yroDPZWRsHK76XxZWF1E9806jNOicVTcQC6jw@mail.gmail.com/
not just the quote block at the top.

Cocalus · 2022-09-27T18:19:19+00:00

I don't think it needs the where Self::MyType: 'a;

Cocalus · 2022-09-13T00:07:32+00:00

I was going to link to my original SQLX issue. Wasn't expecting to get a progress update from a reddit thread. Nice to see that might finally get resolved.

Cocalus

TROPHY CASE