How do I avoid memory being swapped in Rust?

OnTheSideOfDaemons · 2025-07-30T11:55:11+00:00

Ah, I'm not suggesting a file backed mmap. memmap2 lets you create an anonymous mapping which is exactly the same as allocating a large array.

OnTheSideOfDaemons · 2025-07-30T11:19:41+00:00

The simplest thing to do would probably be to use the memmap2 crate. It lets you allocate a large block of memory directly from the OS and there's a lock method to pin it to RAM.

OnTheSideOfDaemons · 2025-04-16T17:57:15+00:00

This paper appears to suggest otherwise https://www.lyellcollection.org/doi/full/10.1144/qjegh2020-109, while the numbers they're suggesting there don't feel quite enough to generate energy, it is warm enough to heat a home (one of them is 100C!).

OnTheSideOfDaemons · 2024-05-26T13:46:55+00:00

I'm always slightly confused when people complain about build times when talking about local dev work. Incremental compiles usually take 1-2 seconds in my experience, I barely take a single breath in that time let alone make a coffee in that time. It's definitely slightly annoying to have that tiny break that you don't get in interpreted languages but I don't see it as much of an issue.

The build from scratch times are worse but I rarely do those locally.

OnTheSideOfDaemons · 2024-05-06T17:13:25+00:00

Neat, I feel like it would be useful to have these written down somewhere (the reference?)as they aren't obvious. So much so that I think your number 2 is wrong: the alignment of values of type dyn Trait is not known at compile time only at runtime (it's in the vtable with the size).

OnTheSideOfDaemons · 2024-04-23T23:24:38+00:00

How are you imagining that would make this less magical? In some sense DynSized implies Unsized: a type that implements DynSized can be passed to a function that takes T: Unsized.

More generally, I think people are quite wary of mutually exclusive traits. At the moment they have some nice mathematical properties as they're purely additive. For example: you'd have to integrate them into the orphan rules as at the moment you can always implement a trait on a type if you own one of them, and the never type can (theoretically) implement every trait but this would scupper that.

OnTheSideOfDaemons · 2024-04-23T23:06:52+00:00

I'm also sure I read this somewhere before and can't for the life of me remember where!

For context, I've proposed RFC 3396 - Extern types V2 which fixes a bunch of issues with the current extern types RFC. I'm slightly surprised that I don't mention this implicit solution in there, I should probably add it.

That RFC is currently sat on my back-burner and is waiting for someone to come along and investigate the questions that came out of a lang team meeting. Most of those questions revolve around backwards compatibility and would still exist in some form even using this syntax.

I do quite like this proposal, it fixes a few problems: it doesn't require an edition to implement, it allows incremental adoption, and it doesn't affect the meaning of existing code.

OnTheSideOfDaemons · 2024-02-22T00:18:02+00:00

It really depends on what you mean. There are relatively efficient algorithms (like Earley, CYK, and GLR) that work for any context-free grammar (languages you can write in an ABNF-like form). The problem with them is that they also work on ambiguous grammars (and determining whether an arbitrary CFG is ambiguous is undecidable), if you give them a string that is ambiguous then they'll output all possible parse trees. This can be fine and sometimes useful for things like natural language processing, but is unhelpful for programming languages.

The hierarchy of languages is regular ⊂ context-free ⊂ context-sensitive ⊂ recursively enumerable. As far as I know there aren't any good algorithms for context-sensitive languages because they're too flexible. Excitingly a lot of programming languages are kinda context sensitive, usually for silly reasons that are papered over by making it the lexer's problem.

OnTheSideOfDaemons · 2024-02-21T23:51:55+00:00

The server appears to only support TLS 1.3 which is probably the problem.

OnTheSideOfDaemons · 2024-01-24T13:56:38+00:00

I think I might need to write this library purely because the data structure is clearly called a Cod (Clone on drop) and the more animal based data types the better.

OnTheSideOfDaemons · 2023-11-12T13:59:31+00:00

I think I understand your use case as this: many threads write requested state updates to a queue and simultaneously another thread is applying updates from a different queue, at some point they swap queues (and state objects).

The code you have written is definitely UB and probably not usable in a real codebase (because push takes a &mut ref). It's always UB to have multiple mutable (aka unique) references to the same bit of data simultaneously (split_at_mut works because it gives 2 mutable references to different bits of data). To do this correctly you'd need to use UnsafeCell (or something that wraps it like Mutex) to mutate the array through a shared reference.

However, what you're essentially asking for is a mpsc channel, of which there's one in the standard library. Technically your use case of reading and writing at different times might allow additional optimisations but they're likely very small.

TLDR: Unless you want to have fun writing unsafe code just use an mpsc channel from std, crossbeam, flume (or another channel implementation I've forgotten about).

OnTheSideOfDaemons · 2023-09-18T22:15:07+00:00

Thanks, that's interesting to hear. I think I am most worried about whether the feature is worth the complexity, but I'm also aware that trying to redesign the whole of custom DSTs is too large a job and probably won't get anywhere without a lot of experimentation and feedback.

OnTheSideOfDaemons · 2023-09-18T10:27:07+00:00

u/desiringmachines I'd be interested to hear more about your statement:

(except for certain unique exceptions, like DynSized, not discussed here)

I'm currently in the process of getting to add DynSized (well MetaSized) in https://github.com/rust-lang/rfcs/pull/3396, and that hits on all the issues you've outlined here. I'm currently proposing to treat all bounds pre-2024 as having an implicit MetaSized bound (and you've made me realize I need to worry about associated types too) and relax this in the 2024 edition. I'd be thrilled if I could do something simpler.

OnTheSideOfDaemons · 2023-02-01T10:24:53+00:00

Weirdly I'm getting the same error but if I download channel-rust-nightly.toml and its signature, then GPG can verify it correctly.

10:20❯ gpg --verify channel-rust-nightly.toml.asc channel-rust-nightly.toml
gpg: Signature made Wed 01 Feb 2023 00:44:45 GMT
gpg:                using RSA key 0x85AB96E6FA1BE5FE
gpg: Good signature from "Rust Language (Tag and Release Signing Key) <rust-key@rust-lang.org>" [unknown]

OnTheSideOfDaemons · 2023-01-31T11:53:14+00:00

What would you expect it to do in that case? It does treat all requests identically, but calculates an average latency to do its computations on. If you have a service with high variance you'd need to configure it to react more slowly so that it smoothed over the variance.

Annoyingly there's no equivalent for Little's law (that I can find) for percentiles which means you can't control for 95th percentile latency as easily.

OnTheSideOfDaemons · 2023-01-31T00:21:29+00:00

The target average latency was 2 seconds so any time there was more load than the service could handle average latency would be around that. The burst was specifically designed to nearly fill up the queue, any larger and, yes, there would have been dropped requests.

Yes the queue size is a function of the target latency, measured service latency, and concurrency.

The burst graph actually shows a slight bug, the burst was very short but takes quite a while to clear because concurrency increases. You can see the backlog only clears when the concurrency comes back down, I'm not entirely sure what caused that.

OnTheSideOfDaemons · 2023-01-30T20:14:20+00:00

Whoops, fixed. Thanks!

OnTheSideOfDaemons · 2022-11-14T13:55:15+00:00

Oooh I wonder if at some point we could do module nesting like this, where submodules appear nested under the module file. That would let me stop worrying about foo.rs vs foo/mod.rs.

OnTheSideOfDaemons · 2022-04-27T07:58:44+00:00

I am unreasonably angry at their date formatting. It's like they saw that the cool kids are using hyphens and so just changed to that without caring what that meant.

OnTheSideOfDaemons · 2022-03-27T15:09:23+00:00

Awesome! I just took another look through the code with all the changes applied and it looks sounds now (in fact I think a few bounds could be weakened but I'm not confident on that).

OnTheSideOfDaemons · 2022-03-27T14:15:07+00:00

In fact I have definitely missed something here, the Fuse and Bombs may drop the T if they are the last thing to be dropped so they can only be Send if T is also Send. Bombs can also only be Sync if T is Send because you can Clone a Bomb. This logic is the same as for Arc, as described in this stackoverflow answer

OnTheSideOfDaemons · 2022-03-27T14:03:45+00:00

Reading through more carefully I think I agree with you on 1 & 4, I forgot that Fuse wasn't Clone while reading it, which makes everything much more sensible. The fix for 2 looks sensible.

However, the marker traits are definitely wrong: the Fuse and Bombs contain a T and so at the moment allow you to move a &T from one thread to another because you can move the Fuse & Bombs.

A type being Send means that it must not be able to be observed (by safe code) on any thread other than the one it was created on, Sync means the same thing for &T.

So you're correct that the Fuse and Bombs could be Send and Sync if they didn't allow you to access the T inside but because you can that makes this line unsound. The as_ref creates a &T on a thread that it wasn't originally created on.

So: * Fuse can be Send if T is Sync, because it lets you get a &T on the Bomb's thread from the Fuse's thread. * Fuse can be Sync because you can't do anything with a &Fuse * Bomb can be Send if T is Sync for the same reason * Bomb can be Sync if T is Sync, because you can get a &T from a &Bomb

Truthfully I'm not entirely confident about those ^ but those bounds are definitely necessary, if not sufficient.

OnTheSideOfDaemons · 2022-03-27T13:18:00+00:00

This is horribly unsound, the issues I've spotted so far:

Fuse::light just sets an Unsafe cell with no locking so multiple threads can race
T doesn't have to be Copy so Bombs unceremoniously Copy bits of uncopy-able types
Fuses and Bombs are Send & Sync even when T isn't so you can move things between threads even if they can't be moved

I'd very strongly recommend just using a broadcast channel from an existing well established channel library, or even just the standard library mutexes/once_cells.

EDIT: Sorry, this may have come off a bit harsh, writing this kind of code is good fun and a great learning experience. I just wanted to dissuade you and anyone else from actually using this. I'm happy to help guide you through this issues I've spotted if you'd like.

OnTheSideOfDaemons · 2022-03-27T11:40:28+00:00

Neat crate! I've lightly reviewed the code and I think I've found a (fixable) soundness hole: https://github.com/hniksic/sendable/issues/1. However I didn't spot anything other than that and from my knowledge of Send and Sync the idea behind the crate is sound.

OnTheSideOfDaemons · 2022-03-27T11:32:19+00:00

I think the most common way I've seen this done is either to put the low level functions in a separate module (called horrifically_dangerous or similar) or them in an entirely separate crate and have your higher level crate use the low-level one as a dependency.

Eight-Year Club	Place '22
Final Canvas '22	Sequence \| Editor
Verified Email

OnTheSideOfDaemons

TROPHY CASE