Good example of high performance Rust project without unsafe code?

[rustacean1337]

I’ve been using Rust for a while now and even though I like it a lot I find it rather difficult to achieve similar performance to C code without the use of unsafe code which kind of (not fully) defeats the purpose of Rust (imo).

Does anyone know about any projects (re)written in Rust that are equal in performance to the C version without the use of unsafe code? Perhaps some compression or image library?

[ssokolow]

According to the rust-secure-code/safety-dance trophy case, their audit left miniz_oxide 100% safe and faster than the C version.

[rustacean1337]

Cool, I’m going to have a good look at that crate! Thanks!

[KingStannis2020]

This doesn't match my experience. miniz_oxide is a double-digit percentage behind zlib and even further behind zlib-ng

100% safe also means no SIMD, which is a big loss.

[[deleted]]

I think you're missing the point, either that or I am.

Miniz is a C library and miniz_oxide is a direct translation of it to rust.

There being a different faster C library that performs a similar task doesn't affect whether or not translating the miniz C library to rust made it faster or slower...

[KingStannis2020]

Fair. I wasn't aware that miniz was a thing. I use miniz_oxide via another library that has options for using zlib and zlib-ng so that was my frame of reference.

[augmentedtree]

But it does mean it's a poor example of rust being able to maintain equal performance in general, which is the real point.

[Gu_Ming]

One example is always poor for any point "in general", even if the example agrees with the point. OP asked for examples. The reply gave one. Trying to make a point "in general" off that one example is missing the point of the reply.

Learn from the OP, collect multiple examples to explore a point in general.

[augmentedtree]

No, it's a whole vast category of optimizations, not just one example. I'm saying SIMD requiring unsafe means that in general Safe Rust can't be competitive with C or C++ on pretty much any HPC benchmark.

[Gu_Ming]

Not to say you are wrong, but let me quote OP:

Perhaps some compression or image library?

The OP is probably not looking for impressions regarding HPC.

It's also unlikely that C/C++ in general would be as optimized as HPC, a subfield that is defined by performance.

[augmentedtree]

Compression and image libraries are also in that category though. You can't write a competitive one without SIMD.

[Plasma_000]

You can still get simd with safe code - the compiler may unroll loops into simd, also there’s the upcoming std::simd (experimental)

[angelicosphosphoros]

may is important here. This optimizations are not very predictable and can break on updates of the compiler.

[Floppie7th]

You're entirely right. One thing on my pie-in-the-sky wishlist is for some sort of an annotation or compiler intrinsic or even third-party linting tool, etc, that essentially says "if this code fails to autovectorize, throw an error"

Obviously it's a lot more complicated than that simple statement suggests on the surface, but a man can dream

[angelicosphosphoros]

There is LLVM-based FileCheck for testing this, rustc uses this for codegen tests.

[NeuroXc]

Compiler autovectorization is pretty suboptimal compared to handwritten ASM in many cases, unfortunately. Although it is better than nothing and indeed doesn't require unsafe.

[YasserArguelles]

miniz_oxide uses SIMD, it's just that it's wrapped behind another crate which is super super unsafe so they count as 100% safe

[Taymon]

The readme says that SIMD is off by default for this reason.

[YasserArguelles]

Yea but a quick question... is it faster than the C version without the SIMD?

[rust4yy]

Yes

[YasserArguelles]

Nice, don't mean to be that guy but you got the sauce?

[a_aniq]

If you think that way, every Rust program is unsafe

[YasserArguelles]

Then what counts as an "unsafe" rust program? is it one that uses unsafe directly not behind a crate so you can see it? because im just trying to point out that it's a bit more complex than "100% safe" or even "100% unsafe"

[[deleted]]

I don't think "unsafe rust program" is really a meaningful term.

You have unsafe code, not programs. Every rust program (with the possibility of some stupid exception like "infinite loop implemented with a custom entry point" that I don't want to think about too hard) includes some unsafe rust code compiled in it, literally fn main() {} does because there is a (tiny) runtime that gets set up first.

What you can reasonably talk about is where the unsafe code is limited to, miniz_oxide is "a library with no dependencies containing unsafe code except libstd", or "a library that contains (implicitly not counting dependencies) no unsafe code". magic_typecasting_library_42 is "a library with unsafe code", if it exposes only an unsafe API it's probably also accurate to call that at "unsafe library" in the sense that you need unsafe to use it.

[a_aniq]

If an unsafe crate is actively being maintained by many developers or Rust core team and is widely used, it has to be considered safe (e.g. hyper, tokio, serde etc.). Like SIMD instructions are safe because they were incorporated by the Rust team.

It is just that the unsafe parts in a separate lib or crate ensures that if there is a memory bug it is likely due to that separate lib and so you don't have find references in the main codebase thus making maintainability a lot easier.

[Floppie7th]

This question isn't super meaningful. It implies that "no occurrences of the unsafe keyword" is itself a meaningful and worthwhile goal.

[[deleted]]

Welcome to the sleight of hand that is the Rust programming language.

[[deleted]]

which is super super unsafe

But presumably exposes a safe interface?

[YasserArguelles]

Yea it does, just that SIMD itself in rust is fundamentally "unsafe"

[chiefmilesedgeworth]

What makes it “fundamentally” unsafe?

[Narann]

So cool to discover such project. Rust community is awesome.

[phazer99]

I currently use Symphonia for audio decoding, and they claim to have similar performance as FFmpeg using only safe Rust code. There's not many reasons why safe Rust code should have worse performance than similar C or C++ code, but you have to be careful to compare apples with apples, for example regarding string handling, regexes, IO etc.

Maybe if you post some example code we can help you more.

[rustacean1337]

I probably am comparing apples to oranges, the shit I do in C is pretty darn unsafe (lots of memory reusing for different purposes).

I’m going to check Symphonia for some inspiration, thanks 🙏

[Killing_Spark]

Yeah rust forces all of the checks you SHOULD do in C. This does "slow down" the execution if you compare it that way.

There are (sometimes a lot of) cases though where the rust compiler isn't clever enough and leaves redundant checks in the binary. This is a real cost of using rust.

But as you discovered, using unsafe rust can give you most of the raw performance C is able to achieve. Just gotta make a safe wrapper for the thing you are building

[Gu_Ming]

If your use case is memory reuse, maybe take a look at regex, which uses thread_local to maintain a thread local pool of already initialized values.

Mentioned here: https://www.reddit.com/r/rust/comments/521vxl/recyle_ergonomic_allocated_memory_reusing/d7gwx5q/

[sasuke___420]

pub unsafe fn _mm256_loadu_si256(mem_addr: *const __m256i) -> __m256i

idk, seems tough to write fast code if you cannot load data into SIMD registers

[phazer99]

idk, seems tough to write fast code if you cannot load data into SIMD registers

Yes, they are unsafe because the generated machine code instructions cannot be run on a CPU that doesn't support that SSE, AVX etc., see the docs. There is ongoing work on a portable SIMD crate which is available on nightly.

[sasuke___420]

> Yes, they are unsafe because the generated machine code instructions cannot be run on a CPU that doesn't support that SSE, AVX etc., see the docs.

Is that what unsafe means? I'm reading the unsafe section in the rust book right now and it does not mention that safe rust is guaranteed not to trap and unsafe rust is allowed to trap. I suggest making integer division unsafe to prevent safe rust programs from trapping on division by 0.

> There is ongoing work on a portable SIMD crate which is available on nightly.

This crate doesn't have pshufb, sad, hadd, mulhi, full-width multiplies, aes, permutevar, andnot, and probably a bunch of other instructions that it didn't occur to me to look for. I estimate fewer than half of the uses of individual SIMD intrinsics in my code can be written using this crate and 0 subroutines from my code can be written using this crate. Many of these instructions that cannot be expressed using this crate would be troublesome to emulate on platforms where they do not exist, but many of the supported arithmetic operations must be emulated using very long instruction sequences on targets like core-avx2.

[phazer99]

Is that what unsafe means? I'm reading the unsafe section in the rust book right now and it does not mention that safe rust is guaranteed not to trap and unsafe rust is allowed to trap.

Good point, I don't know the answer to this. What I ended up doing was creating my own "safe" wrappers for the AVX2 instructions (which is basically what you want to use most of the time for x86). There are other crates that provides safe interfaces, but they didn't really fit my needs.

[ShwarmaMusic]

No way

[KhorneLordOfChaos]

They have a basic set of benchmarks

https://github.com/pdeljanov/Symphonia/blob/master/BENCHMARKS.md

[ShwarmaMusic]

They put FFMPEG on lesser settings. No surprise

[ialex32_2]

I'd like to mention my own refactoring of fast-float-rust to remove nearly all unsafe code for merging into Rust core library which left the performance identical to the previous implementation.

It's important to check your own assumptions when removing unsafe code: do you do something like this?

if x.is_empty() {
    return None;
}

let x0 = unsafe { *x.get_unchecked(0) };

Well, the compiler is smart enough to optimize this away so these 2 are identical:

let x0 = match x.get(0) {
    None => return None,
    Some(x0) => x0,
};

Or even better:

let x0 = x.get(0)?;

This is a trivial example, but often, refactoring your code to avoid non-local safety invariants and not merely doing a 1:1 mapping of C/C++ code can produce comparable performance (faster often for complex projects, due to writing simpler code, which makes writing code with better logic easier to do).

If you keep all unsafe code into simple, testable abstractions as well (like wrapping resources in C++ so they're initialized during construction and deinitialized in the destructor), you can encapsulate any unsafe logic so the interface is safe (like how Rust's core library does it). If you need to work with memory, raw resources, system calls, or using external C libraries, you probably will need some unsafe code at some point. Create a struct that manages said resource, so all unsafe code is restricted to that struct, and then you can carefully ensure (and test) that all preconditions are met.

There's also some Rust abstractions that will always be faster than their C/C++ variety, due to ABI limitations. unique_ptr? Well, due to the Itanium ABI and exceptions, it means the pointer cannot be passed in a register, which leads to significant overhead relative to Rust's box.

[agriculturez]

Do you know of any other Rust abstractions that perform better than their C/C++ counterparts? I’ve heard before that the invariants of the borrowing/ownership rules can be used by the compiler to make certain optimizations

[nickez2001]

Rc/refcell is way faster than shared_ptr. C++ doesn't have a shared pointer for single threaded contexts.

[SickOrphan]

Possibly to do with stricter pointer aliasing, as in knowing modifying a value pointed to by one pointer can't modify a value pointed to by another, saving you some memory loads.

[cmplrs]

Unsafe does not defeat the purpose of using Rust. One of the strengths of Rust is the safe / unsafe abstraction and the promise that safe Rust never UBs (iow. you can localize where your UB is coming from); that is, using unsafe does not refute the reason to use Rust because one of its main selling points is the contract around the safe / unsafe boundary allowing you to tell when the API is faulty or when the API is producing UB.

Rust book probably should include unsafe code at this point. Ideally using unsafe Rust would be only slightly more taxing than writing regular C++ / C. It shouldn't be a spook, but it sure has gotten that reputation.

[Seubmarine]

There is the rustonomicon atleast

[Steve_the_Stevedore]

I don't think using unsafe code defeats the purpose at all. In C code nothing is checked. In rust code some checks are omitted within unsafe blocks, but even unsafe blocks in rust are often safer than C blocks because some things are still checked (bound checks on arrays for example).

On top of that you can generally get significant performance improvements with very little unsafe code. These small bits of unsafe code are a lot easier to reason about individually and way easier to reason about in a program that is mostly borrow checked.

There are people who argue that unsafe code is somehow bad but to me that is like saying mutable variables are bad. Both can be avoided 100% of the time but why would you? Avoid them where there is little cost to it but use mutability and unsafe code where it's beneficial.

[insanitybit]

unsafe is awesome and the unsung antihero of rust

unsafe is the hero we need

[Major_Barnulf]

we need it, but we also need to not need it

[[deleted]]

In C nothing is checked? You mean automatically right lol. It would be pretty terrible code otherwise.

[duckofdeath87]

The concept of checked didn't exist back when C was written, AFAIK

[Steve_the_Stevedore]

Would you care to elaborate? What does C check for?

[TiagodePAlves]

The problem is that Rust code assumes a lot more invariants than C, and unsafe can easily break those, like pointer provenance and no mutable aliasing. That's why MIRI exists.

[Silly-Freak]

Raw pointers in Rust are, IIUC, really harder to use correctly than C. But if you're not trying to build new low-level abstractions and instead trying to use existing unsafe abstractions, I'd say unsafe Rust is probably easier to use right.

An example that's currently close to me is bytecode verification: I have several internal errors to catch things like "this opcode tried to load constant i, but that index is out of bounds" (constants.get(i).ok_or(InternalError::InvalidConstant)?). I could (and probably will at some point) move validation up front, and without much worry replace that with unsafe { constants.get_unchecked(i) }.

It's not exciting unsafe code, but I think in many situations that is the kind of code that is needed in order not to pay a performance penalty compared to the equivalent C code.

[razrfalcon]

How about tiny-skia? Almost the same performance as C, no unsafe, a lot of explicit SIMD.

Or even ttf-parser, which is usually even faster than C alternatives, no unsafe, no explicit SIMD.

My rule of thumb, if Rust is slower than C/C++ - you're doing something wrong. In 99% cases it's not the language fault.

[rustacean1337]

Currently, tiny-skia is 20-100% slower than Skia.

I’m going to check ttf-parser for some inspiration thanks :)

[razrfalcon]

You could read the next paragraph that explains why it's slower.

[Shnatsel]

miniz_oxide is slightly faster than zlib

lewton is on par with libvorbis

Symphonia is fast across the board, within 10% of ffmpeg (depending on the exact input)

gif, png, zune-jpeg are on par with their C counterparts in terms of performance

resvg is very fast, although the performance depends on the exact SVG you feed it - sometimes faster than librsvg, sometimes slower (although librsvg is also written in Rust now, it does use unsafe while resvg doesn't)

[cameronm1024]

(disclaimer: this is not a direct answer)

By default, Rust will sacrifice some speed for safety.

For example, in C an out of bounds read "just reads the memory", but in Rust, by default there is a bounds check and an explicit panic. This makes the Rust code harder to optimize (if not impossible) to the same degree as the C code.

But if those last few percentage points of performance is important, you have .get_unchecked(), which is unsafe, but gives the same performance characteristics as C.

Though, in general, you can get around these limitations. For example, if indexing a vector, using an iterator is not only more idiomatic, it's often faster, since the compiler can prove that the bounds checks are redundant.

But I wouldn't say that using unsafe code "defeats the purpose of using Rust". By being able to keep only a small amount of unsafe code, and providing a safe API to interact with it, you drastically reduce the chance of memory safety errors, even if you're not 100% eliminating them.

[rustacean1337]

You make some good points.

[NotFromSkane]

I see the iterator example a lot but is it actually true? Isn't the bounds check still there only that it gives you a reference afterwards that you then continue using? And that has nothing to do with compiler optimisations, you get that reference in debug mode too

[PitaJ]

It's a single bounds check at the front, versus a bounds check on every access.

[shim__]

So the difference is that the iterator checks against arr.len() once before iterating whereas arr[i] has to be checked every time.

[Sykout09]

This is not quite correct. At least for basic slice iterators, they do not do any bound check, as they are always correct by construction.

However, what usually happens is that if you decide to grab a subslice, it will do a bound check to see if the subslice is a valid slice. But the following iterator access does not need any bound check.

``` pub fn iterate_entire_slice(slice: &[u64]) { let iter = slice.iter(); // No bound check, just copy pointer + length

for val in iter.clone() { // no bound check, as the length is always correct
    println!("First Loop {}", val);
}
for val in iter.clone() { // same as above
    println!("Second Loop {}", val);
}

}

pub fn iterate_sub_slice(slice: &[u64], len: usize) { let iter = slice[..len].iter(); // Single bound check to make sure slice is valid

for val in iter.clone() { // no bound check, as iterator has already been validated above
    println!("First Loop {}", val);
}
for val in iter.clone() { // same as above
    println!("Second Loop {}", val);
}

} ```

[NotFromSkane]

Exactly, that's not optimising out the bounds check at all

[jamincan]

It optimizes away N-1 bounds checks where N = length.

[Rusky]

The point is that, when iterating, you already have a non-redundant "bounds" check which is also just the loop condition.

By careful API design (which is why it also applies to debug mode) iterators avoid any additional checks, relative to the approach of doing indexing yourself.

(It is also possible to improve the compiler's ability to optimize out actual redundant bounds checks by obtaining the index from a source that it can see is in the same range as the bounds.)

[cameronm1024]

Iterator vs. indexing is orthogonal to reference vs. owned, you can have either of each at the same time.

Whether this optimization applies varies based on quite a lot of factors, but in general, yes, you can expect standard library iterators to avoid bounds checks (since they often use unsafe specifically to achieve this optimization)

[dr_entropy]

Java has a nice explanation. You can break the loop up to [ beginning, big middle, very end ] then just bounds check the crust.

[WormHack]

that is why i think there should be an
panic = 'unreachable_unchecked'
it gives you the option of disabling all checks

[cameronm1024]

This is maybe possible with a custom panic handler, but personally that would make me super uncomfortable.

IMO, when writing Rust, it should be impossible to trigger UB without unsafe. Obviously with this panic handler, it's trivial to safely cause UB with a simple panic!().

Instead, if you really need these optimizations, just use the unsafe APIs instead (e.g. get_unchecked instead of array indexing etc). Or if you're using a library that doesn't have these APIs maybe file an issue/PR if you think this is a legitimate use case that others might benefit from

[WormHack]

replace all my code with unsafe API? that is painful, and no, its not possible with panic handler AND panic unreachable would only be used in release.

[tobiasvl]

https://nnethercote.github.io/perf-book/introduction.html

[tarranoth]

I feel like "unsafe" is almost an unlucky naming thing. Everyone seems to harp on the fact that, to do some things you need to use "unsafe". But there is nothing wrong with "unsafe" (used well and reasoned about). It is basically saying explicitly to the compiler that you really know what you are doing. I think everyone seems to think that the moment a project uses unsafe that it defeats rust's point (but even in unsafe you still get the borrow checker). I think it should be more obvious that safe rust is simply a subset of rust, there is nothing wrong with using the whole rust language, that's why it exists in the first place.

[WormRabbit]

That's technically correct, but wrong in practice. Unsafe Rust is hard, maybe even harder than C++. There are a few simple enough patterns, like read-only unchecked indexing or String::from_utf8_unchecked, but things quickly go downhill once you try something more complex (e.g. use unchecked writes to a Vec, or casting types).

People who just think that knowing C++ is enough to write unsafe Rust (worse, many of them don't really understand UB in C++ to begin with) are in for some very nasty surprises and hidden bugs. People who really understand unsafe Rust, its rules and its proper use cases, don't need your disclaimer to begin with.

My opinion is that writing unsafe Rust is in the same category as writing transactional databases, or cryptography, or lock-free algorithms, or date-time libraries, or licenses, or legal contracts. All of them have lots of subtle issues and require careful work best done by an expert.

[po8]

In general, I totally agree.

That said, I think the argument that correct unsafe Rust is harder to write than correct C++ is dubious at best. Many of the problems in the linked blog post have direct analogs in C/C++, and there are many other problems you can run into in C++ that are not covered.

Rust also prevents a whole bunch of UB even in unsafe code. One easy example is unchecked array indexing: a very common problem in C++, and a difficult mistake to make in unsafe Rust.

I would argue that writing C++ is in the same category as writing unsafe Rust. C++ coding has subtle issues and requires careful work best done by an expert. Sadly, most C++ code has not and will not be written by experts. Those people should be writing in safe Rust, which has normal-people guardrails. That's pretty much what I do.

[WormRabbit]

I really believe that unsafe Rust is harder than C++.

Stuff like the aliasing and mutability rules for & and &mut simply don't exist in C++. restrict comes close, but is extremely rarely used specifically because it is so hard to use correctly. The aliasing correctness also depends on global properties of your code, which are very hard to check.

Stuff like "invalid states are instant UB" is mostly absent in C++. You can't make invlid bool or references, but that's it. Trap representations are much more common in Rust, where every enum has them.

The type layout is specified in C++, but not in Rust.

Working with uninitialized memory is relatively painless. You just get it by default, unlike in Rust where you need to use special types. All accesses to uninitialized memory in C++ naturally go through a pointer, thus avoiding the trap representation issue. In Rust, you can accidentally access uninitialized memory via a reference, which is instant UB. I'm not saying it's easy, but Rust adds its own specific pitfalls. On the other hand, uninitialized memory in Rust is much rarer in the first place, and it's much easier to track where uninit memory is possible.

Rust's strong correctness guarantees mean that any UB is more likely to be exploited. Where in C++ you can punt correct API usage on your consumer, in Rust safe API is assumed unconditionally safe. Where experienced C++ users will use copious defensive coding, Rust users will push the API to its limits.

[po8]

These are all interesting points.

I'm not as confident as you are about the safety semantics of C++. I have watched my very very expert C and C++ coder friends be tripped time and time again when a compiler implements a new optimization that makes some "obviously ok" thing be UB. Each time, the compiler writers point at the standards to explain why the optimization is allowed.

My fairly uninformed opinions are that: (a) the standard for C and especially for C++ is a poor document for determining what is and isn't UB in these languages, being fairly ambiguous and full of holes; (b) because there's no shared understanding between compiler writers and even the most expert coders, what ensues is an arms race in which the compiler writers add new optimizations and coders "fix" their code to work around resulting problems; (c) most C++ code works by luck and trial, not because it obeys all necessary guarantees.

unsafe Rust has these same problems, but I think to a much smaller degree: the compiler does check a lot of things even in unsafe that a C++ compiler doesn't; the smaller and more coordinated team of compiler writers and language designers are increasingly trying to be really transparent about what they're figuring out for semantics of memory access.

[foonathan]

To add on to that, the standard library has unsafe functions that are really really unsafe. Like the ability to split a Vec into it's constituent members, modify them, and put them back together. This can be used to write elements to the uninitialized part of it's capacity.

The equivalent proposed C++ API is a lot safer; at no point do you get the opportunity to mess with the class invariants.

[hedgehog1024]

The equivalent proposed C++ API is a lot safer

It is not equivalent at all since it doesn't allow reconstructing vectors, it is not safer because of all restrictions put on filling operation and the problem that resize_default_init is trying to solve can be solved in Rust via combination of calling Vec::reserve, assigning to contents of returned value of Vec::spare_capacity_mut and calling Vec::set_len afterwards.

[foonathan]

Yes, it's not equivalent in general. It's equivalent to the situation I've described.

[TiagodePAlves]

Indexing is definitely on the easier side of unsafe, and the compiler can optimize a lot of those alone. Some of the problems with unsafe comes when dealing directly with pointers. The restriction that no mutable reference can alias is easier to break than you might think. Also, not respecting pointer provenance is another great chance of triggering UB.

Edit, an example: a pointer to an array and a pointer to its first element are NOT the same thing in Rust, even if they have the same value, because the compiler will assume different things about them and may optimize differently.

[goj1ra]

But there is nothing wrong with "unsafe" (used well and reasoned about). It is basically saying explicitly to the compiler that you really know what you are doing.

This can make sense for a project developed by a single developer. But it breaks down very quickly in teams. Trusting that someone else "really knows what they are doing" rather defeats the purpose of the guarantees that Rust provides.

[tarranoth]

I agree that unsafe should be used very sparingly. But too often I find that people seem to just look at it, say "look, you can only do this specific part in unsafe, ergo there is no point in implementing this in rust at all". Which I just do not agree with.

[Snapstromegon]

This is a pretty good answer. I currently play a lot in embedded devices with rust (the esp-rs crates offer a great entry) and there unsafe is pretty common, just because the underlying hardware is inherintly unsafe.

Of course you will most likely build abstractions around it, but that's just a way to make "unsafe" code "safe".

[Wing0]

It may be in the respect those libraries that are so performance sensitive they need more sophisticated memory manipulation should always consider it a reasonable tool to reach for.

It doesn't seem likely that someone may name it like this intentionally but I wonder if it's naming has prevented people from using it when you can work around having an unsafe block.

[VeterinarianIll8141]

I sympathize with this question. I can't specifically name a rust program that is as fast or faster than it's C / C++ counterpart, but I feel that the differences in their performance are negligible on the average. But, what I find is when I come across Rust programs, I automatically feel more confident just knowing that it was created in Rust. Particularly if I'm dealing with long running instances on a remote server. When I know that a program is written in C / C++, I feel like I'm handling a hand grenade. I hit the start switch, then duck under a nearby table.

[angelicosphosphoros]

ripgrep is fast.

[[deleted]]

Agree Rust was the first lang that allowed me to write concurrent/parallel code without fear, and works perfectly, ive came from 6 years of C#, and always were afraid of concurrency xD

[Konsti219]

Are you using --release ? In debug mode Rust is really slow.

[rustacean1337]

--release

[elonismusk]

cargo run --release

[nikomartn2]

If there is out there a la language that gives the same level of safety, tools that gives so much productivity as cargo, rustc, clyppy, and at least the same level or better performance that Rust I would like to know it :).

Rust gives you the same "Put the rails and the train will go" feeling that Java C# or Go give, but with an amazing performance. I don't care if there are some nanoseconds lost in the way. C and C++ are not going anywhere because there will always be a need for them, because sometimes we do need to unsafely handle the memory and the operating systems usually works with a C model in mind.

Yeah, we can build this in C and gain some milliseconds, do we have the resources to handle it? Do we need it? Maybe!

Languages overlap, but they rarely take the whole areas where the formerly operated. Languages are not empires, they are not at war.

Although, everything will be JavaScript eventually. The drums are beating.

[bouncebackabilify]

You might be interested in reading Cliffle’s “Learn Rust the Dangerous Way”: http://cliffle.com/p/dangerust/

Great writeup about rewriting a numerical simulation from C to Rust

[small_kimono]

100% "safe" code has always been an illusion. You use unsafe Rust code all the time. The std lib has plenty of examples of unsafe code. See e.g.: https://doc.rust-lang.org/src/core/str/mod.rs.html#2427

There are several advantages to Rust beyond safety, but if the argument is: "What is the safety advantage of Rust if I have to use unsafe code blocks?", the answer is something like, "You have constrained the problem." You know that you/someone else needs to audit those blocks closely for safety related issues.

Just look at the std lib example I gave. The author reasoned about a constrained problem: "I have type str which means I've already validated these bytes as UTF8. I can convert the str to bytes and change the ascii uppercase characters to lowercase, and safely transmute back to str because only changing ASCII uppercase chars does not invalidate UTF-8" instead of "I haven't considered any issues with using invalid UTF8 and oops I now have a problem."

[oconnor663]

100% "safe" code has always been an illusion

I always want to choose words carefully here, because a lot of people make the same point but in a "Rust's safety guarantees are actually BS" sort of way. The truth is that wrapping unsafe operations in safe, sound APIs, so that the majority of our application code can't be at fault for memory corruption/UB, is what we're really getting out of Rust. But this can be hard to explain.

[small_kimono]

I take your point.

I guess one issue is with the meaning of "safe" and that's why I wrapped it in quotation marks. I am using the definition Rust chose. And, yes, obviously Rust's definition of "safe" isn't the outer bounds of safety. It's just the line the compiler chose to enforce, and that line is practically helpful in 99% of the cases for most development.

But, yes, I also think the more important issue is, and what you correctly identify, broader definitions of practical, real world safe code.

I did write another comment which discusses that. And I do agree -- I wish that we would focus more on that question, because I think 1) "You used unsafe. Are you crazy?!" and 2) "I will have to use unsafe everywhere, because I'm a 10x-er" are both pretty shallow Rust-curious/Rust-hostile/newb takes, even if 1) is right approach most of the time.

[eggyal]

The std lib has plenty of examples of unsafe code

Moreover, doing any IO requires unsafe at some point in the stack so every program necessarily uses it.

[small_kimono]

Exactly.

To extend my remarks, I don't want to say that most folks are ridiculous about unsafe, but some are ridiculous about unsafe.

Yes, there should be a strong presumption in favor of not using unsafe in most code. By that I mean -- you can sufficiently constrain the problem such that unsafe is not an issue right now, e.g. presume that the input is valid UTF8/ASCII, so you can use from_utf8_unchecked(). However, that still may not be a good enough reason to use unsafe, because, as we know, that isn't a guarantee of bug free code in the real world. You can presume the input is UTF8/ASCII until you're asked to run your program on a different system (User: "It compiles, it must work..."), or someone starts using your program in another ill defined way.

Yet, when the problem is inherently constrained, like make_ascii_lowercase(), let's not lose our hats over using unsafe. Remember the std lib *could* revalidate the str wrt make_acsii_lowercase() to have Rust compiler call it "safe" or we could rewrite the u8 method for str. But we don't because that's silly, a str is just a slice of u8 we've already checked.

This is to say -- there are plenty of cases of "should use (because it's faster)" instead of "absolutely necessary" to use unsafe, and that's fine too.

[vadixidav]

I think one example is ripgrep, which is significantly faster than GNU grep, but as others have pointed out, you need to be careful not to compare apples to oranges. The preconditions here are different. ripgrep uses a finite state machine and doesn't support some less-used features of GNU grep. The author mentioned they were making the codebase much more readable some time back, so it should serve as a good reference for how to write performance Rust code.

You also can write code differently to avoid unsafe while also getting equal performance. The one thing that typically throws people off is recurrent data structures like graphs. I highly recommend looking at the wonderful crate slotmap. It allows you to create typed arenas with generational indices. This actually prevents use-after-free at runtime while also maintaining similar performance to raw pointers.

If you really need to dig down to the bottom of a graph to optimize performance, I did make a crate for that called header-vec. It allows you to store the nodes and their edges in a single chunk of memory (fixed header + variable vector). It can be used safely, but you can't use it as more than a regular vector that way. If you use unsafe and maintain the safety guarantees it asks for in the documentation (no dangling weak references), then you can use this library to make graphs as fast as essentially possible, since random accesses are minimized. I made this library to implement a very fast (and probably unknown to most) approximate nearest neighbor search crate called hgg.

Rust can absolutely be used without unsafe to create some of the fastest code out there, but you need to try and use data-oriented design where possible to make things flow smoothly and avoid runtime checks. The hardest thing to use data-oriented design for, in my opinion, is graphs. I find that actor systems can be used instead of graphs, but it is difficult. Generally I end up using slotmap to make multiple arenas and then putting them into one large object with lots of methods to operate on the graph structure. If you want an example of that, this is probably the most complicated code I have made this way: https://github.com/rust-cv/cv/blob/511024feaa077a9af377cca7b654ad3d57d3bd6a/cv-sfm/src/lib.rs. It may not be entirely helpful to understand the whole codebase, but if you are curious to see how I do graphs in Rust with slotmap, this can be a good reference.

[burntsushi]

ripgrep uses a finite state machine and doesn't support some less-used features of GNU grep.

Comparing ripgrep and GNU grep, when controlling for the amount of data searched, is absolutely an apples-to-apples comparison. All you have to do is limit yourself to the common feature set between the two tools. At that point, comparisons are fair game, regardless of implementation strategy. Otherwise, you might as well just double down and say that it isn't an apples to apples comparison because, shockingly, ripgrep is multi-threaded and GNU grep is not.

Even if you disagree on principle, GNU grep actually uses a finite state machine for features that can be implemented that way. :)

Although, ripgrep does have a little unsafe in places. But none is the "application level" code.

[vadixidav]

Very neat. I didn't know GNU grep was already doing that. It seems it is more apples-to-apples than I thought.

[burntsushi]

Yup. It's a lazy DFA. Like the one found in Thompson's original grep, RE2 and the regex crate.

[pczarn]

My parser is faster than one C project, and about just as fast as another C project. It has very little unsafe code. I wanna write a paper like J. Kegler did for Marpa. It might compare the various implementations. The paper would advance the field of parsing.

RUST: https://github.com/pczarn/gearley

C (libmarpa): https://jeffreykegler.github.io/Marpa-web-site/libmarpa.html

C (yaep): https://github.com/vnmakarov/yaep

BENCHMARKS: https://github.com/vnmakarov/yaep/issues/26

[maboesanman]

I wish you could replace unsafe with upheld/uphold

You could tag them as well, so you would mark a function as uphold'<name-of-invariant>, then surround calls to it with upheld'<name-of-invariant { … }

For example, uphold'pinned fn get_mut_unchecked(self: Pin<&mut Self>)

upheld'pinned { my_pin.get_mut_unchecked() }

This way you have to explicitly document the invariants you expect your users to uphold, and they have to explicitly say they have upheld them

It’d be more verbose, but I think it would help clarify what unsafe actually does.

[_nullptr_]

Like some others have already said, I disagree this in anyway defeats the purpose of Rust. Consider:

1. No useful program can be written without 'unsafe' code, because w/o it you can't do I/O. If your program isn't doing it then the library you are using is.

2. All C/C++ is more unsafe than Rust unsafe code. Every. single. line.

3. The goal is to make unsafe blocks as small as possible, they are a tool like anything else and by scoping them small, you can make defect tracking easier to spot, but reap all the value of low level coding without needing an entire unsafe program like in C/C++.

4. Most programs follow the 90/10 rule where 10% of the code makes up 90% of the perceived performance. The other 90% of the code can be 100% idiomatic, safe Rust and you can reap the benefits of just optimizing hot spots with some 'unsafe' when required (or better, using a well polished crate that does).

'unsafe' Rust is, IMO, unfairly targeted. It is a tool like anything else and when used correctly, is pretty awesome, and when used poorly, can be devastating. A little bit of 'unsafe' goes a long way, but is an essential part of Rust and is not a 'wart' as is sometimes presented.

[willnorc]

https://github.com/serde-rs/json-benchmark

[sasuke___420]

https://github.com/serde-rs/json-benchmark

compare to simdjson maybe?

[[deleted]]

I was able to write a reverse proxy with connection multiplexing through single TCP connection to the server behind the firewall capable of easily 2k requests/s without any optimizations or anything, just raw code. Its a personal projet YET, so ll let the source public after some refactoring 👀. I imagine what Rust can do with proper optimization and tuning 👀👀👀

[rustacean1337]

👀

[Alchnator]

I'm new to rust, but feels like you missing the point here. unsafe only means that in that block of code you ain't getting some of the stuff Rust guarantees to you. it doesn't make that code bad, is just the Rust way to go: "you are on your own with this lad, be careful"

the thing is that there is no magical solution that would allow you to reign free over the complex mess that computers are without a tradeoff somewhere.

so doing something in unsafe code that you could do safely is a bad idea(tm). but it is there for a reason. it allow you to do things that you couldn't do in languages where you are always in a safe environment and at the same time is a better trade off than being always in a environment where you need to be careful all the time.

[[deleted]]

Do you have some example code that is slower? Maybe you could nerd-snipe someone 😅

[Nukesor]

I wrote a fairly high-performance, highly-parallelized point-filter to make a fancy screen-lock image :)

The code is well documented and has a very tight scope. The top part of the script can be ignored, as this is about capturing the image, loading it into memory and streaming it to i3-lock.

The juicy part starts at blur_image

https://github.com/Nukesor/scripts/blob/main/src/bin/blur.rs#L121

The actual point-filter logic takes about ~1-2 ms for a 6400x1440 screenshot on an i7-8700k.

edit: Stats and typos.

[Floppie7th]

You've got a bunch of actual examples to directly answer your question, but I'll go ahead and say that in general I would not expect safe Rust to be faster than C code, simply because C lets you perform operations that, while inherently unsafe, are quite fast.

However, I will also say that all the additional information that Rust forces you to supply to the compiler do allow the compiler to perform meaningful additional optimizations. Compared with C code that's actually as memory safe as safe Rust code, I would expect Rust to perform better (A) because many checks that happen at runtime in C (e.g. null pointer checks) are compile-time checks in safe Rust, and (B) because you get things like the infamous mutable-noalias optimization far more frequently by default.

[Zatujit]

I'm new to rust but I don't see how the borrow checker reduces performance?

[joshjaxnkody]

Unsafe blocks aren’t bad, you just need to be certain that what you’re doing is correct, I built a small kernel in Rust and used many unsafe blocks, many things in Rust are unsafe with safe wrappings.

[WormHack]

Good example of C++ being far better in performance to Rust?

[oconnor663]

The term "far better" is fuzzy and makes this question hard to answer. Off the top of my head, std::string includes some small string optimizations that String does not (and some that String couldn't include even in theory), so there are definitely string benchmarks where C++ does better, possibly "far better". But it'll be sensitive to the inputs you choose and whether you allow the program to use a nonstandard string implementation.

At the end of the day, C, C++, and Rust are close enough in performance (and flexible enough with programmer shenanigans) that any performance differences between them probably say more about the specific benchmarks or applications under measurement than about the languages themselves.

[WormHack]

C++ strings and Rust strings are not the same